Video analytics system for automated teller machine

• US 10,313,635 B2
• Filed: 05/12/2011
• Issued: 06/04/2019
• Est. Priority Date: 05/12/2011
• Status: Active Grant

• Current Assignee: Solink Corporation
• Original Assignee: Solink Corporation
• Inventors: Paliga, Andrezj, Matta, Michael, Glick, David, Pinard, Debbie
• Primary Examiner(s): Patel, Jayanti K
• Assistant Examiner(s): Habib, Irfan
• Application Number:
  US14/117,108
• Publication Number:
  US 20140232863A1
• Time in Patent Office: 2,945 Days
• CPC Class Codes: G07F 19/207 Surveillance aspects at ATMs
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: H04N 21/23109 by placing content in organized collections, e.g. EPG data repository details of retrieval of video data and associated meta data in video databases G06F16/739
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/278 Content descriptor database or directory service for end-user access details of content or meta data based information retrieval of video data in video databases G06F16/739
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4431 characterized by the use of Application Program Interface [API] libraries
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A video analytics system includes a first database for storing searchable time-stamped transactional data indicative of activity within a monitored system, a second database for storing time-stamped video metadata, wherein the time-stamped video metadata comprises searchable attributes associated with a raw video data stream; and a rule-based correlation server for comparing the time-stamped transactional data with the time-stamped video metadata to identify correlation events indicating potential activity of interest. An output subsystem reports the correlation events from the correlation engine. The analytics system is useful for detecting fraud in ATM transactions by comparing the transactional data, for example, the presence of a transaction, with video metadata, for example, indicating whether a transaction occurs when a person is present, for how long the person is there.

Claims

1. An analytics system comprising:

a first database for storing searchable time-stamped system metadata relating a monitored system;

a second database for storing time-stamped video metadata relating to a predetermined region associated with the monitored system;

a correlation server for processing the time-stamped system metadata and the time-stamped video metadata to identify correlation events indicating potential activity of interest in dependence upon a predetermined set of configurable rules, each configurable rule relating to detection of a predefined activity; and

an output subsystem for storing within a third database any correlation events identified by the correlation server, wherein the output subsystem generates for each correlation event a web page comprising information relating to the correlation event and a message for transmittal to a predetermined user comprising at least a link to the web page;

wherein

the predetermined user is established in dependence upon the identified correlation event and an aspect of the identified correlation event;

the correlation server processes the time-stamped video metadata as searchable attributes associated with a raw video data stream which have been generated by analyzing the raw video stream in accordance with a stored set of rules;

the output subsystem stores in association with a correlation event one or more corresponding video frames or corresponding raw video is stored with a correlation event;

information relating to the correlation event within the generated webpage includes the one or more corresponding video frames or corresponding raw video stored with the correlation event;

the time-stamped system metadata relating to the monitored system for storage within the first database is provided by a first time-sensitive data server at a first remote location; and

the time-stamped video metadata relating to a predetermined region associated with the monitored system for storage within the second database is provided from a second time-sensitive data server at a second remote location.

2. The analytics system according to claim 1, wherein

the correlation server at least one of:

extracts from the time-stamped video metadata data indicative of the presence of a person at a monitored location;

processes the time-stamped video metadata data to determine whether an instance of a person being present at the monitored location has occurred;

performs a search backwards in time from a time-stamp associated with the presence of a person to establish an approximate arrival time of the person within the region monitored;

performs a search forwards in time from a time-stamp associated with the presence of a person to establish an approximate departure time of the person within the region monitored; and

requests additional at least one of time-stamped system metadata and time-stamped video metadata in response to an indeterminate correlation event.

3. The analytics system according to claim 1, wherein

the correlation server exploits a plurality of correlation modules, each correlation module for establishing a correlation event selected from the group comprising a financial instrument skimming installation, a financial instrument harvesting activity, a transportation capacity failure, presence of a crime perpetrator, and a physical location determination.

4. A method comprising:

providing a trigger generator for generating a trigger in dependence upon a physical event relating to a monitored system;

generating first searchable time-sensitive metadata relating to the monitored system in dependence upon the trigger from the trigger generator;

generating second searchable time-sensitive metadata relating to multimedia image content generated by monitoring a predetermined region associated with the monitored system in dependence upon the trigger from the trigger generator;

determining with a correlation server in dependence upon a predetermined set of configurable rules whether a correlation event exists by processing the first searchable time-sensitive metadata and second searchable time-sensitive metadata for processing the time-stamped system metadata to identify correlation events indicating potential activity of interest;

storing upon determination of a correlation event within a database correlation data, the correlation data comprising at least a predetermined portion of the first searchable time-sensitive metadata and a predetermined portion of the second searchable time-sensitive metadata;

generating for each correlation event a web page comprising information relating to the correlation event; and

transmitting to a predetermined user a message comprising at least a link to the web page;

wherein

the predetermined user is established in dependence upon the identified correlation event and an aspect of the identified correlation event; and

each configurable rule within the predetermined set of configurable rules relates to detection of a predefined activity;

the correlation server processes the second searchable time-sensitive metadata as searchable attributes associated with a raw multimedia data stream which have been generated by analyzing the raw multimedia stream in accordance with a stored set of rules relating to at least one of the potential activity of interest and the trigger generator;

the output subsystem stores in association with a correlation event one or more corresponding video frames or corresponding raw video is stored with a correlation event;

information relating to the correlation event within the generated webpage includes the one or more corresponding video frames or corresponding raw video stored with the correlation event; and

the first searchable time-sensitive metadata relates to a system within a predetermined location accessible by a user and the second searchable time-sensitive metadata relates to multimedia content acquired within a predetermined region associated with the predetermined location.

5. The method according to claim 4, wherein

the correlation server at least one of:

extracts data indicative of the presence of a person at a monitored location from at least a predetermined portion of the first searchable time-sensitive metadata and a predetermined portion of the second searchable time-sensitive metadata;

processes the at least a predetermined portion of the first searchable time-sensitive metadata and a predetermined portion of the second searchable time-sensitive metadata to determine whether an instance of a person being present at the monitored location has occurred;

performs a search backwards in time from a time-stamp associated with the presence of a person to establish an approximate arrival time of the person within the region monitored;

performs a search forwards in time from a time-stamp associated with the presence of a person to establish an approximate departure time of the person within the region monitored; and

requests additional at least one of the at least a predetermined portion of the first searchable time-sensitive metadata and a predetermined portion of the second searchable time-sensitive metadata in response to an indeterminate correlation event.

6. The method according to claim 4, wherein

the correlation server exploits a plurality of correlation modules, each correlation module for establishing a correlation event selected from the group comprising a financial instrument skimming installation, a financial instrument harvesting activity, a transportation capacity failure, presence of a crime perpetrator, and a physical location determination.

7. The method according to claim 4, wherein

the correlation event is determined to occur based upon at least one of:

the processing of the second searchable time-sensitive metadata absent a trigger from the trigger generator; and

processing of the second searchable time-sensitive metadata together with other second searchable time-sensitive metadata from other monitored systems absent triggers from a predetermined portion of the trigger generators associated with the other monitored systems.

8. A correlation server comprising:

a first interface for receiving first searchable time-sensitive metadata relating to the monitored system;

a second interface for receiving second searchable time-sensitive metadata relating to multimedia image content generated by monitoring a predetermined region associated with the monitored system;

one or more rule-based correlation modules for processing the first searchable time-sensitive metadata and second searchable time-sensitive metadata to identify correlation events indicating a potential activity of interest, each rule-based correlation module applying a predetermined set of configurable rules; and

an output interface for generating for each identified correlation event a web page comprising information relating to the correlation event and a message for transmittal to a predetermined user comprising at least a link to the web page;

wherein

the predetermined user is established in dependence upon the identified correlation event by rule-based correlation module and an aspect of the identified correlation; and

each configurable rule within the predetermined set of configurable rules relates to detection of a predefined activity;

the one or more rule-based correlation modules process the time-stamped video metadata as searchable attributes associated with a raw video data stream which have been generated by analyzing the raw video stream in accordance with a stored set of rules;

the output interface stores in association with a correlation event one or more corresponding video frames or corresponding raw video is stored with a correlation event;

information relating to the correlation event within the generated webpage includes the one or more corresponding video frames or corresponding raw video stored with the correlation event;

the time-stamped system metadata relating to the monitored system for storage within the first database is provided by a first time-sensitive data server at a first remote location; and

the time-stamped video metadata relating to a predetermined region associated with the monitored system for storage within the second database is provided from a second time-sensitive data server at a second remote location.

9. The correlation server according to claim 8, wherein

the one or more correlation modules relate to establishing a correlation event selected from the group comprising a financial instrument skimming installation, a financial instrument harvesting activity, a transportation capacity failure, a presence of a crime perpetrator, and a physical location determination.

10. The correlation server according to claim 8, wherein

the correlation event is determined to occur based upon at least one of:

the processing of the second searchable time-sensitive metadata absent a trigger from the trigger generator; and

processing of the second searchable time-sensitive metadata together with other second searchable time-sensitive metadata from other monitored systems absent triggers from a predetermined portion of the trigger generators associated with the other monitored systems.

11. A method comprising:

applying a plurality of rule-based correlation modules to identify correlation events in dependence upon a predetermined set of configurable rules indicating a potential activity of interest relating to the rule-based correlation module, each rule-based correlation module processing a first predetermined portion of at least one of a first searchable time-sensitive metadata and a second searchable time-sensitive metadata;

generating for each identified correlation event a web page comprising information relating to the identified correlation event; and

transmitting a message for transmittal to a predetermined user comprising at least a link to the web page;

wherein

the predetermined user is established in dependence upon the identified correlation event and an aspect of the identified correlation event; and

each configurable rule within the predetermined set of configurable rules relates to detection of a predefined activity;

the plurality of rule-based correlation modules process the time-stamped video metadata as searchable attributes associated with a raw video data stream which have been generated by analyzing the raw video stream in accordance with a stored set of rules;

an output subsystem stores in association with a correlation event one or more corresponding video frames or corresponding raw video is stored with a correlation event;

information relating to the correlation event within the generated webpage includes the one or more corresponding video frames or corresponding raw video stored with the correlation event;

the time-stamped system metadata relating to the monitored system for storage within the first database is provided by a first time-sensitive data server at a first remote location; and

the time-stamped video metadata relating to a predetermined region associated with the monitored system for storage within the second database is provided from a second time-sensitive data server at a second remote location.

12. The method according to claim 11, wherein

the first predetermined portion of at least one of a first searchable time-sensitive metadata and a second searchable time-sensitive metadata were generated by a process comprising:

generating at each location of a plurality of locations first time-sensitive data, the first time-sensitive data relating to a monitored system at that location of the plurality of locations;

generating at each location of the plurality of locations second time-sensitive data relating to multimedia image content, the multimedia content generated by monitoring a predetermined region associated with the monitored system at that location of the plurality of locations;

processing the first time-sensitive data to generate first searchable time-sensitive metadata in dependence upon the monitored system at a particular location of the plurality of locations;

processing the second time-sensitive data to generate the second searchable time-sensitive metadata.

13. The method according to claim 11, wherein

the first searchable time-sensitive metadata and second searchable time-sensitive metadata were previously generated and archived based upon absence of a correlation event being identified by the previous application of another rule-based correlation module.