Event detection based on video metadata

• US 9,269,244 B2
• Filed: 08/24/2007
• Issued: 02/23/2016
• Est. Priority Date: 03/06/2007
• Status: Active Grant

• Current Assignee: Cognyte Technologies Israel Ltd. (Cognyte Software Ltd.)
• Original Assignee: Verint Systems Incorporated
• Inventors: Heier, Kurt, O'dell, Josh, Legendre, Jon, Shoaf, Derek
• Primary Examiner(s): GREENE, JOSEPH L
• Application Number:
  US11/844,759
• Publication Number:
  US 20080218591A1
• Time in Patent Office: 3,105 Days
• Field of Search: 348/150
• US Class Current: 1/1
• CPC Class Codes: G06Q 20/20 Point-of-sale [POS] network systems
• CPC Class Codes: G06Q 30/0185 Product, service or business identity fraud
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G07G 1/14 Systems including one or more distant stations co-operating with a central processing unit data transmission in general H04L; telemetry systems for selectively calling a substation from a main station H04Q9/00
• CPC Class Codes: G07G 3/00 Alarm indicators, e.g. bells
• CPC Class Codes: G07G 3/003 Anti-theft control theft alarms G08B13/00
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream

Abstract

Video data from sources, such as cameras, is analyzed to create metadata descriptive of the events and objects occurring in the video. This metadata, which consists of data about the video, is then analyzed on a transaction basis to determine if a suspicious activity, such as a fraudulent Point of Sale (POS) return event, has occurred in relation to a transaction.

Claims

1. A system for identifying possible fraudulent activity in connection with a transaction which occurred at point-of-sale (POS) device, an employee region of interest and a customer region of interest being associated with the POS device, the system comprising at least one or more programmed processors and one or more storage devices cooperating to:

identify video metadata corresponding to activity which took place in the customer and employee regions of interest during a time period related to the transaction;

based on the examination of the metadata, being configured to;

determine whether a customer was located in the customer region of interest during the time period, the determination including;

(i) identifying a set of objects, if any, in the customer region of interest which meet predetermined criteria; and

(ii) excluding from that set of objects one or more objects as a function of whether they are also located in the employee region of interest;

wherein the excluding includes;

determining a percentage of surface area overlap, of the one or more objects of interest from the set of objects, between the customer region of interest and the employee region of interest and using the percentage information to determine in which region the one or more objects is located.

2. A system according to claim 1, wherein the video metadata includes information descriptive of objects, if any, located in the customer and employee regions of interest during the time period.

3. A system according to claim 2, wherein the video metadata includes information relating to one or more of the position, relative size, and movements of the one or more objects located in the customer and employee regions of interest during the time period.

4. A system according to claim 2, wherein the video metadata includes information relating to at least the position, relative size, and movements of the one or more objects located in the customer and employee regions of interest during the time period.

5. A system according to claim 1, wherein the one or more processors and one or more storage devices also cooperate to post a report of possible fraud at the POS device when a defined set of conditions relating to the set of objects is met.

6. A system according to claim 1, wherein the POS device is a cashier terminal.

7. A system according to claim 1, wherein the one or more processors and one or more storage devices also cooperate to allow the user of the system to select the length of the time period.

8. A system according to claim 1, wherein the video metadata is time stamped and the system identifies the video metadata corresponding to activity which took place in the customer and employee regions of interest during the time period as a function of the time stamps.

9. A system according to claim 1, wherein the determination of whether a customer was located in the customer region of interest during the time period is made, at least in part, by determining whether one or more of the objects were within a predetermined sub region of the customer region of interest.

10. A system according to claim 9, wherein the determination of whether a customer was located in the customer region of interest during the time period is made, at least in part, by determining whether one or more of the objects were within the customer region of interest for at least a predetermined subset of the time period.

11. A system according to claim 1, wherein the determination of whether a customer was located in the customer region of interest during the time period is made, at least in part, by determining whether one or more of the objects were within the customer region of interest for at least a predetermined subset of the time period.

12. A method for identifying possible fraudulent activity in connection with a transaction which occurred at point-of-sale (POS) device, an employee region of interest and a customer region of interest being associated with the POS device, the method being carried out by a system comprising at least one or more programmed processors and one or more storage devices, the method comprising:

the system identifying video metadata corresponding to activity which took place in the customer and employee regions of interest during a time period related to the transaction;

based on the examination of the metadata, being configured to;

the system determining whether a customer was located in the customer region of interest during the time period, the determination including;

(i) identifying a set of objects, if any, in the customer region of interest which meet predetermined criteria; and

(ii) excluding from that set of objects one or more of the objects as a function of whether they are also located in the employee region of interest;

wherein the excluding includes;

determining a percentage of surface area overlap, of the one or more objects of interest from the set of objects, between the customer region of interest and the employee region of interest and using the percentage information to determine in which region the one or more objects is located.

13. A method according to claim 12, wherein the video metadata includes information descriptive of objects, if any, located in the customer and employee regions of interest during the time period.

14. A method according to claim 13, wherein the video metadata includes information relating to one or more of the position, relative size, and movements of the one or more objects located in the customer and employee regions of interest during the time period.

15. A method according to claim 13, wherein the video metadata includes information relating to at least the position, relative size, and movements of the one or more objects located in the customer and employee regions of interest during the time period.

16. A method according to claim 12, wherein the one or more processors and one or more storage devices also cooperate carry out the step of posting a report of possible fraud at the POS device when a defined set of conditions relating to the set of objects is met.

17. A method according to claim 12, wherein the POS device is a cashier terminal.

18. A method according to claim 12, wherein the length of the time period is selected by the user of the system.

19. A method according to claim 12, wherein the video metadata is time stamped and the step of identifying the video metadata corresponding to activity which took place in the customer and employee regions of interest during the time period as a function of the time stamps.

20. A method according to claim 12, wherein the determination of whether a customer was located in the customer region of interest during the time period is made, at least in part, by determining whether one or more of the objects were within a predetermined sub region of the customer region of interest.

21. A method according to claim 20, wherein the determination of whether a customer was located in the customer region of interest during the time period is made, at least in part, by determining whether one or more of the objects were within the customer region of interest for at least a predetermined subset of the time period.

22. A method according to claim 12, wherein the determination of whether a customer was located in the customer region of interest during the time period is made, at least in part, by determining whether one or more of the objects were within the customer region of interest for at least a predetermined subset of the time period.