Event detection based on video metadata

• US 10,068,136 B2
• Filed: 01/15/2016
• Issued: 09/04/2018
• Est. Priority Date: 03/06/2007
• Status: Active Grant

• Current Assignee: Cognyte Technologies Israel Ltd. (Cognyte Software Ltd.)
• Original Assignee: Verint Americas Incorporated (Verint Systems Incorporated)
• Inventors: Heier, Kurt, O'dell, Josh, Legendre, Jon, Shoaf, Derek
• Primary Examiner(s): Greene, Joseph L
• Application Number:
  US14/996,885
• Publication Number:
  US 20160210515A1
• Time in Patent Office: 963 Days
• Field of Search: 348150
• CPC Class Codes: G06Q 20/20 Point-of-sale [POS] network systems
• CPC Class Codes: G06Q 30/0185 Product, service or business identity fraud
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G07G 1/14 Systems including one or more distant stations co-operating with a central processing unit data transmission in general H04L; telemetry systems for selectively calling a substation from a main station H04Q9/00
• CPC Class Codes: G07G 3/00 Alarm indicators, e.g. bells
• CPC Class Codes: G07G 3/003 Anti-theft control theft alarms G08B13/00
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream

Abstract

Video data from sources, such as cameras, is analyzed to create metadata descriptive of the events and objects occurring in the video. This metadata, which consists of data about the video, is then analyzed on a transaction basis to determine if a suspicious activity, such as a fraudulent Point of Sale (POS) return event, has occurred in relation to a transaction.

Claims

1. A system for analyzing video metadata relating to one or more transactions at a point-of-sale (POS) device, the system comprising one or more processors and one or more storage devices cooperating to:

receive a feed of video metadata containing information about a video stream corresponding to video events occurring at a customer POS region and video events occurring at a cashier POS region;

select and execute one or more of a plurality of metadata analysis techniques on the video metadata to identify the presence or absence of a customer at the customer POS region during a time window corresponding to a return transaction;

wherein the one or more selected and executed metadata analysis techniques include at least an inclusion technique that, for a particular transaction, returns an indication of no fraud if an object present in the video metadata, during the time window associated with the return transaction, is determined to overlap with the customer POS region by more than a predetermined percentage of the customer POS region.

2. A system according to claim 1, wherein the metadata includes information descriptive of objects and events in the video at the POS.

3. A system according to claim 2, wherein the metadata includes information relating to one or more of position, relative size, and movements of an object detected in the video.

4. A system according to claim 3, wherein a detection of the absence of a customer at the POS location associated with the customer is an indication of fraud, the system further comprising a poster application that posts a report of fraud to the POS location upon a detection of fraud.

5. A system according to claim 4, wherein the one or more selected and executed metadata analysis techniques further include a mutual exclusion technique that, for a particular transaction, returns an indication of fraud if an object present in the cashier POS region overlaps the customer POS region by more than a predetermined percentage.

6. A system according to claim 5, wherein objects associated with transactions from a first return lane are analyzed using the inclusion technique, and objects associated with transactions associated with a second return lane are analyzed using the mutual exclusion technique.

7. A system according to claim 5, wherein objects associated with transactions recorded from a first camera are analyzed using the inclusion technique, and objects associated with transactions recorded from a second camera are analyzed using the mutual exclusion technique.

8. A method for analyzing video metadata relating to one or more transactions at a point-of-sale (POS) device via a system comprising one or more processors and one or more storage devices, the method comprising:

receiving a feed of video metadata containing information about a video stream corresponding to video events occurring at a customer POS region and video events occurring at a cashier POS region;

selecting and executing one or more of a plurality of metadata analysis techniques on the video metadata to identify the presence or absence of a customer at the customer POS region during a time window corresponding to a return transaction;

wherein the one or more selected and executed metadata analysis techniques include at least an inclusion technique that, for a particular transaction, returns an indication of no fraud if an object present in the video metadata, during the time window associated with the return transaction, is determined to overlap with the customer POS region by more than a predetermined percentage of the customer POS region.

9. A method according to claim 8, wherein the metadata includes information descriptive of objects and events in the video at the POS.

10. A method according to claim 9, wherein the metadata includes information relating to one or more of position, relative size, and movements of an object detected in the video.

11. A method according to claim 10, wherein a detection of the absence of a customer at the POS location associated with the customer is an indication of fraud, the system further comprising a poster application that posts a report of fraud to the POS location upon a detection of fraud.

12. A method according to claim 11, wherein the one or more selected and executed metadata analysis techniques further include:

a mutual exclusion technique that, for a particular transaction, returns an indication of fraud if an object present in the cashier POS region overlaps the customer POS region by more than a predetermined percentage.

13. A method according to claim 12, wherein objects associated with transactions from a first return lane are analyzed using the inclusion technique, and objects associated with transactions associated with a second return lane are analyzed using the mutual exclusion technique.

14. A method according to claim 12, wherein objects associated with transactions recorded from a first camera are analyzed using the inclusion technique, and objects associated with transactions recorded from a second camera are analyzed using the mutual exclusion technique.