Verification of fraudulent activities at a self-checkout terminal

• US 9,564,031 B2
• Filed: 12/23/2013
• Issued: 02/07/2017
• Est. Priority Date: 12/21/2012
• Status: Active Grant

• Current Assignee: NCR Voyix Corporation
• Original Assignee: Joshua Migdal, Malay Kundu, Matthew Farrow
• Inventors: Migdal, Joshua, Farrow, Matthew, Kundu, Malay
• Primary Examiner(s): Kim, Ahshik
• Application Number:
  US14/139,731
• Publication Number:
  US 20140176719A1
• Time in Patent Office: 1,142 Days
• Field of Search: 235/383, 235/385, 705/64, 186/61
• US Class Current: 1/1
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/48 Matching video sequences
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G07G 1/0054 with control of supplementary check-parameters, e.g. weight or number of articles
• CPC Class Codes: G07G 1/0072 with means for detecting the weight of the article of which the code is read, for the verification of the registration weighing combined with recording means, e.g. in weighing scale, G01G19/415
• CPC Class Codes: G07G 3/003 Anti-theft control theft alarms G08B13/00
• CPC Class Codes: G08B 13/196 using television cameras
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound
• CPC Class Codes: G08B 29/188 Data fusion; cooperative systems, e.g. voting among different detectors
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals

Abstract

A verification method of a fraudulent activity taking place at a self-checkout terminal is disclosed. The method verifies the fraudulent activity by confirming an incident of the fraudulent activity with multiple data generated by monitoring the transaction area and the bagging area of the self-checkout terminal. A human validation is optionally performed to verify machine-identified incidents.

Claims

1. A method of verifying a possible activity of interest at a self-checkout terminal, the method operated via a computer processing unit, comprising the steps of:

obtaining a video feed generated from a video source wherein the video feed is generated by monitoring an item during at least a portion of a transaction time, the item being located at a transaction area;

analyzing the video feed to identify the possible activity of interest;

modifying the video feed wherein the video feed is simplified;

transmitting the modified video feed to a human validation terminal for a human validation of the identified possible activity of interest;

receiving a human validation response resulting from the human validation; and

verifying the identified possible activity of interest based on the human validation response.

2. The method of claim 1 wherein the step of modifying the video feed comprises at least one of:

highlighting the video feed;

taking a snapshot from the video feed;

clipping the video feed into a short-duration snippet; and

applying affine transform to the video feed.

3. The method of claim 1 further comprising the step of obtaining a data feed from a data source, the data feed being generated by monitoring the item during at least a portion of the transaction time, and analyzing the data feed to identify the possible activity of interest, wherein the data feed comprises at least one of:

the data feed from a transaction data wherein the transaction data is generated from the self-checkout terminal as the item is scanned at the self-checkout terminal; and

the data feed from a weight sensor wherein the weight sensor detects a weight of the item.

4. The method of claim 3 wherein the step of analyzing the data feed to identify the possible fraudulent activity by obtaining the data feed from a weight sensor comprises

identifying the possible fraudulent activity based on the weight of the item, wherein the weight is obtained to identify whether the weight from the weight sensor corresponds with an expected weight tolerance.

5. The method of claim 1 further comprising the step of issuing an alert if the identified possible activity of interest is verified by the human validation response.

6. The method of claim 3 further comprising the step of issuing an alert if a combined analysis of the identified possible activity of interest based on the data feed, the video feed, and the human validation verifies the identified possible activity of interest.

7. A method of verifying a possible fraudulent activity at a self-checkout terminal, the method operated via a computer processing unit, comprising the steps of:

receiving a data feed of an item from the self-checkout terminal, wherein the data feed is generated from a data source monitoring the item at a transaction area, the data feed being generated during at least a portion of a transaction time, the data source communicatively coupled with the self-checkout terminal; and

identifying the possible fraudulent activity based on the data feed;

modifying a video feed wherein the video feed is simplified, the video feed being generated from a video source monitoring the item during at least a portion of the transaction time;

transmitting the modified video feed to a human validation terminal for a human validation of the identified possible fraudulent activity; and

receiving a human validation response from the human validation terminal to verify the possible fraudulent activity.

8. The method of claim 7 wherein the step of receiving a human validation response from the human validation terminal to verify the possible fraudulent activity comprises correlating the identified possible fraudulent activity based on the data feed with the human validation response to verify the possible fraudulent activity.

9. The method of claim 7 wherein the step of modifying the video feed comprises at least one of:

highlighting the video feed;

taking a snapshot from the video feed;

clipping the video feed into a short-duration snippet; and

applying affine transform to the video feed.

10. The method of claim 7 wherein the step of identifying the possible fraudulent activity based on a data feed comprises at least one of:

identifying the possible fraudulent activity based on a video analysis, the video analysis being performed by the computer processing unit analyzing the video feed

identifying the possible fraudulent activity based on a transaction data wherein the transaction data is generated from the self-checkout terminal as the item is scanned at the self-checkout terminal; and

identifying the possible fraudulent activity based on a weight of the item, the weight being detected by a weight sensor in communication with the self-checkout terminal.

11. The method of claim 7 further comprising the step of:

issuing an alert if a combined analysis of both the identified possible fraudulent activity based on the data feed and the human validation confirm the identified possible fraudulent activity.

12. The method of claim 7 further comprising the step of:

denying an alert if at least one of the identified possible fraudulent activity based on the data feed and the human validation does not confirm the identified possible fraudulent activity.

13. The method of claim 10 wherein the step of identifying the possible fraudulent activity based on a weight of the item comprises identifying whether the weight from the weight sensor corresponds with an expected weight tolerance.

14. A system for verifying a possible fraudulent activity at a self-checkout terminal comprising:

a computer processing unit operable to analyze a data feed generated during at least a portion of a transaction time, the data feed being generated with a data source monitoring an item being transacted at the self-checkout terminal, the computer processing unit being in communication with the data source;

a video source communicatively coupled with the computer processing unit, the video source operable to generate a video feed of the item by monitoring the item during at least a portion of the transaction time; and

the computer processing unit being configured to;

identify the possible fraudulent activity based on the data feed;

modify the video feed wherein the video feed is simplified;

transmit the modified video feed to a human validation terminal for a human validation of the identified possible fraudulent activity; and

receive a human validation response from the human validation terminal to verify the possible fraudulent activity.

15. The system of claim 14 wherein the computer processing unit is further configured to correlate the identified possible fraudulent activity based on the data feed with the human validation response to verify the possible fraudulent activity.

16. The system of claim 14 wherein the computer processing unit is further configured to issue an alert if a combined analysis of both the identified possible fraudulent activity based on the data feed and the human validation confirm the identified possible fraudulent activity.

17. The system of claim 14 wherein the computer processing unit is further configured to deny an alert if at least one of the identified possible fraudulent activity based on the data feed and the human validation does not confirm the identified possible fraudulent activity.

18. The system of claim 14 wherein the step of modifying the video feed comprises at least one of:

highlighting the video feed;

taking a snapshot from the video feed;

clipping the video feed into a short-duration snippet; and

applying affine transform to the video feed.

19. The system of claim 14 wherein the data feed comprises at least one of:

a video analysis obtained by the computer processing unit analyzing the video feed;

a transaction data, wherein the transaction data is generated from the self-checkout terminal as the item is scanned at the self-checkout terminal, the data source being the self-checkout terminal; and

a weight of the item, the weight being detected by a weight sensor in communication with the computer processing unit, the data source being the weight sensor.

20. The system of claim 19 wherein the computer processing unit identifies the possible fraudulent activity based on the weight of the item by identifying whether the weight from the weight sensor corresponds with an expected weight tolerance.