Cart Inspection for Suspicious Items

• US 20080226129A1
• Filed: 03/12/2008
• Published: 09/18/2008
• Est. Priority Date: 03/12/2007
• Status: Active Grant

• Current Assignee: NCR Voyix Corporation
• Original Assignee: Stoplift, Inc. (NCR Voyix Corporation)
• Inventors: Kundu, Malay, Migdal, Joshua, Srinivasan, Vikram, Farrow, Matthew K., O'Donnell, Brian Frank
• Granted Patent: US 8,146,811 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: A47F 9/045 Handling of baskets or shopping trolleys at check-out counters, e.g. unloading, checking
• CPC Class Codes: A47F 9/046 Arrangement of recording means in or on check-out counters record carriers or readers per se G06K
• CPC Class Codes: G06Q 20/202 Interconnection or interaction of plural electronic cash registers [ECR] or to host computer, e.g. network details, transfer of information from host to ECR or from ECR to ECR
• CPC Class Codes: G06Q 30/0185 Product, service or business identity fraud
• CPC Class Codes: G06Q 30/0609 Buyer or seller confidence or verification
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G07G 1/0036 Checkout procedures
• CPC Class Codes: G07G 3/003 Anti-theft control theft alarms G08B13/00
• CPC Class Codes: G08B 13/196 using television cameras
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19673 Addition of time stamp, i.e. time metadata, to video stream
• CPC Class Codes: G08B 21/0423 detecting deviation from an expected pattern of behaviour or schedule
• CPC Class Codes: G08B 31/00 Predictive alarm systems characterised by extrapolation or other computation using updated historic data
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Methods and apparatus provide for a Cart Inspector to create a suspicion level for a transaction when a video image of the transaction portrays an item(s) left in a shopping cart. Specifically, the Cart Inspector obtains video data associated with a time(s) of interest. The video data originates from a video camera that monitors a transaction area. The Cart Inspector analyzes the video data with respect to target image(s) associated with a transaction in the transaction area during the time(s) of interest. The Cart Inspector creates an indication of a suspicion level for the transaction based on analysis of the target image(s). Creation of a high suspicion level for the transaction indicates that the transaction'"'"'s corresponding video images most likely portray occurrences where the purchase price of an item transported through the transaction area was not included in the total amount paid by the customer.

Claims

1. A method comprising:

obtaining video data associated with at least one time of interest, the video data originating from at least one video camera that monitors a transaction area;

analyzing the video data with respect to at least one target image associated with a transaction in the transaction area during the at least one time of interest, the at least one target image provided in the obtained video data; and

creating an indication of a suspicion level for the transaction based on analysis of the at least one target image.

2. The method as in claim 1, wherein obtaining video data associated with the at least one time of interest includes:

identifying a time stamp in transaction data of the transaction;

defining the at least one time of interest as contemporaneous with the time stamp; and

identifying the at least one target image as included in at least one portion of the video data created during the at least one time of interest.

3. The method as in claim 2, wherein identifying the time stamp includes:

identifying a last time stamp in the transaction data.

4. The method as in claim 1, wherein obtaining video data associated with the at least one time of interest includes:

defining at least one critical location in the transaction area;

defining the at least one time of interest as when a cart is the critical location in the transaction area, the cart comprising a device for transporting items through a retail environment; and

identifying the at least one target image as included in at least one portion of the video data created during the at least one time of interest, the at least one target image thereby portraying activity occurring at the critical location during the at least one time of interest.

5. The method as in claim 4, wherein defining the critical location in the transaction area includes:

defining the critical location as a location in the transaction area suited for differentiating between suspicious activity and non-suspicious activity.

6. The method as in claim 4, wherein defining the critical location in the transaction area includes:

defining the critical location as a location where the cart is in the transaction area and most likely to be empty when it is involved in a non-suspicious transaction.

7. The method as in claim 1, wherein analyzing the video data with respect to at least one target image associated with a transaction includes:

performing a comparison of the at least one target image with at least one reference representation to determine the extent that the at least one target image portrays one of imagery of non-suspicious activity and imagery of suspicious activity.

8. The method as in claim 7, wherein performing the comparison of the at least one target image with the at least one reference representation includes:

defining the at least one reference representation as a template depicting a non-suspicious transaction.

9. The method as in claim 7, wherein performing the comparison of the at least one target image with the at least one reference representation includes:

comparing at least one target image depicting a cart proximate to a point of sale location in the transaction area with the at least one reference representation, the at least one reference representation comprising a template of an empty cart, the cart suitable for transporting items through the transaction area.

10. The method as in claim 7, wherein creating the indication of the suspicion level for the transaction based on analysis of the at least one target image includes:

if the comparison results in detection of a similarity between the at least one target image and the at least one reference representation, setting the indication of the suspicion level to a lowest suspicion level.

11. The method as in claim 7, wherein creating the indication of the suspicion level for the transaction based on analysis of the at least one target image includes:

if the comparison results in detection of a difference between the at least one target image and the at least one reference representation, for each portion of the at least one target image that portrays at least one item transported through the transaction area;

adjusting the suspicion level of the transaction based on at least one of

(i) a location of the at least one item in the cart;

(ii) a size of the at least one item;

(iii) a shape of the at least one item;

(iv) a color of the at least one item;

(v) a characteristic of the at least one item;

(vi) a probability that the item qualifies for a false positive classification.

12. The method as in claim 11, wherein adjusting the suspicion level of the transaction based on the location of the at least one item in the cart includes one of:

minimally increasing the suspicion level when the portion of the at least one target image portrays at least one item in an upper compartment of the cart;

moderately increasing the suspicion level when the portion of the at least one target image portrays at least one item in a basket of the cart; and

substantially increasing the suspicion level when the portion of the at least one target image portrays at least one item in beneath the a basket of the cart.

13. The method as in claim 11, wherein adjusting the suspicion level of the transaction based on the probability of the item qualifying for the false positive classification includes:

detecting an indication of movement over time in the portion of the at least one target image that portrays the item transported through the transaction area; and

upon detecting the indication of movement, creating an indication that the item transported through the transaction area is likely a moving item.

14. The method as in claim 11, wherein adjusting the suspicion level of the transaction based on the probability of the item qualifying for the false positive classification includes:

detecting a distribution of a color in the portion of the at least one target image that portrays the item transported through the transaction area, the color corresponding to a shopping bag used in the transaction area; and

upon detecting the distribution of the color, creating an indication that the item transported through the transaction area is likely a bagged item.

15. The method as in claim 11, wherein adjusting the suspicion level of the transaction based on the probability of the item qualifying for the false positive classification includes:

searching for an occurrence of at least one predefined bulk item the transaction data for information related to the identification of the at least one predefined bulk item; and

upon detecting information in the transaction data that is related to the at least one predefined bulk item, creating an indication that the at least one item transported through the transaction area is likely a bulk item.

16. The method as in claim 1, wherein analyzing the video data with respect to at least one target image associated with a transaction includes:

identifying a segment of the video data that portrays at least one image of a cart, a portion of the at least one image of the cart portraying an item transported by the cart;

obtaining a visual representation for each item listed in transaction data;

comparing the portion of the at least one image of the cart portraying an item transported by the cart with the visual representation for each item listed in transaction data;

wherein creating an indication of a suspicion level includes;

increasing the suspicion level if the portion of the at least one image of the cart portraying an item transported by the cart fails to correlate with any visual representation for each item listed in transaction data; and

decreasing the suspicion level if the portion of the at least one image of the cart portraying an item transported by the cart correlates with at least one of the visual representation for each item listed in transaction data.

17. The method as in claim 1, further comprising:

performing the step of obtaining the video data associated with a second time of interest and performing the step of analyzing the video data with respect to at least one second target image associated with a second transaction;

performing the step of creating the indication of a suspicion level to create a second indication level for the second transaction based on analysis of the at least one second target image;

creating a ranking of the at least one target image and the at least one second target image according to suspicion level.

18. The method as in claim 1, further comprising:

defining a threshold suspicion level;

comparing the indication of the suspicion level for the transaction based on analysis of the at least one target image with the threshold suspicion level;

upon detecting the indication of the suspicion level surpasses the threshold suspicion level, creating a notification associated with the transaction.

19. The method as in claim 18, wherein creating the notification associated with the transaction includes:

creating a real-time notification contemporaneously with the transaction as the transaction occurs in at least one of a self-checkout transaction area and an assisted checkout transaction area.

20. The method as in claim 1, wherein obtaining video data associated with the at least one time of interest, the video data originating from the at least one video camera that monitors a transaction area includes:

obtaining video data originating from at least one video camera elevated to a position situated over a horizontal plane where the transaction occurs in the transaction area, the at least one video camera thereby creating video data that represents at least one of;

(i) an overhead view of a cart transporting at least one item through the transaction area; and

(ii) an elevated perspective view of the cart transporting at least one item through the transaction area.

21. A computer system comprising:

a processor;

a memory unit that stores instructions associated with an application executed by the processor; and

an interconnect coupling the processor and the memory unit, enabling the computer system to execute the application and perform operations of;

obtaining video data associated with at least one time of interest, the video data originating from at least one video camera that monitors a transaction area;

analyzing the video data with respect to at least one target image associated with a transaction in the transaction area during the at least one time of interest, the at least one target image provided in the obtained video data; and

creating an indication of a suspicion level for the transaction based on analysis of the at least one target image.

22. A method comprising:

obtaining video data from at least one video camera that monitors a transaction area;

analyzing a plurality of target images in the video data, the plurality of target images associated with a transaction in the transaction area;

based on analysis of each of the plurality of target images, identifying a portion of the plurality of target images that represent a least suspicious state of the transaction;

creating a minimum suspicion level that represents the least suspicious state of the transaction.

23. The method as in claim 22, wherein identifying the portion of the plurality of target images that represent the least suspicious state of the transaction, includes:

identifying at least one least-suspicious image, the at least one least-suspicious image representing at least one time of interest during the transaction that a cart is least likely to contain unpurchased merchandise items.