Using detailed process information at a point of sale

• US 7,962,365 B2
• Filed: 10/31/2008
• Issued: 06/14/2011
• Est. Priority Date: 10/31/2008
• Status: Active Grant

• Current Assignee: Toshiba Global Commerce Solutions Holdings Corporation (Toshiba Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Pankanti, Sharathchandra Umapathirao, Kjeldsen, Frederik, Zhai, Yun, Fan, Quanfu, Yanagawa, Akira, Bobbitt, Russell Patrick, Hampapur, Arun
• Primary Examiner(s): Zeender; F. Ryan
• Assistant Examiner(s): Ortiz Roman; Denisse
• Application Number:
  US12/262,458
• Publication Number:
  US 20100114623A1
• Time in Patent Office: 956 Days
• Field of Search: 235/375
• US Class Current: 705/16
• CPC Class Codes: G06Q 20/20 Point-of-sale [POS] network systems
• CPC Class Codes: G06Q 20/4016 involving fraud or risk level assessment in transaction processing
• CPC Class Codes: G06Q 40/12 Accounting
• CPC Class Codes: G07G 1/0036 Checkout procedures
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: G08B 13/246 Check out systems combined with EAS, e.g. price information stored on EAS tag anti-theft systems in point of sale systems G07G3/003

Abstract

Techniques for using transactional and visual event information to facilitate loss prevention are provided. The techniques include obtaining video of one or more visual events at a point of sale environment and one or more transaction log entries that correspond to the video, wherein the one or more transaction log entries comprise one or more transactional events, categorizing each event as one of one or more model events, using each categorized event to create a revised transaction log, wherein the revised transaction log comprises a sequence of categorized events, wherein each categorized event is a combination of the one or more transactional events and the one or more visual events, examining the revised transaction log to correct one or more mis-categorizations, if any, and revise one or more model event categories with the one or more corrected mis-categorizations, if any, and using the revised transaction log to facilitate loss prevention.

Claims

1. A method for using transactional and visual event information to facilitate loss prevention, comprising the steps of:

obtaining video of one or more visual events at a point of sale environment and one or more transaction log entries that correspond to the video, wherein the one or more transaction log entries comprise one or more transactional events;

categorizing each event as one of one or more model events, wherein categorizing each event as one of one or more model events comprises using a metric to determine a degree of similarity of each event to the one or more model events;

using each categorized event to create a revised transaction log, wherein the revised transaction log comprises a sequence of categorized events, wherein each categorized event is a combination of the one or more transactional events and the one or more visual events;

examining the revised transaction log to correct one or more mis-categorizations, if any, and revise one or more model event categories with the one or more corrected mis-categorizations, if any; and

using the revised transaction log to facilitate loss prevention.

2. The method of claim 1, further comprising automatically learning one or more statistical models of the one or more model events.

3. The method of claim 1, wherein using the revised transaction log to facilitate loss prevention comprises finding one or more patterns in the revised transaction log.

4. The method of claim 3, wherein the one or more patterns comprises at least one of a fake scan, an item void without a customer present, a manager override without manager, a refund without a customer present, a transaction void without a customer present, and a cash drawer opened between transactions.

5. The method of claim 1, wherein the one or more model events comprise at least one of a person present in cashier area, a barcode scanned, multiple people present in customer area, a transaction voided, a person present in customer area, multiple people present in cashier area, a keyboard interaction, an item bagged, a pick-up motion, a scan motion and a drop motion.

6. The method of claim 1, further comprising monitoring a retail checkout with a video camera.

7. The method of claim 1, further comprising analyzing video of the point of sale environment to detect one or more point of sale events.

8. The method of claim 1, further comprising rating the categorization of each event based on a metric.

9. A computer program product comprising a non-transitory computer readable medium having computer readable program code for using transactional and visual event information to facilitate loss prevention, said computer program product including:

computer readable program code for obtaining video of one or more visual events at a point of sale environment and one or more transaction log entries that correspond to the video, wherein the one or more transaction log entries comprise one or more transactional events;

computer readable program code for categorizing each event as one of one or more model events, wherein categorizing each event as one of one or more model events comprises using a metric to determine a degree of similarity of each event to the one or more model events;

computer readable program code for using each categorized event to create a revised transaction log, wherein the revised transaction log comprises a sequence of categorized events, wherein each categorized event is a combination of the one or more transactional events and the one or more visual events;

computer readable program code for examining the revised transaction log to correct one or more mis-categorizations, if any, and revise one or more model event categories with the one or more corrected mis-categorizations, if any; and

computer readable program code for using the revised transaction log to facilitate loss prevention.

10. The computer program product of claim 9, further comprising computer readable program code for automatically learning one or more statistical models of the one or more model events.

11. The computer program product of claim 9, wherein the computer readable program code for using the revised transaction log to facilitate loss prevention comprises computer readable program code for finding one or more patterns in the revised transaction log.

12. The computer program product of claim 11, wherein the one or more patterns comprises at least one of a fake scan, an item void without a customer present, a manager override without manager, a refund without a customer present, a transaction void without a customer present, and a cash drawer opened between transactions.

13. The computer program product of claim 9, wherein the one or more model events comprise at least one of a person present in cashier area, a barcode scanned, multiple people present in customer area, a transaction voided, a person present in customer area, multiple people present in cashier area, a keyboard interaction, an item bagged, a pick-up motion, a scan motion and a drop motion.

14. The computer program product of claim 9, further comprising computer readable program code for analyzing video of the point of sale environment to detect one or more point of sale events.

15. The computer program product of claim 9, further comprising computer readable program code for rating the categorization of each event based on a metric.

16. A system for using transactional and visual event information to facilitate loss prevention, comprising:

a memory; and

at least one processor coupled to said memory and operative to;

obtain video of one or more visual events at a point of sale environment and one or more transaction log entries that correspond to the video, wherein the one or more transaction log entries comprise one or more transactional events;

categorize each event as one of one or more model events, wherein categorizing each event as one of one or more model events comprises using a metric to determine a degree of similarity of each event to the one or more model events;

use each categorized event to create a revised transaction log, wherein the revised transaction log comprises a sequence of categorized events, wherein each categorized event is a combination of the one or more transactional events and the one or more visual events;

examine the revised transaction log to correct one or more mis-categorizations, if any, and revise one or more model event categories with the one or more corrected mis-categorizations, if any; and

use the revised transaction log to facilitate loss prevention.

17. The system of claim 16, wherein the at least one processor coupled to said memory is further operative to automatically learn one or more statistical models of the one or more model events.

18. The system of claim 16, wherein in the using the revised transaction log to facilitate loss prevention, the at least one processor coupled to said memory is further operative to find one or more patterns in the revised transaction log.

19. The system of claim 18, wherein the one or more patterns comprises at least one of a fake scan, an item void without a customer present, a manager override without manager, a refund without a customer present, a transaction void without a customer present, and a cash drawer opened between transactions.

20. The system of claim 16, wherein the one or more model events comprise at least one of a person present in cashier area, a barcode scanned, multiple people present in customer area, a transaction voided, a person present in customer area, multiple people present in cashier area, a keyboard interaction, an item bagged, a pick-up motion, a scan motion and a drop motion.

21. The system of claim 16, wherein the at least one processor coupled to said memory is further operative to analyze video of the point of sale environment to detect one or more point of sale events.

22. The system of claim 16, wherein the at least one processor coupled to said memory is further operative to rate the categorization of each event based on a metric.

23. An apparatus for using transactional and visual event information to facilitate loss prevention, said apparatus comprising:

means for obtaining video of one or more visual events at a point of sale environment and one or more transaction log entries that correspond to the video, wherein the one or more transaction log entries comprise one or more transactional events;

means for categorizing each event as one of one or more model events, wherein categorizing each event as one of one or more model events comprises using a metric to determine a degree of similarity of each event to the one or more model events;

means for using each categorized event to create a revised transaction log, wherein the revised transaction log comprises a sequence of categorized events, wherein each categorized event is a combination of the one or more transactional events and the one or more visual events;

means for examining the revised transaction log to correct one or more mis-categorizations, if any, and revise one or more model event categories with the one or more corrected mis-categorizations, if any; and

means for using the revised transaction log to facilitate loss prevention.