Location-aware event detection

• US 8,638,380 B2
• Filed: 05/04/2012
• Issued: 01/28/2014
• Est. Priority Date: 11/29/2008
• Status: Active Grant

• Current Assignee: Toshiba Global Commerce Solutions Holdings Corporation (Toshiba Corporation)
• Original Assignee: Solutions Holdings Pty Limited (Solutions Investments Pty Limited), Toshiba Global Commerce
• Inventors: Bobbitt, Russell Patrick, Fan, Quanfu, Hampapur, Arun, Kjeldsen, Frederik, Pankanti, Sharathchandra Umapathirao, Yanagawa, Akira, Zhai, Yun
• Primary Examiner(s): NANO, SARGON N
• Application Number:
  US13/464,328
• Publication Number:
  US 20120218414A1
• Time in Patent Office: 634 Days
• Field of Search: 37524001-24029, 348 36- 39, 348 42- 60, 348/150, 348/157, 348/159, 348/169, 348/274
• US Class Current: 348/274
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

Techniques for detecting one or more events are provided. The techniques include using multiple overlapping regions of interest on a video sequence to cover a location for one or more events, wherein each event is associated with at least one of the multiple overlapping regions of interest, applying multiple-instance learning to the video sequence to select one or more of the multiple overlapping regions of interest to construct one or more location-aware event models, and applying the models to the video sequence to detect the one or more events and to determine the one or more regions of interest that are associated with the one or more events.

Claims

1. A method for detecting one or more events, comprising:

using multiple overlapping regions of interest on a video sequence to cover a respective location for one or more events at a point of sale, wherein each event is associated with at least one of the multiple overlapping regions of interest;

applying multiple-instance learning to the video sequence to select one or more of the multiple overlapping regions of interest to construct one or more location-aware event models, wherein applying multiple-instance learning comprises using a learning technique to learn the one or more location-aware event models for events including at least a pickup and a drop; and

applying the models to the video sequence to detect the one or more events and to determine the one or more regions of interest that are associated with the one or more events.

2. The method of claim 1, wherein the one or more events at the point of sale comprise the pickup, a scan and the drop, wherein a pickup comprises a cashier picking up an item, a scan comprises a cashier at least one of reading the barcode on an item via a scanner and weighing an item, and a drop comprises a cashier placing an item onto a take-away belt area.

3. The method of claim 2, wherein the learning technique comprises a support vector machine- (SVM)-based multiple-instance learning technique.

4. The method of claim 1, wherein the multiple overlapping regions of interest comprise one or more shapes.

5. The method of claim 1, wherein the multiple overlapping regions of interest comprise one or more sizes.

6. The method of claim 1, further comprising extracting one or more features from each region of interest.

7. The method of claim 6, wherein the one or more features comprise at least one of color, edge and motion.

8. A computer program product comprising a tangible computer readable recordable storage medium having computer readable program code for detecting one or more events, said computer program product including:

computer readable program code for using multiple overlapping regions of interest on a video sequence to cover a respective location for one or more events at a point of sale, wherein each event is associated with at least one of the multiple overlapping regions of interest;

computer readable program code for applying multiple-instance learning to the video sequence to select one or more of the multiple overlapping regions of interest to construct one or more location-aware event models, wherein applying multiple-instance learning comprises using a learning technique to learn the one or more location-aware event models for events including at least a pickup and a drop; and

computer readable program code for applying the models to the video sequence to detect the one or more events and to determine the one or more regions of interest that are associated with the one or more events.

9. The computer program product of claim 8, wherein the one or more events at the point of sale comprise the pickup, a scan and the drop, wherein a pickup comprises a cashier picking up an item, a scan comprises a cashier at least one of reading the barcode on an item via scanner and weighing an item, and a drop comprises a cashier placing an item onto a take-away belt area.

10. The computer program product of claim 8, wherein the learning technique comprises a support vector machine- (SVM)-based multiple-instance learning technique.

11. The computer program product of claim 8, further comprising computer readable program code for extracting one or more features from each region of interest.

12. The computer program product of claim 11, wherein the one or more features comprise at least one of color, edge and motion.

13. The computer program product of claim 8, wherein the multiple overlapping regions of interest comprise one or more shapes and one or more sizes.

14. A system for detecting one or more events, comprising:

a memory; and

at least one processor coupled to said memory and operative to;

use multiple overlapping regions of interest on a video sequence to cover a respective location for one or more events at a point of sale, wherein each event is associated with at least one of the multiple overlapping regions of interest;

apply multiple-instance learning to the video sequence to select one or more of the multiple overlapping regions of interest to construct one or more location-aware event models, wherein applying multiple-instance learning comprises using a learning technique to learn the one or more location-aware event models for events including at least a pickup and a drop; and

apply the models to the video sequence to detect the one or more events and to determine the one or more regions of interest that are associated with the one or more events.

15. The system of claim 14, wherein the one or more events at the point of sale comprise the pickup, a scan and the drop, wherein a pickup comprises a cashier picking up an item, a scan comprises a cashier at least one of reading the barcode on an item via a scanner and weighing an item, and a drop comprises a cashier placing an item onto a take-away belt area.

16. The system of claim 14, wherein the learning technique comprises a support vector machine- (SVM)-based multiple-instance learning technique.

17. The system of claim 14, wherein the at least one processor coupled to said memory is further operative to extract one or more features from each region of interest.