People searches by multisensor event correlation

• US 20070294207A1
• Filed: 06/16/2006
• Published: 12/20/2007
• Est. Priority Date: 06/16/2006
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Lisa Marie, Hampapur, Arun, Lu, Zuoxuan, Pankanti, Sharathchandra U., Senior, Andrew William, Tian, Ying-Li, Shu, Chiao-fe
• Granted Patent: US 10,078,693 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G07G 1/0036 Checkout procedures

Abstract

A data indexing system and method includes acquiring activity data in a context and indexing the activity data in accordance with contextual conditions. The activity data is stored in accordance with indices. An event is correlated with the activity data by using the indices to review the activity data in the context.

Claims

1. A data indexing method, comprising:

acquiring activity data in a context;

indexing the activity data in accordance with contextual conditions;

storing the activity data in accordance with indices; and

correlating an event with the activity data by using the indices to review the activity data in the context.

2. The method as recited in claim 1, wherein acquiring activity data includes employing at least one of sensors and cameras.

3. The method as recited in claim 1, wherein the activity data include activities of an individual and the context includes a retail establishment, the method further comprising:

associating separate observations of the individual using the indices.

4. The method as recited in claim 3, wherein associating separate observations is performed while the individual remains in the retail establishment.

5. The method as recited in claim 1, further comprising enabling searches of a database storing the indices and conducting the searches to prevent one of returns fraud by an individual and insurance claim fraud.

6. The method as recited in claim 1, wherein the indices include one or more of:

facial appearance, speed, time of observation, color, anthropometric measurements, gait, biometric identifiers.

7. The method as recited in claim 1, wherein the indices include data from one or more of an RFID tag, a pressure pad, a motion sensor, a beam breaker, a local positioning system.

8. The method as recited in claim 1, further comprising tracking an individual in the context to provide a history of the individual'"'"'s activities within the context.

9. The method as recited in claim 1, further comprising archiving the activity data in accordance with the indices.

10. A computer program product for data indexing comprising a computer useable medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to perform the steps of:

acquiring activity data in a context;

indexing the activity data in accordance with contextual conditions;

storing the activity data in accordance with indices; and

correlating an event with the activity data by using the indices to review the activity data in the context.

11. A data indexing system, comprising:

a plurality of acquisition devices configured to acquire activity data in a context;

a surveillance engine configured to process the activity data and derive search indices for the activity data;

an index association engine to correlate different observations using the indices;

a user-interface configured to conduct searches and present search results at a user station, wherein the searches correlate stored activity data with newly acquired activity data at the user station based on the indices.

12. The system as recited in claim 11, wherein the acquisition devices include at least one of sensors and cameras.

13. The system as recited in claim 11, wherein the activity data include activities of an individual and the context includes a retail establishment, and the index association engine is configured to match the indices for the activity data in the context at an earlier time with the newly acquired indices at the user station.

14. The system as recited in claim 13, wherein the user-interface conducts the searches while the individual remains in the retail establishment.

15. The system as recited in claim 11, wherein fraud is detected by locating video, based on indices, of a customer acting in a manner inconsistent with an action made by the customer at a different time.

16. The system as recited in claim 11, wherein the indices include one or more of:

facial appearance, speed, time of observation, color, anthropometric measurements, gait, biometric identifiers.

17. The system as recited in claim 11, wherein the indices include data from one or more of an RFID tag, a pressure pad, a motion sensor, a beam breaker, a local positioning system.

18. The system as recited in claim 11, wherein the index association engine is configured to store indexed activities.

19. The system as recited in claim 11, further comprising a database configured to archive activity data in accordance with the indices.

20. The system as recited in claim 11, further comprising an index server configured to satisfy queries from the user-interface by conducting searches.