Facilitating People Search in Video Surveillance

• US 20120030208A1
• Filed: 07/28/2010
• Published: 02/02/2012
• Est. Priority Date: 07/28/2010
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Lisa M., Feris, Rogerio S., Hampapur, Arun, Vaquero, Daniel A.
• Granted Patent: US 10,424,342 B2
• Time in Patent Office: Days
• US Class Current: 707/741
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

Techniques for facilitating a video surveillance search of a person are provided. The techniques include maintaining a database of one or more attributes of one or more people captured on one or more video cameras, indexing the one or more attributes in the database extracted from the one or more video cameras, and pruning one or more images captured from the one or more video cameras using the one or more attributes and one or more items of qualifying information to facilitate a video surveillance search of a person.

Claims

1. A method for facilitating a video surveillance search of a person, wherein the method comprises:

maintaining a database of one or more attributes of one or more people captured on one or more video cameras;

indexing the one or more attributes in the database extracted from the one or more video cameras; and

pruning one or more images captured from the one or more video cameras using the one or more attributes and one or more items of qualifying information to facilitate a video surveillance search of a person.

2. The method of claim 1, wherein indexing comprises ranking and ordering performed by an optimization module, wherein the ranking is based on optimization results that return a likelihood assessment for each person in the database.

3. The method of claim 1, wherein pruning comprises culling the one or more images captured from the one or more video cameras to a subset.

4. The method of claim 1, wherein maintaining a database comprises recording each person seen in a camera in the database with each of one or more detected attributes of the person.

5. The method of claim 1, wherein one or more items of qualifying information comprise a detection accuracy level associated with each attribute.

6. The method of claim 1, wherein one or more items of qualifying information comprise at least one of empirical information, historical information and error information.

7. The method of claim 1, wherein one or more items of qualifying information comprise information provided by a user associated with an attribute.

8. The method of claim 1, wherein one or more items of qualifying information comprise a correlation between two or more attributes.

9. The method of claim 1, wherein the one or more attributes comprise at least one of clothing type, shirt color, pants color, presence of sunglasses, presence of eyeglasses, presence of a beard, presence of a mustache, presence of a hat, nose type, sideburns, clothes type, skin color, eye color, presence of a carried bag, and absence or presence of a companion.

10. The method of claim 1, further comprising using a user interface to enable a user to supply information about the person.

11. The method of claim 1, further comprising sending a user a set of one or more candidates in the search for the person, wherein the set of one or more candidates are sent to the user in order of their likelihood based on joint probabilities for each candidate.

12. The method of claim 1, further comprising iterating the process of pruning one or more images captured from the one or more video cameras using the one or more attributes and one or more items of qualifying information to facilitate a video surveillance search of a person if the person is not found until a satisfactory match is found or the database is exhausted.

13. The method of claim 1, further comprising modifying attribute description and detection, and finding a most likely feature set based on correlation between one or more attributes and corresponding error distributions if the person is not found.

14. The method of claim 1, further comprising repeating the process of pruning one or more images captured from the one or more video cameras using the one or more attributes and one or more items of qualifying information to facilitate a video surveillance search of a person in a second image captured by the one or more video cameras if the person is found in one image captured by the one or more video cameras.

15. The method of claim 14, further comprising using an updated description of the person based on results from the one image captured by the one or more video cameras.

16. The method of claim 1, further comprising providing a system, wherein the system comprises one or more distinct software modules, each of the one or more distinct software modules being embodied on a tangible computer-readable recordable storage medium, and wherein the one or more distinct software modules comprise a person descriptor module, and a search optimization module executing on a hardware processor.

17. A computer program product comprising a tangible computer readable recordable storage medium including computer useable program code for facilitating a video surveillance search of a person, the computer program product including:

computer useable program code for maintaining a database of one or more attributes of one or more people captured on one or more video cameras;

computer useable program code for indexing the one or more attributes in the database extracted from the one or more video cameras; and

computer useable program code for pruning one or more images captured from the one or more video cameras using the one or more attributes and one or more items of qualifying information to facilitate a video surveillance search of a person.

18. The computer program product of claim 17, wherein one or more items of qualifying information comprise at least one of a detection accuracy level associated with each attribute, empirical information, historical information, error information, information provided by a user associated with an attribute, and a correlation between two or more attributes.

19. The computer program product of claim 17, further comprising:

computer useable program code for using a user interface to enable a user to supply information about the person; and

computer useable program code for sending a user a set of one or more candidates in the search for the person, wherein the set of one or more candidates are sent to the user in order of their likelihood based on joint probabilities for each candidate.

20. The computer program product of claim 17, further comprising computer useable program code for modifying attribute description and detection, and finding a most likely feature set based on correlation between one or more attributes and corresponding error distributions if the person is not found.

21. A system for facilitating a video surveillance search of a person, comprising:

a memory; and

at least one processor coupled to the memory and operative to;

maintain a database of one or more attributes of one or more people captured on one or more video cameras;

index the one or more attributes in the database extracted from the one or more video cameras; and

prune one or more images captured from the one or more video cameras using the one or more attributes and one or more items of qualifying information to facilitate a video surveillance search of a person.

22. The system of claim 21, wherein one or more items of qualifying information comprise at least one of a detection accuracy level associated with each attribute, empirical information, historical information, error information, information provided by a user associated with an attribute, and a correlation between two or more attributes.

23. The system of claim 21, wherein the at least one processor coupled to the memory is further operative to:

use a user interface to enable a user to supply information about the person; and

send a user a set of one or more candidates in the search for the person, wherein the set of one or more candidates are sent to the user in order of their likelihood based on joint probabilities for each candidate.

24. The system of claim 21, wherein the at least one processor coupled to the memory is further operative to modifying attribute description and detection, and finding a most likely feature set based on correlation between one or more attributes and corresponding error distributions if the person is not found.

25. An apparatus for facilitating a video surveillance search of a person, the apparatus comprising:

means for maintaining a database of one or more attributes of one or more people captured on one or more video cameras;

means for indexing the one or more attributes in the database extracted from the one or more video cameras; and

means for pruning one or more images captured from the one or more video cameras using the one or more attributes and one or more items of qualifying information to facilitate a video surveillance search of a person.