Attribute-based person tracking across multiple cameras

• US 9,134,399 B2
• Filed: 07/28/2010
• Issued: 09/15/2015
• Est. Priority Date: 07/28/2010
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Lisa M., Feris, Rogerio S., Hampapur, Arun, Vaquero, Daniel A.
• Primary Examiner(s): Lazaro, David
• Assistant Examiner(s): HENRY, MARIEGEORGES A
• Application Number:
  US12/845,119
• Publication Number:
  US 20120026335A1
• Time in Patent Office: 1,875 Days
• Field of Search: 348/159, 348/208.14
• US Class Current: 1/1
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks

Abstract

Techniques for tracking an individual across two or more cameras are provided. The techniques include detecting an image of one or more individuals in each of two or more cameras, tracking each of the one or more individuals in a field of view in each of the two or more cameras, applying a set of one or more attribute detectors to each of the one or more individuals being tracked by the two or more cameras, and using the set of one or more attribute detectors to match an individual tracked in one of the two or more cameras with an individual tracked in one or more other cameras of the two or more cameras.

Claims

1. A method for tracking an individual across two or more cameras, wherein the method comprises:

detecting an image of one or more individuals in each of two or more cameras;

tracking each of the one or more individuals in a field of view in each of the two or more cameras;

applying a set of multiple attribute detectors to the images of each of the one or more individuals being tracked by the two or more cameras, wherein said attribute detectors (i) provide a probability that given attributes are present in an image and (ii) are learned from training images in multiple levels of resolution to produce robustness to multiple changes in lighting and viewpoint; and

using the set of multiple attribute detectors to match an individual tracked in one of the two or more cameras with an individual tracked in one or more other cameras of the two or more cameras, wherein using the set of multiple attribute detectors to match an individual tracked in one of the two or more cameras with an individual tracked in one or more other cameras of the two or more cameras comprises;

using a maximum confidence value of each of the multiple attribute detectors to generate a feature vector of each of the one or more individuals;

determining a weight associated with each of the multiple attribute detectors, wherein said weight corresponds to a reliability measure of the corresponding attribute detector relative to the remaining attribute detectors in the set of multiple attribute detectors;

calculating a distance between each vector using a weighted vector distance between each vector based on the determined weight associated with each of the multiple attribute detectors; and

comparing the distance to a threshold to determine if the individual tracked in one of the two or more cameras is the same individual as the individual tracked in one or more other cameras of the two or more cameras.

2. The method of claim 1, further comprising assigning a unique tracking identifier for each trajectory of an individual locally tracked in each of the two or more cameras to facilitate tracking in a global coordinate system.

3. The method of claim 1, further comprising obtaining a feature vector for the individual when the individual moves within the field of view of a second camera.

4. The method of claim 3, further comprising determining an amount of time taken by the individual to leave a view of the first camera and appear within a view of the second camera to determine if the amount of time is consistent with an established range of time.

5. The method of claim 4, further comprising computing, if the amount of time is consistent with the established range time, a weighted vector distance between the first camera and the second camera and comparing the weighted vector distance to a threshold to determine if the individual in the view of the first camera corresponds to the individual in the view of the second camera.

6. The method of claim 5, further comprising unifying a trajectory of the first camera and the second camera to correspond to a single track if the individual in the view of the first camera corresponds to the individual in the view of the second camera.

7. The method of claim 1, wherein the reliability of each of the multiple attribute detectors is determined using an artificial neural network.

8. The method of claim 1, wherein the set of multiple attribute detectors comprises a set of multiple fine-grained Adaptive Boosting attribute detectors.

9. The method of claim 1, wherein the set of multiple attribute detectors comprises one or more attribute detectors for at least one of presence of a beard, presence of a moustache, presence of eyeglasses, presence of sunglasses, presence of a hat, absence of hair, shape of body features, shape of clothing features, clothing color and clothing texture.

10. The method of claim 1, wherein tracking each of the one or more individuals in a local field of view in each of the two or more cameras comprises using one or more computer vision algorithms.

11. The method of claim 1, wherein the multiple attribute detectors are learned from a set of one or more training images in one or more levels of resolution.

12. The method of claim 1, wherein using the set of multiple attribute detectors to match an individual tracked in one of the two or more cameras with an individual tracked in one or more other cameras of the two or more cameras comprises using at least one of a geometric configuration of the two or more cameras, time information, and a set of individual attributes.

13. The method of claim 1, further comprising using one or more non-visual attributes from one or more sensors.

14. The method of claim 13, wherein the one or more non-visual attributes comprises at least one of odor and temperature.

15. The method of claim 1, further comprising providing a system, wherein the system comprises one or more distinct software modules, each of the one or more distinct software modules being embodied on a tangible computer-readable recordable storage medium, and wherein the one or more distinct software modules comprise a person detection module, a person tracking module, a human parsing module and a matching module executing on a hardware processor.

16. A computer program product comprising a tangible computer readable recordable storage device including computer useable program code for tracking an individual across two or more cameras, the computer program product including:

computer useable program code for detecting an image of one or more individuals in each of two or more cameras;

computer useable program code for tracking each of the one or more individuals in a field of view in each of the two or more cameras;

computer useable program code for applying a set of multiple attribute detectors to the images of each of the one or more individuals being tracked by the two or more cameras, wherein said attribute detectors (i) provide a probability that given attributes are present in an image and (ii) are learned from training images in multiple levels of resolution to produce robustness to multiple changes in lighting and viewpoint; and

computer useable program code for using the set of multiple attribute detectors to match an individual tracked in one of the two or more cameras with an individual tracked in one or more other cameras of the two or more cameras, wherein the computer useable program code for using the set of multiple attribute detectors to match an individual tracked in one of the two or more cameras with an individual tracked in one or more other cameras of the two or more cameras comprises;

computer useable program code for using a maximum confidence value of each of the multiple attribute detectors to generate a feature vector of each of the one or more individuals;

computer useable program code for determining a weight associated with each of the multiple attribute detectors, wherein said weight corresponds to a reliability measure of the corresponding attribute detector relative to the remaining attribute detectors in the set of multiple attribute detectors;

computer useable program code for calculating a distance between each vector using a weighted vector distance between each vector based on the determined weight associated with each of the multiple attribute detectors; and

computer useable program code for comparing the distance to a threshold to determine if the individual tracked in one of the two or more cameras is the same individual as the individual tracked in one or more other cameras of the two or more cameras.

17. The computer program product of claim 16, further comprising computer useable program code for assigning a unique tracking identifier for each trajectory of an individual locally tracked in each of the two or more cameras to facilitate tracking in a global coordinate system.

18. The computer program product of claim 16, further comprising computer useable program code for wherein the computer useable program code for using the set of multiple attribute detectors to match an individual tracked in one of the two or more cameras with an individual tracked in one or more other cameras of the two or more cameras comprises computer useable program code for using at least one of a geometric configuration of the two or more cameras, time information, and a set of individual attributes.

19. The computer program product of claim 16, further comprising:

computer useable program code for obtaining a feature vector for the individual when the individual moves within the field of view of a second camera; and

computer useable program code for determining an amount of time taken by the individual to leave a view of the first camera and appear within a view of the second camera to determine if the amount of time is consistent with an established range of time.

20. A system for tracking an individual across two or more cameras, comprising:

a memory; and

at least one processor coupled to the memory and operative to;

detect an image of one or more individuals in each of two or more cameras;

track each of the one or more individuals in a field of view in each of the two or more cameras;

apply a set of multiple attribute detectors to the images of each of the one or more individuals being tracked by the two or more cameras, wherein said attribute detectors (i) provide a probability that given attributes are present in an image and (ii) are learned from training images in multiple levels of resolution to produce robustness to multiple changes in lighting and viewpoint; and

using the set of multiple attribute detectors to match an individual tracked in one of the two or more cameras with an individual tracked in one or more other cameras of the two or more cameras, wherein at least one processor coupled to the memory and operative to using the set of multiple attribute detectors to match an individual tracked in one of the two or more cameras with an individual tracked in one or more other cameras of the two or more cameras is further operative to;

use a maximum confidence value of each of the multiple attribute detectors to generate a feature vector of each of the one or more individuals;

determine a weight associated with each of the multiple attribute detectors, wherein said weight corresponds to a reliability measure of the corresponding attribute detector relative to the remaining attribute detectors in the set of multiple attribute detectors;

calculate a distance between each vector using a weighted vector distance between each vector based on the determined weight associated with each of the multiple attribute detectors; and

compare the distance to a threshold to determine if the individual tracked in one of the two or more cameras is the same individual as the individual tracked in one or more other cameras of the two or more cameras.

21. The system of claim 20, wherein the at least one processor coupled to the memory is further operative to assign a unique tracking identifier for each trajectory of an individual locally tracked in each of the two or more cameras to facilitate tracking in a global coordinate system.

22. The system of claim 20, wherein the at least one processor coupled to the memory is operative to use the set of multiple attribute detectors to match an individual tracked in one of the two or more cameras with an individual tracked in one or more other cameras of the two or more cameras is further operative to use at least one of a geometric configuration of the two or more cameras, time information, and a set of individual attributes.

23. The system of claim 20, wherein the at least one processor coupled to the memory is further operative to:

obtain a feature vector for the individual when the individual moves within the field of view of a second camera; and

determine an amount of time taken by the individual to leave a view of the first camera and appear within a view of the second camera to determine if the amount of time is consistent with an established range of time.