Multispectral detection of personal attributes for video surveillance

• US 9,330,312 B2
• Filed: 05/07/2013
• Issued: 05/03/2016
• Est. Priority Date: 07/28/2010
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Lisa Marie, Feris, Rogerio S., Hampapur, Arun, Vaquero, Daniel Andre
• Primary Examiner(s): Bitar, Nancy
• Application Number:
  US13/888,797
• Publication Number:
  US 20130243256A1
• Time in Patent Office: 1,092 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Techniques, systems, and articles of manufacture for multispectral detection of attributes for video surveillance. A method includes generating one or more training sets of one or more multispectral images, generating a group of one or more multispectral box features, using the one or more training sets to select one or more of the one or more multispectral box features to generate a multispectral attribute detector, and using the multispectral attribute detector to identify a location of an attribute in video surveillance, wherein using the multispectral attribute detector comprises, for one or more locations on each spectral band level of the multispectral image, applying the multispectral attribute detector and producing an output indicating attribute detection or an output indicating no attribute detection, and wherein the attribute corresponds to the multispectral attribute detector.

Claims

1. A method comprising:

generating one or more training sets of one or more multispectral images, wherein said generating one or more training sets comprises generating a positive training set by;

collecting one or more multispectral images of a given attribute for which a multispectral attribute detector is to be trained;

selecting one or more rectangular regions in the one or more collected multispectral images that correspond to the given attribute; and

rescaling the one or more selected regions to correspond to a desired multispectral attribute detector size;

generating a group of one or more multispectral box features;

using the one or more training sets to select one or more of the one or more multispectral box features to generate a multispectral attribute detector; and

using the multispectral attribute detector to identify a location of an attribute in video surveillance, wherein using the multispectral attribute detector comprises, for one or more locations on each spectral band level of the multispectral image, applying the multispectral attribute detector and producing an output indicating attribute detection or an output indicating no attribute detection, and wherein the attribute corresponds to the multispectral attribute detector.

2. The method of claim 1, wherein the one or more multispectral images comprise at least one of a thermal infrared image and a visible image.

3. The method of claim 1, wherein an attribute comprises at least one of facial hair type, nose shape, head characteristic, presence of hat, absence of hat, hairstyle, scar, clothing color, clothing type, and eyewear type.

4. The method of claim 1, wherein generating one or more training sets of one or more multispectral images comprises, for a given attribute for which a detector is to be trained, using a desired detector size as input.

5. The method of claim 1, wherein generating one or more training sets of one or more multispectral images comprises generating a negative training set, wherein generating a negative training set comprises:

collecting one or more multispectral images that do not contain a given attribute for which a detector is to be trained; and

randomly sampling portions of the one or more multispectral images of a size that corresponds to a desired detector size.

6. The method of claim 1, wherein a multispectral box feature comprises a sum of one or more pixel values along one or more regions in a three-dimensional space given by one or more captured images.

7. The method of claim 1, wherein using the one or more training sets to select one or more of the one or more multispectral box features to generate a multispectral attribute detector comprises using a positive training set, a negative training set and the group of multispectral box features as input.

8. The method of claim 7, wherein using the one or more training sets to select one or more of the one or more multispectral box features to generate a multispectral attribute detector comprises using an algorithm to select one or more features for discrimination between items from the positive training set and negative training set.

9. The method of claim 1, wherein using the multispectral attribute detector to identify a location of an attribute in video surveillance comprises using a multispectral image to be classified and the multispectral attribute detector as input, wherein for every location and scale of the multispectral image, applying the multispectral attribute detector to identify the attribute in each spectra of the multispectral image.

10. The method of claim 1, further comprising classifying one or more new multispectral images.

11. The method of claim 1, further comprising using adaptive boosting learning to assemble one or more weak classifiers into one single strong classifier.

12. The method of claim 11, wherein assembling one or more weak classifiers into one single strong classifier comprises initializing one or more sample weights and, for each cycle, identifying a classifier that performs on a weighted sample and increasing weights of one or more misclassified examples.

13. The method of claim 1, wherein said generating a group of one or more multispectral box features comprises the steps of:

receiving input of a two-dimensional detector size of a width and height, a number of spectral bands in the one or more multispectral images, and integer values representing a minimum and maximum width and height of one or more multispectral box features;

fixing a feature width and height;

generating one or more feature building blocks with the fixed width and height;

for one or more locations in the two-dimensional detector size, placing a feature building block at a same location for each spectral band level; and

enumerating one or more combinations of the one or more feature building blocks through each spectral band level until all sizes within the integer values have been covered, and wherein each combination determines a multispectral box feature.

14. A computer program product comprising a tangible computer readable recordable storage device including computer useable program code, the computer program product including:

computer useable program code for generating one or more training sets of one or more multispectral images, wherein said generating one or more training sets comprises generating a positive training set by;

collecting one or more multispectral images of a given attribute for which a multispectral attribute detector is to be trained;

selecting one or more rectangular regions in the one or more collected multispectral images that correspond to the given attribute; and

rescaling the one or more selected regions to correspond to a desired multispectral attribute detector size;

computer useable program code for generating a group of one or more multispectral box features;

computer useable program code for using the one or more training sets to select one or more of the one or more multispectral box features to generate a multispectral attribute detector; and

computer useable program code for using the multispectral attribute detector to identify a location of an attribute in video surveillance, wherein using the multispectral attribute detector comprises, for one or more locations on each spectral band level of the multispectral image, applying the multispectral attribute detector and producing an output indicating attribute detection or an output indicating no attribute detection, and wherein the attribute corresponds to the multispectral attribute detector.

15. The computer program product of claim 14, wherein the computer useable program code for generating one or more training sets of one or more multispectral images comprises computer useable program code for generating a negative training set, and where the computer useable program code for generating a negative training set further comprises:

computer useable program code for collecting one or more multispectral images that do not contain a given attribute for which a detector is to be trained; and

computer useable program code for randomly sampling portions of the one or more multispectral images of a size that corresponds to a desired detector size.

16. The computer program product of claim 14, wherein said generating a group of one or more multispectral box features comprises:

receiving input of a two-dimensional detector size of a width and height, a number of spectral bands in the one or more multispectral images, and integer values representing a minimum and maximum width and height of one or more multispectral box features;

fixing a feature width and height;

generating one or more feature building blocks with the fixed width and height;

for one or more locations in the two-dimensional detector size, placing a feature building block at a same location for each spectral band level; and

enumerating one or more combinations of the one or more feature building blocks through each spectral band level until all sizes within the integer values have been covered, and wherein each combination determines a multispectral box feature.

17. A system comprising:

a memory; and

at least one processor coupled to the memory and operative to;

generate one or more training sets of one or more multispectral images, wherein said generating one or more training sets comprises generating a positive training set by;

collecting one or more multispectral images of a given attribute for which a multispectral attribute detector is to be trained;

selecting one or more rectangular regions in the one or more collected multispectral images that correspond to the given attribute; and

rescaling the one or more selected regions to correspond to a desired multispectral attribute detector size;

generate a group of one or more multispectral box features;

use the one or more training sets to select one or more of the one or more multispectral box features to generate a multispectral attribute detector; and

use the multispectral attribute detector to identify a location of an attribute in video surveillance, wherein using the multispectral attribute detector comprises, for one or more locations on each spectral band level of the multispectral image, applying the multispectral attribute detector and producing an output indicating attribute detection or an output indicating no attribute detection, and wherein the attribute corresponds to the multispectral attribute detector.

18. The system of claim 17, wherein said generating a group of one or more multispectral box features comprises:

receiving input of a two-dimensional detector size of a width and height, a number of spectral bands in the one or more multispectral images, and integer values representing a minimum and maximum width and height of one or more multispectral box features;

fixing a feature width and height;

generating one or more feature building blocks with the fixed width and height;

for one or more locations in the two-dimensional detector size, placing a feature building block at a same location for each spectral band level; and

enumerating one or more combinations of the one or more feature building blocks through each spectral band level until all sizes within the integer values have been covered, and wherein each combination determines a multispectral box feature.