Real-time clothing recognition in surveillance videos

• US 8,379,920 B2
• Filed: 01/05/2011
• Issued: 02/19/2013
• Est. Priority Date: 05/05/2010
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Laboratories America Inc (NEC Corporation)
• Inventors: Yang, Ming, Yu, Kai
• Primary Examiner(s): Wang, Claire X
• Assistant Examiner(s): Wang, Carol
• Application Number:
  US12/985,003
• Publication Number:
  US 20110274314A1
• Time in Patent Office: 776 Days
• Field of Search: 382/103, 382/111, 382/195, 382/209, 382/224, 382/173, 348/77, 706/12, 706/20
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20156 Automatic seed setting
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/178 estimating age from face image; using age information for improving recognition

Abstract

Systems and methods are disclosed to recognize clothing from videos by detecting and tracking a human; performing face alignment and occlusal detection; and performing age and gender estimation, skin area extraction, and clothing segmentation to a linear support vector machine (SVM) to recognize clothing worn by the human.

Claims

1. A method to recognize clothing from videos, comprising

detecting and tracking a human,

evaluating texture features based on histogram of oriented gradient (HOG) in multiple spatial cells, a bag of dense SIFT features, and DCT responses, wherein the gradient orientations on 8 directions (every 45 degree) are computed on color segmentation results;

performing face alignment and occlusal detection; and

performing age and gender estimation, skin area extraction, and clothing segmentation to a linear support vector machine (SVM) to recognize clothing worn by the human.

2. The method of claim 1, comprising checking for quality of shape/texture and color histograms.

3. The method of claim 1, comprising evaluating different clothing representations combining color histograms with a histogram of oriented gradient (HOG), Bag-of-Words (BOW) features, and discrete cosine transform (DCT) features.

4. The method of claim 1, comprising combining gender and age, skin area ratios of limbs, color, shape and texture features using the clothing segmentation results as a mask.

5. The method of claim 1, comprising using a Voronoi image to conduct seed selection in a region growing segmentation method.

6. The method of claim 1, comprising determining gradient orientations at every 45 degrees on a color segmentation.

7. The method of claim 1, comprising spatially dividing top and bottom parts of a human image into cells, and concatenating histograms of all cells as a HOG feature.

8. The method of claim 1, comprising quantizing local features with a visual codebook and normalizing frequencies of the codewords to generate a BoW descriptor.

9. The method of claim 1, comprising packing components of DCT coefficients to form a DCT feature.

10. The method of claim 1, comprising determining local shape descriptors of color segments on the human.

11. A system to recognize clothing from videos, comprising

means for detecting and tracking a human,

means for valuating texture features based on histogram of oriented gradient (HOG) in multiple spatial cells, a bag of dense SIFT features, and DCT responses, wherein the gradient orientations on 8 directions (every 45 degree) are computed on color segmentation results;

means for performing face alignment and occlusal detection; and

means for performing age and gender estimation, skin area extraction, and clothing segmentation to a linear support vector machine (SVM) to recognize clothing worn by the human.

12. The system of claim 11, comprising means for checking for quality of shape/texture and color histograms.

13. The system of claim 11, comprising means for evaluating different clothing representations combining color histograms with a histogram of oriented gradient (HOG), Bag-of-Words (BOW) features, and discrete cosine transform (DCT) features.

14. The system of claim 11, comprising means for combining gender and age, skin area ratios of limbs, color, shape and texture features using the clothing segmentation results as a mask.

15. The system of claim 11, comprising a Voronoi image to conduct seed selection in a region growing segmentation method.

16. The system of claim 11, comprising means for determining gradient orientations at every 45 degrees on a color segmentation.

17. The system of claim 11, comprising means for spatially dividing top and bottom parts of a human image into cells, and concatenating histograms of all cells as a HOG feature.

18. The system of claim 11, comprising means for quantizing local features with a visual codebook and normalizing frequencies of the codewords to generate a BoW descriptor.

19. The system of claim 11, comprising means for packing components of DCT coefficients to form a DCT feature.

20. The system of claim 11, comprising means for determining local shape descriptors of color segments on the human.