Demographic classification using image components

• US 7,505,621 B1
• Filed: 10/22/2004
• Issued: 03/17/2009
• Est. Priority Date: 10/24/2003
• Status: Active Grant

• Current Assignee: Videomining Corporation
• Original Assignee: Videomining Corporation
• Inventors: Sharma, Rajeev, Agrawal, Pyush
• Primary Examiner(s): Mariam, Daniel G
• Application Number:
  US10/972,316
• Time in Patent Office: 1,607 Days
• Field of Search: 382/118, 382/224, 382/159, 382/226, 382/227, 707 1- 10
• US Class Current: 382/159
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06V 10/80 Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level multimodal speaker identification or verification G10L17/10
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/178 estimating age from face image; using age information for improving recognition

Abstract

The present invention includes a system and method for automatically extracting the demographic information from images. The system detects the face in an image, locates different components, extracts component features, and then classifies the components to identify the age, gender, or ethnicity of the person(s) in the image. Using components for demographic classification gives better results as compared to currently known techniques. Moreover, the described system and technique can be used to extract demographic information in more robust manner than currently known methods, in environments where high degree of variability in size, shape, color, texture, pose, and occlusion exists. This invention also performs classifier fusion using Data Level fusion and Multi-level classification for fusing results of various component demographic classifiers. Besides use as an automated data collection system wherein given the necessary facial information as the data, the demographic category of the person is determined automatically, the system could also be used for targeting of the advertisements, surveillance, human computer interaction, security enhancements, immersive computer games and improving user interfaces based on demographic information.

Claims

1. A system for extracting demographic information using facial components comprising;

a) means for detecting one or a plurality of face images from one or a plurality of input images,

b) means for detecting facial components in the face images,

c) means for extracting one or a plurality of facial component features from the facial components,

d) means for automatically generating component information from the facial component features and the facial components, and

e) means for processing the component information using classifiers including data level fusion classifiers and hierarchical fusion classifiers for demographic information identification,

wherein the data level fusion classifiers concatenate preprocessed component information into a single vector,

wherein the hierarchical fusion classifiers process results from component demographic classifiers,

wherein the preprocessed component information from each component is preprocessed differently from each other to improve the accuracy of each component demographic classifier,

wherein the demographic information comprises age, gender, or ethnicity,

whereby said facial components are defined as artifacts that are individual parts of which a composite entity is made up, and

whereby the input images are acquired from an image sensing device or internet.

2. The system according to claim 1, wherein said demographic information comprises one or a plurality of demographic categories that further comprises two or more demographic sub-categories,

whereby of the demographic categories include age, gender, or ethnicity, and

whereby the demographic sub-categories include male, female for the gender demographic category.

3. The system according to claim 1, wherein the classifiers comprise one or a plurality of demographic classifiers,

wherein said demographic classifier comprises a combination of two or more:

a) means for collecting data,

wherein the data is divided into three mutually disjoint parts, including training set, bootstrapping set, and testing set,

b) means for training of classifiers using cross validation,

c) means for bootstrapping to obtain the best classifier, or

d) means for testing classifiers using testing data,

whereby generating best said one or a plurality of classifiers.

4. The system according to claim 1, wherein the system further comprises means for arranging the classifiers in serial, parallel, or hybrid organization, wherein in serial configuration of the classifiers, the classifiers are organized in a cascade method.

5. The system according to claim 1, wherein the classifiers comprise means for performing fusion by concatenating the component information for extracting said demographic information.

6. The system according to claim 1, wherein the classifiers comprise means for passing component demographic classifier results to the hierarchical fusion classifiers,

wherein the hierarchical fusion classifiers are multi-level classifiers,

wherein component information from each component may be preprocessed differently, and

wherein the hierarchical fusion classifiers may perform fusion on the basis of Majority Voting, Support Vector Machine, Hidden Markov Model, Bayesian Networks, Neural Networks, CART, or any other technique available in the classification literature.

7. A method for extracting demographic information using facial components comprising steps of;

a) detecting one or a plurality of face images from one or a plurality of input images,

b) detecting facial components in the face images,

c) extracting one or a plurality of facial component features from the facial components,

d) automatically generating component information from the facial component features and the facial components, and

e) processing the component information using classifiers including data level fusion classifiers and hierarchical fusion classifiers for demographic information identification,

wherein the data level fusion classifiers concatenate preprocessed component information into a single vector,

wherein the hierarchical fusion classifiers process results from component demographic classifiers,

wherein the preprocessed component information from each component is preprocessed differently from each other to improve the accuracy of each component demographic classifier,

wherein the demographic information comprises age, gender, or ethnicity,

whereby said facial components are defined as artifacts that are individual parts of which a composite entity is made up, and

whereby the input images are acquired from an image sensing device or internet.

8. The method according to claim 7, wherein said demographic information comprises one or a plurality of demographic categories that further comprises two or more demographic sub-categories,

whereby the demographic categories include age, gender, or ethnicity, and whereby the demographic sub-categories include male, female for the gender demographic category.

9. The method according to claim 7, wherein the method further comprises a step of using one or a plurality of demographic classifiers,

wherein said demographic classifier comprises a combination of two or more steps of a) data collection,

wherein the data is divided into three mutually disjoint parts, including training set, bootstrapping set, and testing set,

b) training of classifiers using cross validation,

c) bootstrapping to obtain the best classifier, or

d) testing classifiers using testing data,

whereby generating best said one or a plurality of classifiers.

10. The method according to claim 7, wherein the method further comprises a step of arranging the classifiers in serial, parallel, or hybrid organization,

wherein in serial configuration of the classifiers, the classifiers are organized in a cascade method.

11. The method according to claim 7, wherein the method further comprises a step of performing fusion by concatenating the component information for extracting said demographic information.

12. The method according to claim 7, wherein the method further comprises a step of passing component demographic classifier results to the hierarchical fusion classifiers,

wherein the hierarchical fusion classifiers are multi-level classifiers,

wherein component information from each component may be preprocessed differently, and

wherein the hierarchical fusion classifiers may perform fusion on the basis of Majority Voting, Support Vector Machine, Hidden Markov Model, Bayesian Networks, Neural Networks, CART, or any other technique available in the classification literature.