Method and system for determining the age category of people based on facial images

• US 7,912,246 B1
• Filed: 01/29/2008
• Issued: 03/22/2011
• Est. Priority Date: 10/28/2002
• Status: Active Grant

• Current Assignee: Videomining Corporation
• Original Assignee: Videomining Corporation
• Inventors: Moon, Hankyu, Sharma, Rajeev, Jung, Namsoon
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US12/011,748
• Time in Patent Office: 1,148 Days
• Field of Search: 382/100, 382/103, 382/118, 382/224, 382/226
• US Class Current: 382/103
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 10/76 based on eigen-space representations, e.g. from pose or different illumination conditions; Shape manifolds
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/178 estimating age from face image; using age information for improving recognition

Abstract

The present invention is a system and method for performing age classification or age estimation based on the facial images of people, using multi-category decomposition architecture of classifiers. In the multi-category decomposition architecture, which is a hybrid multi-classifier architecture specialized to age classification, the task of learning the concept of age against significant within-class variations, is handled by decomposing the set of facial images into auxiliary demographics classes, and the age classification is performed by an array of classifiers where each classifier, called an auxiliary class machine, is specialized to the given auxiliary class. The facial image data is annotated to assign the gender and ethnicity labels as well as the age labels. Each auxiliary class machine is trained to output both the given auxiliary class membership likelihood and the age group likelihoods. Faces are detected from the input image and individually tracked. Age sensitive feature vectors are extracted from the tracked faces and are fed to all of the auxiliary class machines to compute the desired likelihood outputs. The outputs from all of the auxiliary class machines are combined in a manner to make a final decision on the age of the given face.

Claims

1. A method for determining age categories of people, comprising the following steps of:

a) annotating a facial image database according to the demographics classes of the individual face,

b) training a plurality of learning machines so that each learning machine outputs auxiliary demographics class information and age information of any given facial image,

c) detecting and tracking a facial image from the input image frame,

d) processing said facial image to extract image features, and

e) processing said image features obtained from said facial image using classification techniques for determining age or age categories,

whereby the age classes can be any partition based on age in multiple groups,

wherein the method further comprises a step of determining the target outputs of the plurality of learning machines so that each learning machine maps a first input data, whose facial images belong to a first auxiliary demographics class, to first vector-valued points on a manifold in the space of facial images, whereas each learning machine maps a second input data, whose facial images do not belong to a first auxiliary demographics class, to second vector-valued points away from the manifold.

3. The method according to claim 1, wherein the method further comprises a step of annotating the facial image database with the auxiliary class labels and the age labels of each facial image,

whereby the auxiliary class is a demographics class defined by the same gender and ethnicity categories.

4. The method according to claim 1, wherein the method further comprises a step of setting up a plurality of learning machines so that each learning machine represents an auxiliary demographics class,

whereby the auxiliary demographics class comprises facial images having the same gender and ethnicity classes.

5. The method according to claim 1, wherein the method further comprises a step of determining the target age vector outputs of the plurality of learning machines so that each learning machine outputs an age vector,

wherein each component of the age vector evaluates a function that corresponds to a specific fixed age value.

6. The method according to claim 1, wherein the method further comprises a step of training the plurality of learning machines so that each learning machine is trained to force the distance between the ground truth age vector and the age vector computed by the learning machine to have either of two target values,

wherein the target value of the learning machine is zero when the input data belong to the given auxiliary class, and the target value is a fixed large value when the input data do not belong to the given auxiliary class.

7. The method according to claim 1, wherein the method further comprises a step of projecting the age vector computed by the learning machine to the closest point on the auxiliary class manifold to determine the likelihood of an input data belonging to an auxiliary demographics class by computing the distance between the age vector and the closest point on the auxiliary class manifold, and to directly use the closest point on the auxiliary manifold as information relevant to determine the age.

8. The method according to claim 1, wherein the method further comprises a step of combining the auxiliary class information and the age information from the plurality of learning machines to estimate the age or age group of the input face, so that the degree of contribution of the age information from each learning machine is determined by the auxiliary class information from the same machine.

9. The method according to claim 1, wherein the method further comprises a step of extracting a set of age sensitive features from a facial image,

wherein the set of age sensitive features is fed to auxiliary demographics class machines,

wherein each of the auxiliary demographics class machines computes an auxiliary class likelihood output of the facial image belonging to the auxiliary demographics class and age outputs, and

whereby the set of age sensitive features are computed by applying a set of high frequency filters that are designed to extract the positions and shapes of facial boundary, facial features, and to extract facial wrinkles.

10. The method according to claim 9, wherein the method further comprises a step of training a plurality of learning machines using the age sensitive features as training data to classify the age of the facial image,

wherein a desired age vector and auxiliary class likelihood output is calculated for each training data, and

wherein each training data corresponds to one of the auxiliary demographics classes.

11. The method according to claim 1, wherein the method further comprises a step of arranging the plurality of learning machines in serial,

wherein the arrangement can be any sequential ordering of the plurality of classifiers.

12. The method according to claim 1, wherein the method further comprises a step of arranging the plurality of learning machines in parallel.

13. The method according to claim 1, wherein the method further comprises a step of arranging a plurality of classifiers in a manner of combining chains of serial classifiers in parallel,

whereby the first classifier in each chain of serial classifiers performs a gross-level classification into gross-level classes by computing gross-level class information and the second classifier in each chain of serial classifiers performs a finer level classification that is specialized to one of the gross-level classes by computing the finer level class information.

2. An apparatus for determining age categories of people, comprising:

a) means for annotating a facial image database according to the demographics classes of the individual face,

b) means for training a plurality of learning machines so that each learning machine outputs auxiliary demographics class information and age information of any given facial image,

c) means for detecting and tracking a facial image from the input image frame,

d) means for processing said facial image to extract image features, and

e) means for processing said image features obtained from said facial image using classification techniques for determining age or age categories,

whereby the age classes can be any partition based on age in multiple groups,

wherein the apparatus further comprises means for determining the target outputs of the plurality of learning machines so that each learning machine maps a first input data, whose facial images belong to a first auxiliary demographics class, to first vector-valued points on a manifold in the space of facial images, whereas each learning machine maps a second input data, whose facial images do not belong to a first auxiliary demographics class, to second vector-valued points away from the manifold.

14. The apparatus according to claim 2, wherein the apparatus further comprises means for annotating the facial image database with the auxiliary class labels and the age labels of each facial image,

whereby the auxiliary class is a demographics class defined by the same gender and ethnicity categories.

15. The apparatus according to claim 2, wherein the apparatus further comprises means for setting up a plurality of learning machines so that each learning machine represents an auxiliary demographics class,

whereby the auxiliary demographics class comprises facial images having the same gender and ethnicity classes.

16. The apparatus according to claim 2, wherein the apparatus further comprises means for determining the target age vector outputs of the plurality of learning machines so that each learning machine outputs an age vector,

wherein each component of the age vector evaluates a function that corresponds to a specific fixed age value.

17. The apparatus according to claim 2, wherein the apparatus further comprises means for training the plurality of learning machines so that each learning machine is trained to force the distance between the ground truth age vector and the age vector computed by the learning machine to have either of two target values,

wherein the target value of the learning machine is zero when the input data belong to the given auxiliary class, and the target value is a fixed large value when the input data do not belong to the given auxiliary class.

18. The apparatus according to claim 2, wherein the apparatus further comprises means for projecting the age vector computed by the learning machine to the closest point on the auxiliary class manifold to determine the likelihood of an input data belonging to an auxiliary demographics class by computing the distance between the age vector and the closest point on the auxiliary class manifold, and to directly use the closest point on the auxiliary manifold as information relevant to determine the age.

19. The apparatus according to claim 2, wherein the apparatus further comprises means for combining the auxiliary class information and the age information from the plurality of learning machines to estimate the age or age group of the input face, so that the degree of contribution of the age information from each learning machine is determined by the auxiliary class information from the same machine.

20. The apparatus according to claim 2, wherein the apparatus further comprises means for extracting a set of age sensitive features from a facial image,

wherein the set of age sensitive features is fed to auxiliary demographics class machines,

wherein each of the auxiliary demographics class machines computes an auxiliary class likelihood output of the facial image belonging to the auxiliary demographics class and age outputs, and

whereby the set of age sensitive features are computed by applying a set of high frequency filters that are designed to extract the positions and shapes of facial boundary, facial features, and to extract facial wrinkles.

21. The apparatus according to claim 20, wherein the apparatus further comprises means for training a plurality of learning machines using the age sensitive features as training data to classify the age of the facial image,

wherein a desired age vector and auxiliary class likelihood output is calculated for each training data, and

wherein each training data corresponds to one of the auxiliary demographics classes.

22. The apparatus according to claim 2, wherein the apparatus further comprises means for arranging the plurality of learning machines in serial,

wherein the arrangement can be any sequential ordering of the plurality of classifiers.

23. The apparatus according to claim 2, wherein the apparatus further comprises means for arranging the plurality of learning machines in parallel.

24. The apparatus according to claim 2, wherein the apparatus further comprises means for arranging a plurality of classifiers in a manner of combining chains of serial classifiers in parallel,

whereby the first classifier in each chain of serial classifiers performs a gross-level classification into gross-level classes by computing gross-level class information and the second classifier in each chain of serial classifiers performs a finer level classification that is specialized to one of the gross-level classes by computing the finer level class information.