Demographic Analysis of Facial Landmarks

• US 20150086087A1
• Filed: 12/04/2014
• Published: 03/26/2015
• Est. Priority Date: 09/27/2011
• Status: Active Grant

• Current Assignee: University of North Carolina At Wilmington (University of North Carolina System)
• Original Assignee: University of North Carolina At Wilmington (University of North Carolina System)
• Inventors: Ricanek, Karl Jr., Wang, Yishi, Chang, Yaw, Chen, Cuixian
• Granted Patent: US 9,317,740 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A set of training vectors may be identified. Each training vector may be mapped to either a male gender or a female gender, and each training vector may represent facial landmarks derived from a respective facial image. An input vector of facial landmarks may also be identified. The facial landmarks of the input vector may be derived from a particular facial image. A feature vector may containing a subset of the facial landmarks may be selected from the input vector. A weighted comparison may be performed between the feature vector and each of the training vectors. Based on a result of the weighted comparison, the particular facial image may be classified as either the male gender or the female gender.

Claims

1. A method comprising:

obtaining a set of training vectors, wherein each training vector is mapped to either a male gender or a female gender, and wherein the training vectors represent facial landmarks derived from respective facial images;

identifying, by a computing device, an input vector of facial landmarks, wherein the facial landmarks of the input vector are derived from a particular facial image;

selecting, by the computing device, a feature vector containing a subset of the facial landmarks of the input vector, wherein selecting the feature vector comprises;

(i) constructing a graph representing the training vectors, each vertex in the graph corresponding to one of the training vectors, (ii) connecting pairs of vertices in the graph using respective uniform weight edges where the corresponding training vectors map to the same gender, and (iii) selecting the feature vector based on an adjacency matrix of the graph; and

based on a weighted comparison between the feature vector and at least some of the training vectors, classifying, by the computing device, the particular facial image as either the male gender or the female gender.

2. The method of claim 1, wherein the feature vector is less than one-half the size of the input vector.

3. The method of claim 1, wherein the feature vector is less than one-quarter the size of the input vector.

4. The method of claim 1, wherein selecting the feature vector further comprises using a memetic algorithm to select the feature vector, wherein the memetic algorithm determines the fitness of a candidate feature vector based on a classification accuracy of a facial image associated with the candidate feature vector, and wherein the memetic algorithm uses a recombination process that produces a child feature vector of a size no greater than that of the each parent feature vector of the child feature vector.

5. The method of claim 4, wherein a non-selected feature vector contains facial landmarks from one of the training vectors that are not in the candidate feature vector, and wherein using the memetic algorithm comprises:

ranking the features in the candidate feature vector and the non-selected feature vector each in order of importance according to a univariate filter; and

modifying the candidate feature vector by adding the m highest-ranked features of the non-selected feature vector into the candidate feature vector, and removing the n lowest-ranked features from the candidate feature vector.

6. The method of claim 5, wherein modification of the candidate feature vector is performed for all m and n less than a particular value, and wherein a variation of the candidate feature vector for which the selected m and n result in a sufficiently high fitness as determined by the memetic algorithm is selected as the feature vector.

7. The method of claim 1, wherein selecting the feature vector is also based on eigenvectors of an equation involving a training matrix representing the training vectors.

8. The method of claim 1, wherein selecting the feature vector further comprises:

determining a training matrix representing the training vectors;

determining a covariance matrix of the training matrix; and

selecting the feature vector based on an eigenvector of the covariance matrix.

9. The method of claim 1, wherein selecting the feature vector further comprises:

determining a training matrix representing the training vectors;

determining a covariance matrix of the training matrix; and

using a random forest technique to (i) build a plurality of trees, wherein each node of each tree in the plurality of trees represents a random selection of the facial landmarks, (ii) calculating the Gini importance of the facial landmarks, and (iii) based on the calculated Gini importance, selecting the feature vector.

10. An article of manufacture including a non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by a computing device, cause the computing device to perform operations comprising:

obtaining a set of training vectors, wherein each training vector is mapped to either a male gender or a female gender, and wherein the training vectors represent facial landmarks derived from respective facial images;

identifying an input vector of facial landmarks, wherein the facial landmarks of the input vector are derived from a particular facial image;

selecting a feature vector containing a subset of the facial landmarks of the input vector, wherein selecting the feature vector comprises;

(i) constructing a graph representing the training vectors, each vertex in the graph corresponding to one of the training vectors, (ii) connecting pairs of vertices in the graph using respective uniform weight edges where the corresponding training vectors map to the same gender, and (iii) selecting the feature vector based on an adjacency matrix of the graph; and

based on a weighted comparison between the feature vector and at least some of the training vectors, classifying the particular facial image as either the male gender or the female gender.

11. The article of manufacture of claim 10, wherein selecting the feature vector further comprises using a memetic algorithm to select the feature vector, wherein the memetic algorithm determines the fitness of a candidate feature vector based on a classification accuracy of a facial image associated with the candidate feature vector, and wherein the memetic algorithm uses a recombination process that produces a child feature vector of a size no greater than that of the each parent feature vector of the child feature vector.

12. The article of manufacture of claim 11, wherein a non-selected feature vector contains facial landmarks from one of the training vectors that are not in the candidate feature vector, and wherein using the memetic algorithm comprises:

ranking the features in the candidate feature vector and the non-selected feature vector each in order of importance according to a univariate filter; and

modifying the candidate feature vector by adding the m highest-ranked features of the non-selected feature vector into the candidate feature vector, and removing the n lowest-ranked features from the candidate feature vector.

13. The article of manufacture of claim 12, wherein modification of the candidate feature vector is performed for all m and n less than a particular value, and wherein a variation of the candidate feature vector for which the selected m and n result in a sufficiently high fitness as determined by the memetic algorithm is selected as the feature vector.

14. The article of manufacture of claim 10, wherein selecting the feature vector is also based on eigenvectors of an equation involving a training matrix representing the training vectors.

15. The article of manufacture of claim 10, wherein selecting the feature vector further comprises:

determining a training matrix representing the training vectors;

determining a covariance matrix of the training matrix; and

selecting the feature vector based on an eigenvector of the covariance matrix.

16. The article of manufacture of claim 10, wherein selecting the feature vector further comprises:

determining a training matrix representing the training vectors;

determining a covariance matrix of the training matrix; and

using a random forest technique to (i) build a plurality of trees, wherein each node of each tree in the plurality of trees represents a random selection of the facial landmarks, (ii) calculating the Gini importance of the facial landmarks, and (iii) based on the calculated Gini importance, selecting the feature vector.

17. A computing system comprising:

at least one processor;

memory; and

program instructions, stored in the memory, that upon execution by the processor cause the computing system to perform operations including;

obtaining a set of training vectors, wherein each training vector is mapped to either a male gender or a female gender, and wherein the training vectors represent facial landmarks derived from respective facial images;

identifying an input vector of facial landmarks, wherein the facial landmarks of the input vector are derived from a particular facial image;

selecting a feature vector containing a subset of the facial landmarks of the input vector, wherein selecting the feature vector comprises;

(i) constructing a graph representing the training vectors, each vertex in the graph corresponding to one of the training vectors, (ii) connecting pairs of vertices in the graph using respective uniform weight edges where the corresponding training vectors map to the same gender, and (iii) selecting the feature vector based on an adjacency matrix of the graph; and

based on a weighted comparison between the feature vector and at least some of the training vectors, classifying the particular facial image as either the male gender or the female gender.

18. The computing device of claim 17, wherein selecting the feature vector further comprises using a memetic algorithm to select the feature vector, wherein the memetic algorithm determines the fitness of a candidate feature vector based on a classification accuracy of a facial image associated with the candidate feature vector, and wherein the memetic algorithm uses a recombination process that produces a child feature vector of a size no greater than that of the each parent feature vector of the child feature vector.

19. The computing device of claim 18, wherein a non-selected feature vector contains facial landmarks from one of the training vectors that are not in the candidate feature vector, and wherein using the memetic algorithm comprises:

ranking the features in the candidate feature vector and the non-selected feature vector each in order of importance according to a univariate filter; and

modifying the candidate feature vector by adding the m highest-ranked features of the non-selected feature vector into the candidate feature vector, and removing the n lowest-ranked features from the candidate feature vector.

20. The computing device of claim 19, wherein modification of the candidate feature vector is performed for all m and n less than a particular value, and wherein a variation of the candidate feature vector for which the selected m and n result in a sufficiently high fitness as determined by the memetic algorithm is selected as the feature vector.