Face Recognition Apparatus and Methods

• US 20120230545A1
• Filed: 11/30/2009
• Published: 09/13/2012
• Est. Priority Date: 11/30/2009
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Zhang, Tong, Zhang, Wei
• Granted Patent: US 8,818,034 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/285 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06V 10/87 using selection of the recognition techniques, e.g. of a classifier in a multiple classifier system
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

One or more facial recognition categories are assigned to a face region detected in an input image (24). Each of the facial recognition categories is associated with a respective set of one or more different feature extraction modules (66) and a respective set of one or more different facial recognition matching modules (76). For each of the facial recognition categories assigned to the face region, the input image (24) is processed with each of the feature extraction modules (66) associated with the facial recognition category to produce a respective facial region descriptor vector of facial region descriptor values characterizing the face region. A recognition result (96) between the face region and a reference face image (28) is determined based on application of the one or more facial recognition matching modules (76) associated with the facial recognition categories assigned to the face region to the facial region descriptor vectors produced for the face region detected in the input image (24).

Claims

1. A method, comprising:

detecting a face region in an input image (24);

assigning to the face region one or more facial recognition categories each of which is associated with a respective set of one or more different feature extraction modules (66) and a respective set of one or more different facial recognition matching modules (76);

for each of the facial recognition categories assigned to the face region, processing the input image (24) with each of the feature extraction modules (66) associated with the facial recognition category to produce a respective facial region descriptor vector of facial region descriptor values characterizing the face region; and

determining a recognition result (96) between the face region and a reference face image (28) based on application of the facial recognition matching modules (76) associated with the facial recognition categories assigned to the face region to the facial region descriptor vectors produced for the face region detected in the input image (24);

wherein the detecting, the assigning, the processing, and the determining are performed by a computer (140).

2. The method of claim 1, further comprising ascertaining one or more facial landmarks in the face region, wherein the assigning comprises classifying the face region into at least one of the one or more facial recognition categories based on the one or more detected facial landmarks.

3. The method of claim 2, wherein the classifying comprises estimating a pose of the face region based on the one or more ascertained facial landmarks, and classifying the face region into a respective one of the facial recognition categories based on the estimated pose of the face region.

4. The method of claim 2, wherein the ascertaining comprises determining a respective location of one or more eyes in the face region and a respective eye location confidence score associated with each of the one or more determined eye locations.

5. The method of claim 4, wherein:

in response to a determination that the one or more eye location confidence scores satisfy a specified predicate, the assigning comprises detecting sunglasses in the face region; and

in response to a detection of sunglasses in the face region, the assigning comprises classifying the face region into a sunglasses-based facial recognition category.

6. The method of claim 5, wherein the processing comprises excluding one or more regions associated with detected sunglasses in the face region from the processing of the input image (24) with each of the feature extraction modules (66) associated with the sunglasses-based facial recognition category.

7. The method of claim 1, wherein the determining comprises for each of the facial recognition categories assigned to the face region, applying the associated set of one or more facial recognition matching modules (76) to the facial region descriptor vectors produced from the input image (24) for the facial recognition category and corresponding facial region descriptor vectors produced from the reference face image (28) to generate a respective matching-module-specific matching score.

8. The method of claim 7, wherein, in response to a determination that an estimated pose of the face region is different from an estimated pose of the reference face image (28), the applying comprises generating a matching-module-specific matching score based on a warping of the facial region descriptor vectors respectively produced for at least one of the input image (24) and the reference face image (28) to a common pose orientation in accordance with a three-dimensional virtual view facial recognition matching module process.

9. The method of claim 7, wherein the applying comprises for each of the facial recognition categories, selecting a different respective set of one or more facial recognition matching modules (76) from a set comprising:

a spatial visual codebook based facial recognition matching module (78);

a manifold learning based facial recognition matching module (80);

a sunglasses based facial recognition matching module (82); and

a three-dimensional virtual view based facial recognition matching module (84).

10. The method of claim 7, wherein the determining comprises, in response to the generation of multiple matching-module-specific matching scores, determining an output matching score (96) based on a combination of the matching-module-specific matching scores.

11. The method of claim 1, wherein the assigning comprises selecting the one or more facial recognition categories assigned to the face region from a set comprising:

a pose based facial recognition category;

a sunglasses based facial recognition category;

an illumination based facial recognition category;

an expression based facial recognition category; and

an aging based facial recognition category.

12. The method of claim 1, wherein the determining comprises determining an output matching score (96) indicating a degree of match between the face region and a reference face image (28).

13. Apparatus, comprising:

a computer-readable medium (144, 148) storing computer-readable instructions; and

a data processor (142) coupled to the computer-readable medium, operable to execute the instructions, and based at least in part on the execution of the instructions operable to perform operations comprising

detecting a face region in an input image (24);

assigning to the face region one or more facial recognition categories each of which is associated with a respective set of one or more different feature extraction modules (66) and a respective set of one or more different facial recognition matching modules (76);

for each of the facial recognition categories assigned to the face region, processing the input image (24) with each of the feature extraction modules (66) associated with the facial recognition category to produce a respective facial region descriptor vector of facial region descriptor values characterizing the face region; and

determining a recognition result between the face region and a reference face image (28) based on application of the facial recognition matching modules (76) associated with the facial recognition categories assigned to the face region to the facial region descriptor vectors produced for the face region detected in the input image (24)

14. The apparatus of claim 13, wherein based at least in part on the execution of the instructions the data processor is operable to perform operations comprising ascertaining one or more facial landmarks in the face region, wherein in the assigning the data processor is operable to perform operations comprising classifying the face region into at least one of the one or more facial recognition categories based on the one or more detected facial landmarks.

15. The apparatus of claim 13, wherein based at least in part on the execution of the instructions the data processor is operable to perform operations comprising determining a respective location of one or more eyes in the face region and a respective eye location confidence score associated with each of the one or more determined eye locations;

and

wherein;

in response to a determination that the one or more eye location confidence scores satisfy a specified predicate, the data processor performs operations comprising detecting sunglasses in the face region; and

in response to a detection of sunglasses in the face region, the data processor performs operations comprising classifying the face region into a sunglasses-based facial recognition category.

16. The apparatus of claim 13, wherein in the determining the data processor is operable to perform operations comprising, for each of the facial recognition categories assigned to the face region, applying the associated set of one or more facial recognition matching modules (76) to the facial region descriptor vectors produced from the input image (24) for the facial recognition category and corresponding facial region descriptor vectors produced from the reference face image (28) to generate a respective matching-module-specific matching score.

17. At least one computer-readable medium (144, 148) having computer-readable program code embodied therein, the computer-readable program code adapted to be executed by a computer (140) to implement a method comprising:

detecting a face region in an input image (24);

assigning to the face region one or more facial recognition categories each of which is associated with a respective set of one or more different feature extraction modules (66) and a respective set of one or more different facial recognition matching modules (76);

for each of the facial recognition categories assigned to the face region, processing the input image (24) with each of the feature extraction modules (66) associated with the facial recognition category to produce a respective facial region descriptor vector of facial region descriptor values characterizing the face region; and

determining a recognition result between the face region and a reference face image (28) based on application of the facial recognition matching modules (76) associated with the facial recognition categories assigned to the face region to the facial region descriptor vectors produced for the face region detected in the input image (24).

18. The at least one computer-readable medium of claim 17, wherein the method further comprises ascertaining one or more facial landmarks in the face region, wherein the assigning comprises classifying the face region into at least one of the one or more facial recognition categories based on the one or more detected facial landmarks.

19. The at least one computer-readable medium of claim 17, wherein the method further comprises determining a respective location of one or more eyes in the face region and a respective eye location confidence score associated with each of the one or more determined eye locations;

and

wherein;

in response to a determination that the one or more eye location confidence scores satisfy a specified predicate, the assigning comprises detecting sunglasses in the face region; and

in response to a detection of sunglasses in the face region, the assigning comprises classifying the face region into a sunglasses-based facial recognition category.

20. The at least one computer-readable medium of claim 17, wherein the determining comprises for each of the facial recognition categories assigned to the face region, applying the associated set of one or more facial recognition matching modules (76) to the facial region descriptor vectors produced from the input image (24) for the facial recognition category and corresponding facial region descriptor vectors produced from the reference face image (28) to generate a respective matching-module-specific matching score.