Face detection, representation, and recognition

• US 10,043,058 B2
• Filed: 03/09/2016
• Issued: 08/07/2018
• Est. Priority Date: 03/09/2016
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group), Kia Corp.
• Original Assignee: International Business Machines Corporation
• Inventors: Ahmed, Mohamed N.
• Primary Examiner(s): Thomas, Mia M
• Application Number:
  US15/065,021
• Publication Number:
  US 20170262695A1
• Time in Patent Office: 881 Days
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

In an approach to face recognition in an image, one or more computer processors receive an image that includes at least one face and one or more face parts. The one or more computer processors detect the one or more face parts in the image with a face component model. The one or more computer processors cluster the detected one or more face parts with one or more stored images. The one or more computer processors extract, from the clustered images, one or more face descriptors. The one or more computer processors determine a recognition score of the at least one face, based, at least in part, on the extracted one or more face descriptors.

Claims

1. A method for face recognition in an image, the method comprising:

receiving, by one or more computer processors, an image that includes at least one face and one or more face parts;

normalizing, by the one or more computer processors, the received image;

comparing, by the one or more computer processors, the normalized image to one or more existing templates of facial features in a first a face descriptor of a plurality of face descriptors;

determining, by the one or more computer processors, whether a minimum distance between the normalized image and the one or more existing templates exceeds a threshold;

responsive to determining the minimum distance between the normalized image and the one or more existing templates exceeds a threshold, determining, by the one or more computer processors, whether a quantity of the one or more existing templates is less than a pre-defined maximum quantity;

responsive to determining the quantity of the one or more existing templates is less than a pre-defined maximum quantity, creating, by the one or more computer processors, a new template from the normalized image, wherein the new template is appended to the one or more existing templates in the first face descriptor;

detecting, by the one or more computer processors, the one or more face parts in the image with a face component model;

clustering, by the one or more computer processors, the detected one or more face parts with one or more stored images;

extracting, by the one or more computer processors, from the clustered images, one or more face descriptors of the plurality of face descriptors; and

determining, by the one or more computer processors, a recognition score of the at least one face, based, at least in part, on the extracted one or more face descriptors.

2. The method of claim 1, wherein detecting the one or more face parts with a face component model further comprises:

applying, by the one or more computer processors, a root filter to the image;

initializing, by the one or more computer processors, a set of the one or more face parts;

determining, by the one or more computer processors, whether a presence of one or more occluding objects is detected in the image that exceeds a threshold; and

responsive to determining the presence of one or more occluding objects is detected in the image that exceeds a threshold, adding, by the one or more computer processors, the one or more occluding objects to the set of one or more face parts.

3. The method of claim 2, further comprising:

determining, by the one or more computer processors, whether the one or more occluding objects overlap one or more face parts in the image; and

responsive to determining the one or more occluding objects overlap at least one of the one or more face parts in the image, removing, by the one or more computer processors, the one or more overlapped face parts from the set the one or more of face parts.

4. The method of claim 1, further comprising, responsive to determining the quantity of the one or more existing templates is not less than a pre-defined maximum quantity, updating, by the one or more computer processors, a first template of the one or more existing templates, wherein the first template most closely matches the normalized image.

5. The method of claim 1, wherein determining the recognition score of the at least one face further comprises using, by the one or more computer processors, a trained convolution deep neural network.

6. The method of claim 5, wherein the trained convolution deep neural network uses as input at least one of the one or more face descriptors of the plurality of face descriptors or one or more occlusion maps.

7. The method of claim 1, wherein the received image includes at least one object that occludes at least one of the one or more face parts.

8. A computer program product for face recognition in an image, the computer program product comprising:

one or more computer readable storage devices and program instructions stored on the one or more computer readable storage devices, the stored program instructions comprising;

program instructions to receive an image that includes at least one face and one or more face parts;

program instructions to normalize the received image;

program instructions to compare the normalized image to one or more existing templates of facial features in a first a face descriptor of a plurality of face descriptors;

program instructions to determine whether a minimum distance between the normalized image and the one or more existing templates exceeds a threshold;

responsive to determining the minimum distance between the normalized image and the one or more existing templates exceeds a threshold, program instructions to determine whether a quantity of the one or more existing templates is less than a pre-defined maximum quantity;

responsive to determining the quantity of the one or more existing templates is less than a pre-defined maximum quantity, program instructions to create a new template from the normalized image, wherein the new template is appended to the one or more existing templates in the first face descriptor;

program instructions to detect the one or more face parts in the image with a face component model;

program instructions to cluster the detected one or more face parts with one or more stored images;

program instructions to extract, from the clustered images, one or more face descriptors of the plurality of face descriptors; and

program instructions to determine a recognition score of the at least one face, based, at least in part, on the extracted one or more face descriptors.

9. The computer program product of claim 8, wherein the program instructions to detect the one or more face parts with a face component model comprise:

program instructions to apply a root filter to the image;

program instructions to initialize a set of the one or more face parts;

program instructions to determine whether a presence of one or more occluding objects is detected in the image that exceeds a threshold; and

responsive to determining the presence of one or more occluding objects is detected in the image that exceeds a threshold, program instructions to add the one or more occluding objects to the set of one or more face parts.

10. The computer program product of claim 9, the stored program instructions further comprising:

program instructions to determine whether the one or more occluding objects overlap one or more face parts in the image; and

responsive to determining the one or more occluding objects overlap at least one of the one or more face parts in the image, program instructions to remove the one or more overlapped face parts from the set the one or more of face parts.

11. The computer program product of claim 8, the stored program instructions further comprising, responsive to determining the quantity of the one or more existing templates is not less than a pre-defined maximum quantity, program instructions to update a first template of the one or more existing templates, wherein the first template most closely matches the normalized image.

12. The computer program product of claim 8, wherein the program instructions to determine the recognition score of the at least one face comprise program instructions to use a trained convolution deep neural network.

13. A computer system for face recognition in an image, the computer system comprising:

one or more computer processors;

one or more computer readable storage devices;

program instructions stored on the one or more computer readable storage devices for execution by at least one of the one or more computer processors, the stored program instructions comprising;

program instructions to receive an image that includes at least one face and one or more face parts;

program instructions to normalize the received image;

program instructions to compare the normalized image to one or more existing templates of facial features in a first a face descriptor of a plurality of face descriptors;

program instructions to determine whether a minimum distance between the normalized image and the one or more existing templates exceeds a threshold;

responsive to determining the minimum distance between the normalized image and the one or more existing templates exceeds a threshold, program instructions to determine whether a quantity of the one or more existing templates is less than a pre-defined maximum quantity;

responsive to determining the quantity of the one or more existing templates is less than a pre-defined maximum quantity, program instructions to create a new template from the normalized image, wherein the new template is appended to the one or more existing templates in the first face descriptor;

program instructions to detect the one or more face parts in the image with a face component model;

program instructions to cluster the detected one or more face parts with one or more stored images;

program instructions to extract, from the clustered images, one or more face descriptors of the plurality of face descriptors; and

program instructions to determine a recognition score of the at least one face, based, at least in part, on the extracted one or more face descriptors.

14. The computer system of claim 13, wherein the program instructions to detect the one or more face parts with a face component model comprise:

program instructions to apply a root filter to the image;

program instructions to initialize a set of the one or more face parts;

program instructions to determine whether a presence of one or more occluding objects is detected in the image that exceeds a threshold; and

responsive to determining the presence of one or more occluding objects is detected in the image that exceeds a threshold, program instructions to add the one or more occluding objects to the set of one or more face parts.

15. The computer system of claim 14, the stored program instructions further comprising:

program instructions to determine whether the one or more occluding objects overlap one or more face parts in the image; and

responsive to determining the one or more occluding objects overlap at least one of the one or more face parts in the image, program instructions to remove the one or more overlapped face parts from the set the one or more of face parts.

16. The computer system of claim 13, the stored program instructions further comprising, responsive to determining the quantity of the one or more existing templates is not less than a pre-defined maximum quantity, program instructions to update a first template of the one or more existing templates, wherein the first template most closely matches the normalized image.

17. The computer system of claim 13, wherein the program instructions to determine the recognition score of the at least one face comprise program instructions to use a trained convolution deep neural network.