Learning model for salient facial region detection

• US 10,579,860 B2
• Filed: 03/03/2017
• Issued: 03/03/2020
• Est. Priority Date: 06/06/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Wang, Xiaolong, Zhou, Yin, Li, Bo, Currey, Jonathan J.
• Primary Examiner(s): Liu, Li
• Application Number:
  US15/449,266
• Publication Number:
  US 20170351905A1
• Time in Patent Office: 1,096 Days
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 40/164 using holistic features
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/175 Static expression
• CPC Class Codes: G06V 40/178 estimating age from face image; using age information for improving recognition

Abstract

One embodiment provides a method comprising receiving a first input image and a second input image. Each input image comprises a facial image of an individual. For each input image, a first set of facial regions of the facial image is distinguished from a second set of facial regions of the facial image based on a learning based model. The first set of facial regions comprises age-invariant facial features, and the second set of facial regions comprises age-sensitive facial features. The method further comprises determining whether the first input image and the second input images comprise facial images of the same individual by performing face verification based on the first set of facial regions of each input image.

Claims

1. A method comprising:

receiving, via a processor, a first input image and a second input image, wherein each input image comprises a facial image of an individual;

for each input image;

based on a learning based model trained to learn different facial features relevant to different facial image analysis tasks and further trained to distinguish different facial regions that include the different facial features, determining, via the processor, a corresponding first set of facial regions of the facial image and a corresponding second set of facial regions of the facial image, wherein the corresponding first set of facial regions comprises at least one facial region including one or more age-invariant facial features, and the corresponding second set of facial regions comprises at least one other facial region including one or more age-sensitive facial features; and

performing, via the processor, at least one of the different facial image analysis tasks based on at least one facial feature extracted from at least one set of facial regions.

2. The method of claim 1, wherein performing, via the processor, at least one of the different facial image analysis tasks based on at least one facial feature extracted from at least one set of facial regions comprises:

determining whether the first input image and the second input image comprise facial images of a same individual by performing face verification based on at least one age-invariant facial feature extracted from each first set of facial regions corresponding to each input image; and

for each input image, estimating an age of the individual captured in the input image by performing age estimation based on at least one age-sensitive facial feature extracted from the second set of facial regions corresponding to the input image.

3. The method of claim 2, wherein the age estimation and the face verification are performed concurrently.

4. The method of claim 1, wherein the learning based model is simultaneously trained to distinguish one or more facial regions comprising one or more age-sensitive facial features from one or more other facial regions comprising one or more age-invariant facial features based on a pair of training images comprising facial images of a same individual at different ages.

5. The method of claim 1, wherein a facial region comprising one or more age-invariant facial features comprises a region of a face that includes one of eyes, nose, or mouth.

6. The method of claim 1, wherein a facial region comprising one or more age-sensitive facial features comprises a region of a face that includes a forehead.

7. The method of claim 1, wherein the learning based model is configured to apply different degrees of face verification including a first degree of face verification and a second degree of face verification, and the first degree of face verification is based more on facial regions comprising age-invariant facial features than the second degree of face verification.

8. The method of claim 1, wherein the learning based model comprises one of:

a deep neural network, or one or more support vector machines.

9. A system, comprising:

at least one processor; and

a non-transitory processor-readable memory device storing instructions that when executed by the at least one processor causes the at least one processor to perform operations including;

receiving a first input image and a second input image, wherein each input image comprises a facial image of an individual;

for each input image;

based on a learning based model trained to learn different facial features relevant to different facial image analysis tasks and further trained to distinguish different facial regions that include the different facial features, determining a corresponding first set of facial regions of the facial image and a corresponding second set of facial regions of the facial image, wherein the corresponding first set of facial regions comprises at least one facial region including one or more age-invariant facial features, and the corresponding second set of facial regions comprises at least one other facial region including one or more age-sensitive facial features; and

performing at least one of the different facial image analysis tasks based on at least one facial feature extracted from at least one set of facial regions.

10. The system of claim 9, wherein performing at least one of the different facial image analysis tasks based on at least one facial feature extracted from at least one set of facial regions comprises:

determining whether the first input image and the second input image comprise facial images of a same individual by performing face verification based on at least one age-invariant facial feature extracted from each first set of facial regions corresponding to each input image; and

for each input image, estimating an age of the individual captured in the input image by performing age estimation based on at least one age-sensitive facial feature extracted from the second set of facial regions corresponding to the input image.

11. The system of claim 10, wherein the age estimation and the face verification are performed concurrently.

12. The system of claim 9, wherein the learning based model is simultaneously trained to distinguish one or more facial regions comprising one or more age-sensitive facial features from one or more other facial regions comprising one or more age-invariant facial features based on a pair of training images comprising facial images of a same individual at different ages.

13. The system of claim 9, wherein the learning based model is configured to apply different degrees of face verification including a first degree of face verification and a second degree of face verification, and the first degree of face verification is based more on facial regions comprising age-invariant facial features than the second degree of face verification.

14. The system of claim 9, wherein the learning based model comprises one of:

a deep neural network, or one or more support vector machines.

15. A non-transitory computer readable storage medium including instructions to perform a method comprising:

receiving a first input image and a second input image, wherein each input image comprises a facial image of an individual;

for each input image;

based on a learning based model trained to learn different facial features relevant to different facial image analysis tasks and further trained to distinguish different facial regions that include the different facial features, determining a corresponding first set of facial regions of the facial image and a corresponding second set of facial regions of the facial image, wherein the corresponding first set of facial regions comprises at least one facial region including one or more age-invariant facial features, and the corresponding second set of facial regions comprises at least one other facial region including one or more age-sensitive facial features; and

performing at least one of the different facial image analysis tasks based on at least one facial feature extracted from at least one set of facial regions.

16. The computer readable storage medium of claim 15, wherein performing at least one of the different facial image analysis tasks based on at least one facial feature extracted from at least one set of facial regions comprises:

determining whether the first input image and the second input image comprise facial images of a same individual by performing face verification based on at least one age-invariant facial feature extracted from each first set of facial regions corresponding to each input image; and

for each input image, estimating an age of the individual captured in the input image by performing age estimation based on at least one age-sensitive facial feature extracted from the second set of facial regions corresponding to the input image.

17. The computer readable storage medium of claim 16, wherein the age estimation and the face verification are performed concurrently.

18. The computer readable storage medium of claim 15, wherein the learning based model is simultaneously trained to distinguish one or more facial regions comprising one or more age-sensitive facial features from one or more other facial regions comprising one or more age-invariant facial features based on a pair of training images comprising facial images of a same individual at different ages.

19. The computer readable storage medium of claim 15, wherein the learning based model is configured to apply different degrees of face verification including a first degree of face verification and a second degree of face verification, and the first degree of face verification is based more on facial regions comprising age-invariant facial features than the second degree of face verification.

20. The computer readable storage medium of claim 15, wherein the learning based model comprises one of:

a deep neural network, or one or more support vector machines.