Facial matching system

• US 10,043,061 B2
• Filed: 08/02/2017
• Issued: 08/07/2018
• Est. Priority Date: 11/16/2015
• Status: Active Grant

• Current Assignee: Idemia Identity & Security USA, LLC (Advent International Corporation)
• Original Assignee: MorphoTrak LLC (Advent International Corporation)
• Inventors: Lo, Peter Zhen-Ping, Chen, Hui
• Primary Examiner(s): Kholdebarin, Iman K
• Application Number:
  US15/667,219
• Publication Number:
  US 20170330028A1
• Time in Patent Office: 370 Days
• Field of Search: 382118
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 10/754 involving a deformation of the sample pattern or of the reference pattern; Elastic matching
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

In some implementations, a computer-implemented method for recognizing facial images may include a multi-stage facial verification process to improve the speed and accuracy of a facial recognition operation. For example, a facial recognition module may include multiple stages where a subset of data is analyzed recursively to improve the speed of the facial recognition processes. The multiple stages may be arranged in a fast-to-slow and coarse-to-fine arrangements such that a match decision may be made at each successive stage.

Claims

1. A method performed by one or more computers, the method comprising:

obtaining data indicating (i) a facial template for an input facial image, and (ii) reference facial templates for multiple reference facial images;

computing, by a first facial matcher, a first set of scores representing respective similarities between the input facial image and each of the obtained reference facial templates;

determining, by the first facial matcher and based on the first set of computed scores, that the input facial image does not match any of the multiple reference facial images;

in response to determining that the input facial image does not match any of the multiple reference facial images, computing, by a second facial matcher, a second set of scores representing respective similarities between the input facial image and each of the obtained reference facial images,

wherein a facial verification technique applied by the second facial matcher has a higher sensitivity than a facial verification technique applied by the first facial matcher;

determining, by the second facial matcher and based at least on the second set of computed scores, that the input facial image matches a particular reference facial image from among the multiple reference facial images;

combining the first set of scores and the second set of scores to compute a match score between the input facial image and the particular reference facial image; and

providing the match score for output.

2. The method of claim 1, wherein:

determining, by the first facial matcher, that the input facial image does not match any of the multiple reference facial images comprises determining that scores included within the first set of scores does not satisfy a first threshold; and

determining, by the second facial matcher, that the input facial image matches the particular reference facial image from among the multiple reference facial images comprises determining that a particular score, representing the respective similarity between the facial template and a reference facial template for the particular reference facial image, satisfies a second threshold.

3. The method of claim 1, wherein a value of the second threshold exceeds a value of the first threshold.

4. The method of claim 1, wherein computing the first set of scores comprises:

identifying, by the first facial matcher, a first set of perturbation points within the input facial image;

generating, by the first facial matcher, a first set of perturbed facial templates for the input facial image based at least on the first set of perturbation points within the input facial image;

comparing, by the first facial matcher, each perturbed template included within the first set of perturbed facial templates to each of the reference facial templates; and

computing, by the first facial matcher, the first set of scores based on the comparison of each perturbed facial template included within the first set of perturbed facial templates and each of the reference facial templates.

5. The method of claim 4, wherein computing the second set of scores comprises:

identifying, by the second facial matcher, a second set of perturbation points within the input facial image;

generating, by the second facial matcher, a second set of perturbed facial templates for the input facial image based at least on the second set of perturbation points within the input facial image;

comparing, by the second facial matcher, each perturbed template included within the second set of perturbed facial templates to each of the reference facial templates; and

computing, by the second facial matcher, the second set of scores based on the comparison of each perturbed facial template included within the second set of perturbed facial templates and each of the reference facial templates.

6. The method of claim 5, wherein a number of perturbed facial template included within the second set of perturbed facial templates exceeds a number of perturbed facial templates included within the first set of perturbed facial templates.

7. The method of claim 5, wherein a number of perturbation points included within the second set of perturbation points exceeds a number of perturbation points included within the first set perturbation points.

8. A system comprising:

one or more computers; and

one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising;

obtaining data indicating (i) a facial template for an input facial image, and (ii) reference facial templates for multiple reference facial images;

computing, by a first facial matcher, a first set of scores representing respective similarities between the input facial image and each of the obtained reference facial templates;

determining, by the first facial matcher and based on the first set of computed scores, that the input facial image does not match any of the multiple reference facial images;

in response to determining that the input facial image does not match any of the multiple reference facial images, computing, by a second facial matcher, a second set of scores representing respective similarities between the input facial image and each of the obtained reference facial images,

wherein a facial verification technique applied by the second facial matcher has a higher sensitivity than a facial verification technique applied by the first facial matcher;

determining, by the second facial matcher and based at least on the second set of computed scores, that the input facial image matches a particular reference facial image from among the multiple reference facial images;

combining the first set of scores and the second set of scores to compute a match score between the input facial image and the particular reference facial image; and

providing the match score for output.

9. The system of claim 8, wherein:

determining, by the first facial matcher, that the input facial image does not match any of the multiple reference facial images comprises determining that scores included within the first set of scores does not satisfy a first threshold; and

determining, by the second facial matcher, that the input facial image matches the particular reference facial image from among the multiple reference facial images comprises determining that a particular score, representing the respective similarity between the facial template and a reference facial template for the particular reference facial image, satisfies a second threshold.

10. The system of claim 8, wherein a value of the second threshold exceeds a value of the first threshold.

11. The system of claim 8, wherein computing the first set of scores comprises:

identifying, by the first facial matcher, a first set of perturbation points within the input facial image;

generating, by the first facial matcher, a first set of perturbed facial templates for the input facial image based at least on the first set of perturbation points within the input facial image;

comparing, by the first facial matcher, each perturbed template included within the first set of perturbed facial templates to each of the reference facial templates; and

computing, by the first facial matcher, the first set of scores based on the comparison of each perturbed facial template included within the first set of perturbed facial templates and each of the reference facial templates.

12. The system of claim 11, wherein computing the second set of scores comprises:

identifying, by the second facial matcher, a second set of perturbation points within the input facial image;

generating, by the second facial matcher, a second set of perturbed facial templates for the input facial image based at least on the second set of perturbation points within the input facial image;

comparing, by the second facial matcher, each perturbed template included within the second set of perturbed facial templates to each of the reference facial templates; and

computing, by the second facial matcher, the second set of scores based on the comparison of each perturbed facial template included within the second set of perturbed facial templates and each of the reference facial templates.

13. The system of claim 12, wherein a number of perturbed facial template included within the second set of perturbed facial templates exceeds a number of perturbed facial templates included within the first set of perturbed facial templates.

14. The system of claim 12, wherein a number of perturbation points included within the second set of perturbation points exceeds a number of perturbation points included within the first set perturbation points.

15. A non-transitory computer-readable storage device encoded with computer program instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

obtaining data indicating (i) a facial template for an input facial image, and (ii) reference facial templates for multiple reference facial images;

computing, by a first facial matcher, a first set of scores representing respective similarities between the input facial image and each of the obtained reference facial templates;

determining, by the first facial matcher and based on the first set of computed scores, that the input facial image does not match any of the multiple reference facial images;

in response to determining that the input facial image does not match any of the multiple reference facial images, computing, by a second facial matcher, a second set of scores representing respective similarities between the input facial image and each of the obtained reference facial images,

wherein a facial verification technique applied by the second facial matcher has a higher sensitivity than a facial verification technique applied by the first facial matcher;

determining, by the second facial matcher and based at least on the second set of computed scores, that the input facial image matches a particular reference facial image from among the multiple reference facial images;

combining the first set of scores and the second set of scores to compute a match score between the input facial image and the particular reference facial image; and

providing the match score for output.

16. The device of claim 15, wherein:

determining, by the first facial matcher, that the input facial image does not match any of the multiple reference facial images comprises determining that scores included within the first set of scores does not satisfy a first threshold; and

determining, by the second facial matcher, that the input facial image matches the particular reference facial image from among the multiple reference facial images comprises determining that a particular score, representing the respective similarity between the facial template and a reference facial template for the particular reference facial image, satisfies a second threshold.

17. The device of claim 15, wherein a value of the second threshold exceeds a value of the first threshold.

18. The device of claim 15, wherein computing the first set of scores comprises:

identifying, by the first facial matcher, a first set of perturbation points within the input facial image;

generating, by the first facial matcher, a first set of perturbed facial templates for the input facial image based at least on the first set of perturbation points within the input facial image;

comparing, by the first facial matcher, each perturbed template included within the first set of perturbed facial templates to each of the reference facial templates; and

computing, by the first facial matcher, the first set of scores based on the comparison of each perturbed facial template included within the first set of perturbed facial templates and each of the reference facial templates.

19. The device of claim 18, wherein computing the second set of scores comprises:

identifying, by the second facial matcher, a second set of perturbation points within the input facial image;

generating, by the second facial matcher, a second set of perturbed facial templates for the input facial image based at least on the second set of perturbation points within the input facial image;

comparing, by the second facial matcher, each perturbed template included within the second set of perturbed facial templates to each of the reference facial templates; and

computing, by the second facial matcher, the second set of scores based on the comparison of each perturbed facial template included within the second set of perturbed facial templates and each of the reference facial templates.

20. The device of claim 19, wherein a number of perturbed facial template included within the second set of perturbed facial templates exceeds a number of perturbed facial templates included within the first set of perturbed facial templates.