Facial feature detection

• US 8,515,139 B1
• Filed: 03/15/2012
• Issued: 08/20/2013
• Est. Priority Date: 03/15/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Nechyba, Michael Christian, Sipe, Michael Andrew
• Primary Examiner(s): Park, Edward
• Application Number:
  US13/421,668
• Time in Patent Office: 523 Days
• Field of Search: 382/115, 382/118, 351/204, 340/5.1, 340/5.92, 704/246
• US Class Current: 382/118
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

An example method includes capturing, by a camera of a computing device, an image including at least a face of a user, calculating a face template of the face of the user in the image, and analyzing the face template to determine whether the face includes at least one of a removable facial feature that decreases a level of distinctiveness between two faces and a non-removable facial feature that decreases a level of distinctiveness between two faces. When the face includes the removable facial feature, the method further includes outputting a notification for the user to remove the removable facial feature. When the face includes the non-removable facial feature, the method further includes adjusting a first similarity score threshold to a second similarity score threshold.

Claims

1. A method, comprising:

capturing, by a computing device, an image including at least a face of a user;

calculating, by the computing device, a face template of the face of the user in the image;

analyzing, by the computing device, the face template to determine whether the face includes at least one of a removable facial feature that decreases a level of distinctiveness between two faces and a non-removable facial feature that decreases a level of distinctiveness between two faces,

wherein analyzing the face template to determine whether the face includes the removable facial feature that decreases the level of distinctiveness between two faces further comprises;

determining, by the computing device, a plurality of removable facial feature similarity scores between the face template and one or more removable facial feature templates; and

calculating, by the computing device, a weighted sum of the plurality of removable facial feature similarity scores,

wherein when the weighted sum of the plurality of removable facial feature similarity scores is positive, determining that the face includes the removable facial feature, and

wherein when the weighted sum of the plurality of removable facial feature similarity scores is negative, determining that the face does not include the removable facial feature,

when the face includes the removable facial feature, the method comprises;

outputting, by the computing device, a notification for the user to remove the removable facial feature; and

when the face includes the non-removable facial feature, the method comprises;

adjusting, by the computing device, a first similarity score threshold to a second similarity score threshold.

2. The method of claim 1, further including:

storing a group of removable facial feature templates, wherein the group of removable facial feature templates includes one or more positive removable facial feature templates and one or more negative removable facial feature templates, wherein the positive removable facial feature templates are images of people having the removable facial feature, and wherein the negative removable facial feature templates are images of people not having the removable facial feature.

3. The method of claim 2, wherein analyzing the face template to determine whether the face includes the removable facial feature that decreases the level of distinctiveness between two faces further comprises:

comparing, by the computing device, the face template to the group of removable facial feature templates,

wherein determining the plurality of removable facial feature similarity scores comprises determining, by the computing device, the plurality of removable facial feature similarity scores between the face template and one or more of the positive removable facial feature templates and one or more of the negative removable facial feature templates.

4. The method of claim 1, wherein the image is a first image and the face template is a first face template, and wherein when the face includes the removable facial feature, the method further comprises:

capturing, by the computing device, a second image including the face of the user;

calculating, by the computing device, a second face template of the face of the user in the second image;

calculating a similarity score between the second face template of the user in the second image and an enrolled template of the user; and

determining, by the computing device, whether the similarity score is above the first similarity score threshold.

5. The method of claim 4, wherein the enrolled template of the user is calculated from a previously captured image of the user.

6. The method of claim 4, upon determining that the similarity score is greater than the first similarity score threshold, granting authentication by facial recognition.

7. The method of claim 4, upon determining that the similarity score is less than the first similarity score threshold, denying authentication by facial recognition.

8. The method of claim 1, further including:

storing a group of non-removable facial feature templates, wherein the group of non-removable facial feature templates includes one or more positive non-removable facial feature templates and one or more negative non-removable facial feature templates, and wherein the positive non-removable facial feature templates are images of people having the non-removable facial feature, and wherein the negative removable facial feature templates are images of people not having the non-removable facial feature.

9. The method of claim 8,

wherein analyzing the face template to determine whether the face includes the non-removable facial feature that decreases the level of distinctiveness between two faces further comprises:

comparing, by the computing device, the face template to the group of non-removable facial feature templates;

determining, by the computing device, a plurality of non-removable facial feature similarity scores between the face template and one or more of the positive non-removable facial feature templates and each one or more of the negative non-removable facial feature templates; and

calculating, by the computing device, a weighted sum of the plurality of non-removable facial feature similarity scores,

wherein when the weighted sum of the plurality of non-removable facial feature similarity scores is positive, determining that the face includes the non-removable facial feature, and

wherein when the weighted sum of the plurality of non-removable facial feature similarity scores is negative, determining that the face does not include the non-removable facial feature.

10. The method of claim 1, wherein when the face includes the non-removable facial feature, the method further comprises:

calculating a similarity score between the face template of the user in the image and an enrolled template of the user; and

determining, by the computing device, whether the similarity score is above the second similarity score threshold.

11. The method of claim 10, wherein when the similarity score is above the second similarity score threshold, the method further comprises:

granting authentication by facial recognition; and

transitioning the computing device from a locked state to an unlocked state, and

wherein when the similarity score is below the second similarity score threshold, the method further comprises;

denying authentication by facial recognition; and

preventing the computing device from transitioning from a locked state to an unlocked state.

12. The method of claim 10, further comprises:

determining, by the computing device, when an authentication session is complete; and

responsive to determining the authentication session is complete, adjusting the second similarity score threshold back to the first similarity score threshold.

13. The method of claim 12, wherein determining when the authentication session is complete further comprises:

determining, by the computing device, whether authentication by facial recognition was at least one of denied and granted.

14. The method of claim 1, wherein when the first face template does not include the removable facial feature and the non-removable facial feature, the method further comprises:

calculating, by the computing device, a similarity score between the face template of the user in the image and an enrolled template of the user; and

determining, by the computing device, whether the similarity score is above the first similarity score threshold,

when the similarity score is above the first similarity score threshold, the method further comprises;

granting authentication by facial recognition; and

transitioning the computing device from a locked state to an unlocked state; and

when the similarity score is below the first similarity score threshold, the method further comprises;

denying authentication by facial recognition; and

preventing the computing device from transitioning from a locked state to an unlocked state.

15. The method of claim 1, wherein analyzing the first face template to determine whether the face includes at least one of a removable facial feature that decreases a level of distinctiveness between two faces and a non-removable facial feature that decreases a level of distinctiveness between two faces comprises analyzing the first face template using a facial recognition module, and wherein calculating the similarity score between the first face template of the user in the first image and the enrolled template of the user comprises calculating by the facial recognition module.

16. The method of claim 1, wherein the removable facial feature comprises sunglasses.

17. The method of claim 1, wherein the non-removable facial feature comprises facial hair.

18. The method of claim 1, wherein the second similarity score threshold is greater than the first similarity score threshold.

19. A computing device comprising:

at least one processor;

at least one camera operable by the at least one processor to capture a first image including at least a first face of a user; and

at least one output device, wherein the at least one processor is configured to calculate a first face template of the face of the user in the first image,

wherein the at least one processor is configured to analyze the first face template to determine whether the face includes at least one of a removable facial feature that decreases a level of distinctiveness between two faces and a non-removable facial feature that decreases a level of distinctiveness between two faces, wherein analyzing the face template to determine whether the face includes the removable facial feature that decreases the level of distinctiveness between two faces further comprises;

determining, by the at least one processor, a plurality of removable facial feature similarity scores between the face template and a removable facial feature templates; and

calculating, by the at least one processor, a weighted sum of the plurality of removable facial feature similarity scores,

wherein when the weighted sum of the plurality of removable facial feature similarity scores is positive, determining that the face includes the removable facial feature, and

wherein when the weighted sum of the plurality of removable facial feature similarity scores is negative, determining that the face does not include the removable facial feature, wherein when the face includes the removable facial feature, the at least one output device is configured to output a notification for the user to remove the removable facial feature, and wherein when the face includes the non-removable facial feature, the at least one processor adjusts a first similarity score threshold to a second similarity score threshold.

20. A method, comprising:

capturing, by a computing device, an image including at least a face of a user;

calculating, by the computing device, a face template of the face of the user in the image;

analyzing, by the computing device, the face template to determine whether the face includes a non-removable facial feature that decreases a level of distinctiveness between two faces,

wherein analyzing the face template to determine whether the face includes the non-removable facial feature that decreases the level of distinctiveness between two faces further comprises;

determining, by the computing device, a plurality of non-removable facial feature similarity scores between the face template and one or more non-removable facial feature templates; and

calculating, by the computing device, a weighted sum of the plurality of non-removable facial feature similarity scores,

wherein when the weighted sum of the plurality of non-removable facial feature similarity scores is positive, determining that the face includes the non-removable facial feature, and

wherein when the weighted sum of the plurality of non-removable facial feature similarity scores is negative, determining that the face does not include the non-removable facial feature; and

adjusting, by the computing device and in response to determining that the face includes the non-removable facial feature, a first similarity score threshold to a second similarity score threshold.

21. A method, comprising:

capturing, by a computing device, an image including at least a face of a user;

calculating, by the computing device, a face template of the face of the user in the image;

analyzing, by the computing device, the face template to determine whether the face includes a removable facial feature that decreases a level of distinctiveness between two faces,

wherein analyzing the face template to determine whether the face includes the removable facial feature that decreases the level of distinctiveness between two faces further comprises;

determining, by the computing device, a plurality of removable facial feature similarity scores between the face template and a removable facial feature templates; and

calculating, by the computing device, a weighted sum of the plurality of removable facial feature similarity scores,

wherein when the weighted sum of the plurality of removable facial feature similarity scores is positive, determining that the face includes the removable facial feature, and

wherein when the weighted sum of the plurality of removable facial feature similarity scores is negative, determining that the face does not include the removable facial feature; and

outputting, by the computing device and in response to determining that the face includes the removable facial feature, a notification for the user to remove the removable facial feature.