Facial recognition

• US 9,117,109 B2
• Filed: 06/25/2014
• Issued: 08/25/2015
• Est. Priority Date: 06/26/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Nechyba, Michael Christian, Schneiderman, Henry Will, Sipe, Michael Andrew
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US14/315,100
• Publication Number:
  US 20140307929A1
• Time in Patent Office: 426 Days
• Field of Search: 382/118
• US Class Current: 1/1
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G06V 40/45 Detection of the body part being alive

Abstract

An example method includes receiving a first image and a second image of a face of a user, where one or both images have been granted a match by facial recognition. The method further includes detecting a liveness gesture based on at least one of a yaw angle of the second image relative to the first image and a pitch angle of the second image relative to the first image, where the yaw angle corresponds to a transition along a horizontal axis, and where the pitch angle corresponds to a transition along a vertical axis. The method further includes generating a liveness score based on a yaw angle magnitude and/or a pitch angle magnitude, comparing the liveness score to a threshold value, and determining, based on the comparison, whether to deny authentication to the user with respect to accessing one or more functionalities controlled by the computing device.

Claims

1. A method comprising:

receiving, by a computing device, a first image and a second image;

detecting, by the computing device, a facial gesture at least in part by determining whether a difference between the first image and the second image indicates a facial gesture;

selecting, based at least in part on the detected facial gesture, the first image and the second image from among a plurality of images that includes at least the first image, the second image, and a third image;

subsequent to selecting the first image and the second image, determining, by the computing device, whether the first image and the second image are associated with a consistent identity, based at least in part on a similarity between the first image and the second image; and

determining, based at least in part on detecting the facial gesture and on determining whether the first image and the second image are associated with the consistent identity, whether to deny authentication with respect to accessing one or more functionalities controlled by the computing device.

2. The method of claim 1, wherein the facial gesture comprises a blink gesture.

3. The method of claim 1, wherein the facial gesture comprises at least one of a wink gesture, an ocular movement, a smile gesture, a frown gesture, a tongue protrusion gesture, an open mouth gesture, an eyebrow movement, a forehead wrinkle gesture, and a nose wrinkle gesture.

4. The method of claim 1, further comprising:

identifying, by the computing device, a first facial landmark in the first image;

identifying, by the computing device, a second facial landmark in the second image, the second facial landmark corresponding to the first facial landmark;

extracting, by the computing device, a first sub-image from the first image, wherein the first sub-image represents a portion of the first image, and wherein the first sub-image includes a representation of the first facial landmark; and

extracting, by the computing device, a second sub-image from the second image, wherein the second sub-image represents a portion of the second image, and wherein the second sub-image includes a representation of the second facial landmark,

wherein determining whether the difference between the first image and the second image indicates the facial gesture comprises determining, by the computing device, whether a difference between the first sub-image and the second sub-image indicates the facial gesture.

5. The method of claim 4,

wherein the first facial landmark comprises a first representation of two eyes included in the first sub-image, and

wherein the second facial landmark comprises a second representation of two eyes included in the second sub-image.

6. The method of claim 5, further comprising:

receiving, by the computing device, the third image; and

extracting, by the computing device from the third image, a third sub-image that includes a third facial landmark that includes a third representation of two eyes included in the third sub-image.

7. The method of claim 6, wherein detecting the facial gesture comprises determining whether a transition from the first image to the second image to the third image indicates the facial gesture, at least in part by determining whether the transition indicates an open-shut-open movement of two eyes, based on the first, second, and third sub-images, respectively.

8. The method of claim 6, further comprising determining whether the third image is associated with the consistent user identity, wherein determining whether to grant authentication is based on whether the third image is associated with the consistent user identity.

9. The method of claim 4, wherein the second sub-image is larger than the first image.

10. The method of claim 4, further comprising normalizing, by the computing device, the second sub-image to form a normalized version of the second sub-image,

wherein detecting the facial gesture comprises measuring, by the computing device, a distance between the first sub-image and the normalized version of the second sub-image, at least in part by determining, by the computing device, one or more differences between corresponding pixel locations of the first sub-image and the normalized version of the second sub-image, and

wherein determining whether the difference between the first image and the second image indicates the facial gesture is based at least in part on the measured distance between the first sub-image and the normalized version of the second sub-image.

11. The method of claim 10, wherein determining whether the difference between the first image and the second image indicates a facial gesture based at least in part on the measured distance between the first sub-image and the normalized version of the second sub-image comprises:

comparing, by the computing device, the measured distance to a threshold value; and

if the measured distance is equal to or greater than the threshold value, determining, by the computing device, that the difference between the first image and the second image indicates the facial gesture.

12. The method of claim 10, wherein determining the one or more differences between the corresponding pixel locations of the first sub-image and the normalized version of the second sub-image comprises:

determining, by the computing device, that a first number of pixel transitions included in the first sub-image is less than a second number of pixel transitions included in the normalized version of the second sub-image.

13. The method of claim 10, wherein determining the one or more differences between the corresponding pixel locations of the first sub-image and the normalized version of the second sub-image comprises:

determining, by the computing device, that the first sub-image represents at least a respective iris and a respective cornea of each of two eyes; and

determining, by the computing device, that the normalized version of the second sub-image represents at least a respective eyelid of each of the two eyes.

14. The method of claim 10, wherein the measured distance is a first distance, the method further comprising:

receiving, by the computing device, the third image;

extracting, by the computing device from the third image, a third sub-image that includes a third facial landmark that includes a third representation of two eyes included in the third sub-image; and

measuring, by the computing device, at least one of;

a second distance between the normalized version of the second sub-image and the third sub-image extracted from the third image, and a third distance between the first sub-image and the third sub-image,

wherein detecting the facial gesture is based on one or more of the first distance, the second distance, and the third distance.

15. The method of claim 14, wherein detecting the facial gesture comprises determining that the third distance is less than a threshold distance.

16. The method of claim 1, further comprising normalizing, by the computing device, the second sub-image to form a plurality of normalized versions of the second sub-image, wherein detecting the facial gesture comprises:

measuring, by the computing device, a respective distance between the first sub-image and each normalized version of the second sub-image included in the plurality of normalized versions of the second sub-image; and

selecting, by the computing device, a minimum distance among the respective measured distances,

wherein determining whether the difference between the first image and the second image indicates a facial gesture is based at least in part on the selected minimum distance.

17. A device comprising:

a memory; and

one or more processors operable to;

receive a first image and a second image;

detect a facial gesture at least in part by determining whether a difference between the first image and the second image indicates a facial gesture;

select, based at least in part on the detected facial gesture, the first image and the second image from among a plurality of images that includes at least the first image, the second image, and a third image;

subsequent to selecting the first image and the second image, determine whether the first image and the second image are associated with a consistent identity, based at least in part on a similarity between the first image and the second image; and

determine, based at least in part on detecting the facial gesture and on determining whether the first image and the second image are associated with the consistent identity, whether to deny authentication with respect to accessing one or more functionalities controlled by the computing device.

18. The device of claim 17, wherein the facial gesture comprises a blink gesture.

19. The device of claim 17, wherein the one or more processors are further operable to:

identify a first facial landmark in the first image;

identify a second facial landmark in the second image, the second facial landmark corresponding to the first facial landmark;

extract a first sub-image from the first image, wherein the first sub-image represents a portion of the first image, and wherein the first sub-image includes a representation of the first facial landmark; and

extract a second sub-image from the second image, wherein the second sub-image represents a portion of the second image, and wherein the second sub-image includes a representation of the second facial landmark,

wherein, to determine whether the difference between the first image and the second image indicates the facial gesture, the one or more processors are configured to determine whether a difference between the first sub-image and the second sub-image indicates the facial gesture.

20. A computer-readable storage device encoded with instructions that, when executed, cause one or more programmable processors of a computing device to:

receive a first image and a second image;

detect a facial gesture at least in part by determining whether a difference between the first image and the second image indicates a facial gesture;

select, based at least in part on the detected facial gesture, the first image and the second image from among a plurality of images that includes at least the first image, the second image, and a third image;

subsequent to selecting the first image and the second image, determine whether the first image and the second image are associated with a consistent identity, based at least in part on a similarity between the first image and the second image; and

determine, based at least in part on detecting the facial gesture and on determining whether the first image and the second image are associated with the consistent identity, whether to deny authentication with respect to accessing one or more functionalities controlled by the computing device.