Facial recognition

• US 8,542,879 B1
• Filed: 06/26/2012
• Issued: 09/24/2013
• Est. Priority Date: 06/26/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Nechyba, Michael Christian, Schneiderman, Henry Will, Sipe, Michael Andrew
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US13/533,834
• Time in Patent Office: 455 Days
• Field of Search: 382/103, 382115-127, 382/118, 382/181, 382/289, 382/218, 340/5.1, 340/5.2, 340/5.3, 340/5.52, 340/5.53, 340/5.82, 340/5.83, 902/3, 902/6
• US Class Current: 382/103
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G06V 40/45 Detection of the body part being alive

Abstract

An example method includes receiving a first image and a second image of a face of a user, where one or both images have been granted a match by facial recognition. The method further includes detecting a liveness gesture based on at least one of a yaw angle of the second image relative to the first image and a pitch angle of the second image relative to the first image, where the yaw angle corresponds to a transition along a horizontal axis, and where the pitch angle corresponds to a transition along a vertical axis. The method further includes generating a liveness score based on a yaw angle magnitude and/or a pitch angle magnitude, comparing the liveness score to a threshold value, and determining, based on the comparison, whether to deny authentication to the user with respect to accessing one or more functionalities controlled by the computing device.

Claims

1. A method comprising:

receiving, from an image capture device coupled to a computing device, a first image of a face of a user, and a second image of the face of the user;

detecting a liveness gesture based on at least one of a yaw angle of the second image relative to the first image and a pitch angle of the second image relative to the first image, wherein the yaw angle corresponds to a transition centered around a vertical axis, and wherein the pitch angle corresponds to a transition centered around a horizontal axis;

generating a liveness score based on at least one of a yaw angle magnitude associated with the yaw angle and a pitch angle magnitude associated with the pitch angle;

comparing the liveness score to a threshold value; and

determining, based at least in part on the comparison, whether to deny authentication to the user with respect to accessing one or more functionalities controlled by the computing device.

2. The method of claim 1, further comprising:

prompting the user to perform a predetermined liveness gesture;

determining whether the detected liveness gesture matches the predetermined liveness gesture;

if the detected liveness gesture does not match the predetermined liveness gesture, denying authentication to the user with respect to accessing one or more functionalities controlled by the computing device; and

if the detected liveness gesture matches the predetermined liveness gesture and the liveness score exceeds the threshold value, granting authentication to the user with respect to accessing one or more functionalities controlled by the computing device.

3. The method of claim 1, wherein detecting the liveness gesture further comprises detecting the liveness gesture based at least in part on at least one landmark represented in one or both of the first image and the second image.

4. The method of claim 3, wherein detecting the liveness gesture based at least in part on the at least one landmark represented in one or both of the first image and the second image further comprises:

identifying both a left eye and a right eye in the first image;

identifying only one of the left eye and the right eye in the second image; and

obtaining the yaw angle of the second image relative to the first image.

5. The method of claim 3, wherein detecting the liveness gesture based at least in part on the at least one landmark represented in one or both of the first image and the second image further comprises:

identifying both a left eye and a right eye in the first image;

determining that either the left eye or the right eye is absent from the second image; and

obtaining the yaw angle of the second image relative to the first image.

6. The method of claim 3, wherein detecting the liveness gesture based at least in part on the at least one landmark represented in one or both of the first image and the second image further comprises:

detecting an absence of a representation of an earlobe in the first image;

detecting the representation of the earlobe in the second image; and

obtaining the yaw angle of the second image relative to the first image.

7. The method of claim 3, wherein detecting the liveness gesture based at least in part on the at least one landmark represented in one or both of the first image and the second image further comprises:

detecting a representation of a nose base in the first image;

detecting an absence of the representation of the nose base in the second image; and

obtaining the yaw angle of the second image relative to the first image.

8. The method of claim 3, wherein detecting the liveness gesture based at least in part on the at least one landmark represented in one or both of the first image and the second image further comprises:

detecting an absence of a representation of a nose bridge in the first image;

detecting the representation of the nose bridge in the second image; and

obtaining the yaw angle of the second image relative to the first image.

9. The method of claim 1,

wherein the image capture device has video capture capabilities, and

wherein at least one of the first image and the second image form a portion of video data received from the image capture device.

10. The method of claim 1, further comprising:

performing facial recognition analysis on each of the first image and the second image;

if at least one of the first image and second image fail the facial recognition analysis, denying authentication to the user with respect to accessing one or more functionalities controlled by the computing device; and

if both of the first image and the second image pass the facial recognition analysis and the liveness score exceeds the threshold value, granting authentication to the user with respect to accessing one or more functionalities controlled by the computing device.

11. The method of claim 1, wherein detecting the liveness gesture comprises:

determining a first head orientation associated with the first image and a second head orientation associated with the second image.

12. The method of claim 11, wherein determining the first head orientation and the second head orientation comprises:

comparing a representation of a face of the user in each of the first image and the second image to at least one head pose model.

13. The method of claim 12, wherein determining at least one of the first head orientation and the second head orientation comprises determining based at least in part on the yaw angle of the second image relative to the first image, and wherein the at least one head pose model conforms to one of a frontal orientation, a partial profile orientation, and a profile orientation.

14. The method of claim 13, wherein the partial profile orientation displays an angle of between 35 degrees and 55 degrees from the frontal orientation, and wherein the profile orientation displays an angle of between 80 degrees and 100 degrees from the frontal orientation.

15. The method of claim 12, wherein determining at least one of the first head orientation and the second head orientation is based at least in part on the pitch angle between the second image and the first image, and wherein the at least one head pose model conforms to one of a frontal orientation, an upward-facing orientation, and a downward-facing orientation.

16. The method of claim 1, wherein detecting the liveness gesture comprises:

detecting, using a three-dimensional (3-D) rangefinder coupled to the computing device, at least one of the yaw angle and the pitch angle.

17. The method of claim 1, further comprising:

granting a match by facial recognition of one or both of the first image and the second image based at least in part on comparing one or both of the first image and the second image to at least one enrollment image of the face of an authorized user.

18. The method of claim 1, wherein receiving the first image and the second image further comprises selecting the first image and the second image from a plurality of images that includes at least the first image, the second image, and a third image.

19. The method of claim 18, wherein selecting the first image and the second image from the plurality of images comprises selecting the first image and the second image based at least in part on comparing the liveness score to a threshold value.

20. The method of claim 1, wherein determining whether to deny authentication to the user comprises denying the authentication irrespective of a match by facial recognition with respect to one or both of the first image of the face of the user and the second image of the face of the user.

21. The method of claim 1, further comprising:

determining, by the computing device, whether the first image displays at least a threshold similarity relative to the second image, wherein the threshold similarity indicates a consistent user identity.

22. A computer-readable storage device encoded with instructions that, when executed, cause one or more processors of a computing device to perform operations comprising:

receiving, from an image capture device coupled to the computing device, a first image of a face of a user, and a second image of the face of the user;

detecting a liveness gesture based on at least one of a yaw angle of the second image relative to the first image and a pitch angle of the second image relative to the first image, wherein the yaw angle corresponds to a transition centered around a vertical axis, and wherein the pitch angle corresponds to a transition centered around a horizontal axis;

generating a liveness score based on at least one of a yaw angle magnitude associated with the yaw angle and a pitch angle magnitude associated with the pitch angle;

comparing the liveness score to a threshold value; and

determining, based at least in part on the comparison, whether to deny authentication to the user with respect to accessing one or more functionalities controlled by the computing device.

23. A computing device comprising:

a memory,

at least one processor;

an image capture device operable by the at least one processor to capture a first image of a face of a user, and a second image of the face of the user;

a facial recognition module operable by the at least one processor to grant a match, by facial recognition, to at least one of the first image and the second image; and

an anti-spoofing module operable by the at least one processor to;

detect a liveness gesture based on at least one of a yaw angle of the second image relative to the first image and a pitch angle of the second image relative to the first image, wherein the yaw angle corresponds to a transition centered around a vertical axis, and wherein the pitch angle corresponds to a transition centered around a horizontal axis;

generate a liveness score based on at least one of a yaw angle magnitude associated with the yaw angle and a pitch angle magnitude associated with the pitch angle;

compare the liveness score to a threshold value; and

determine, based at least in part on the comparison, whether to deny authentication to the user with respect to accessing one or more functionalities controlled by the computing device.