Facial recognition

• US 8,798,336 B2
• Filed: 09/23/2013
• Issued: 08/05/2014
• Est. Priority Date: 06/26/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Nechyba, Michael Christian, Schneiderman, Henry Will, Sipe, Michael Andrew
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US14/033,802
• Publication Number:
  US 20140016837A1
• Time in Patent Office: 316 Days
• Field of Search: 382/118
• US Class Current: 382/118
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G06V 40/45 Detection of the body part being alive

Abstract

An example method includes receiving a first image and a second image of a face of a user, where one or both images have been granted a match by facial recognition. The method further includes detecting a liveness gesture based on at least one of a yaw angle of the second image relative to the first image and a pitch angle of the second image relative to the first image, where the yaw angle corresponds to a transition along a horizontal axis, and where the pitch angle corresponds to a transition along a vertical axis. The method further includes generating a liveness score based on a yaw angle magnitude and/or a pitch angle magnitude, comparing the liveness score to a threshold value, and determining, based on the comparison, whether to deny authentication to the user with respect to accessing one or more functionalities controlled by the computing device.

Claims

1. A method comprising:

receiving, by a computing device, a first authentication image;

outputting, by the computing device, a prompt to perform a predetermined liveness gesture;

receiving, by the computing device, a second authentication image;

detecting, by the computing device, a liveness gesture based on at least one of a yaw angle of the second authentication image relative to the first authentication image and a pitch angle of the second authentication image relative to the first authentication image, wherein the yaw angle corresponds to a transition centered around a vertical axis, and wherein the pitch angle corresponds to a transition centered around a horizontal axis;

determining, by the computing device, whether the detected liveness gesture matches the predetermined liveness gesture; and

determining, by the computing device, based on whether the detected liveness gesture matches the predetermined liveness gesture, whether to deny authentication with respect to accessing one or more functionalities controlled by the computing device.

2. The method of claim 1, further comprising:

generating, by the computing device, a liveness score based on at least one of a yaw angle magnitude associated with the yaw angle and a pitch angle magnitude associated with the pitch angle; and

comparing, by the computing device, the liveness score to a threshold value,

wherein determining, by the computing device, whether to deny authentication with respect to accessing the one or more functionalities controlled by the computing device is further based on the comparison of the liveness score to the threshold value.

3. The method of claim 1, wherein detecting the liveness gesture further comprises:

detecting, by the computing device, the liveness gesture based at least in part on at least one facial landmark represented in one or both of the first authentication image and the second authentication image.

4. The method of claim 3, wherein detecting the liveness gesture based at least in part on the at least one facial landmark represented in one or both of the first authentication image and the second authentication image further comprises:

identifying, by the computing device, both a left eye and a right eye in the first authentication image;

identifying, by the computing device, only one of the left eye and the right eye in the second authentication image; and

obtaining, by the computing device, the yaw angle of the second authentication image relative to the first authentication image.

5. The method of claim 3, wherein detecting the liveness gesture based at least in part on the at least one facial landmark represented in one or both of the first authentication image and the second authentication image further comprises:

identifying, by the computing device, both a left eye and a right eye in the first authentication image;

determining, by the computing device, that either the left eye or the right eye is absent from the second authentication image; and

obtaining, by the computing device, the yaw angle of the second authentication image relative to the first authentication image.

6. The method of claim 3, wherein detecting the liveness gesture based at least in part on the at least one facial landmark represented in one or both of the first authentication image and the second authentication image further comprises:

detecting, by the computing device, an absence of a representation of an earlobe in the first authentication image;

detecting, by the computing device, the representation of the earlobe in the second authentication image; and

obtaining, by the computing device, the yaw angle of the second authentication image relative to the first authentication image.

7. The method of claim 3, wherein detecting the liveness gesture based at least in part on the at least one facial landmark represented in one or both of the first authentication image and the second authentication image further comprises:

detecting, by the computing device, a representation of a nose base in the first authentication image;

detecting, by the computing device, an absence of the representation of the nose base in the second authentication image; and

obtaining, by the computing device, the yaw angle of the second authentication image relative to the first authentication image.

8. The method of claim 3, wherein detecting the liveness gesture based at least in part on the at least one facial landmark represented in one or both of the first authentication image and the second authentication image further comprises:

detecting, by the computing device, an absence of a representation of a nose bridge in the first authentication image;

detecting, by the computing device, the representation of the nose bridge in the second authentication image; and

obtaining, by the computing device, the yaw angle of the second authentication image relative to the first authentication image.

9. The method of claim 1,

wherein at least one of the first authentication image and the second authentication image comprises a frame included in video data received by the computing device.

10. The method of claim 1, further comprising:

performing, by the computing device, facial recognition analysis on each of the first authentication image and the second authentication image;

if at least one of the first authentication image and second authentication image fails the facial recognition analysis, denying, by the computing device, authentication with respect to accessing the one or more functionalities controlled by the computing device; and

if both of the first authentication image and the second authentication image pass the facial recognition analysis and the detected liveness gesture matches the predetermined liveness gesture, granting, by the computing device, authentication with respect to accessing the one or more functionalities controlled by the computing device.

11. The method of claim 1, wherein detecting the liveness gesture comprises:

determining, by the computing device, a first head orientation associated with the first authentication image and a second head orientation associated with the second authentication image.

12. The method of claim 11, wherein determining the first head orientation and the second head orientation comprises:

comparing, by the computing device, a representation of a face in each of the first authentication image and the second authentication image to at least one head pose model.

13. The method of claim 12, wherein determining at least one of the first head orientation and the second head orientation comprises:

determining, by the computing device, at least one of the first head orientation and the second head orientation based at least in part on the yaw angle of the second authentication image relative to the first authentication image, and

wherein the at least one head pose model conforms to one of a frontal orientation, a partial profile orientation, and a profile orientation.

14. The method of claim 13, wherein the partial profile orientation displays an angle of between 35 degrees and 55 degrees from the frontal orientation, and wherein the profile orientation displays an angle of between 80 degrees and 100 degrees from the frontal orientation.

15. The method of claim 12, wherein determining at least one of the first head orientation and the second head orientation comprises determining, by the computing device, at least one of the first head orientation and the second head orientation based at least in part on the pitch angle between the second authentication image and the first authentication image, and wherein the at least one head pose model conforms to one of a frontal orientation, an upward-facing orientation, and a downward-facing orientation.

16. The method of claim 1, further comprising:

granting, by the computing device, a match by facial recognition of one or both of the first authentication image and the second authentication image based at least in part on comparing one or both of the first authentication image and the second authentication image to at least one authorized enrollment image.

17. The method of claim 1, wherein receiving the first authentication image and the second authentication image further comprises selecting, by the computing device, the first authentication image and the second authentication image from a plurality of authentication images that includes at least the first authentication image, the second authentication image, and a third authentication image.

18. The method of claim 17, wherein selecting the first authentication image and the second authentication image from the plurality of authentication images comprises selecting, by the computing device, the first authentication image and the second authentication image based at least in part on the detected liveness gesture.

19. The method of claim 1, wherein determining whether to deny authentication with respect to the one or more functionalities controlled by the computing device comprises denying, by the computing device, the authentication irrespective of a match by facial recognition with respect to one or both of the first authentication image and the second authentication image.

20. The method of claim 1, further comprising:

determining, by the computing device, whether the first authentication image and second authentication image are associated with a consistent user identity, at least in part by determining whether the first authentication image displays at least a threshold similarity relative to the second authentication image.

21. A computer-readable storage device encoded with instructions that, when executed, cause one or more processors of a computing device to perform operations comprising:

receiving a first authentication image;

outputting a prompt to perform a predetermined liveness gesture;

receiving a second authentication image;

detecting a liveness gesture based on at least one of a yaw angle of the second authentication image relative to the first authentication image and a pitch angle of the second authentication image relative to the first authentication image, wherein the yaw angle corresponds to a transition centered around a vertical axis, and wherein the pitch angle corresponds to a transition centered around a horizontal axis;

determining whether the detected liveness gesture matches the predetermined liveness gesture;

determining, based on whether the detected liveness gesture matches the predetermined liveness gesture, whether to deny authentication with respect to accessing one or more functionalities controlled by the computing device.

22. A device comprising:

a memory; and

one or more programmable processors configured to;

receive a first authentication image;

output a prompt to perform a predetermined liveness gesture;

receive a second authentication image;

detect a liveness gesture based on at least one of a yaw angle of the second authentication image relative to the first authentication image and a pitch angle of the second authentication image relative to the first authentication image, wherein the yaw angle corresponds to a transition centered around a vertical axis, and wherein the pitch angle corresponds to a transition centered around a horizontal axis;

determine whether the detected liveness gesture matches the predetermined liveness gesture; and

determine, based on whether the detected liveness gesture matches the predetermined liveness gesture, whether to deny authentication with respect to accessing one or more functionalities controlled by the device.

23. The device of claim 22, wherein the one or more programmable processors are further configured to:

generate a liveness score based on at least one of a yaw angle magnitude associated with the yaw angle and a pitch angle magnitude associated with the pitch angle; and

compare the liveness score to a threshold value,

wherein, to determine whether to deny authentication with respect to accessing the one or more functionalities controlled by the device, the one or more programmable processors are configured to determine whether to deny authentication based on the comparison of the liveness score to the threshold value.

24. The device of claim 22, wherein to detect the liveness gesture, the one or more programmable processors are further configured to detect the liveness gesture based at least in part on at least one facial landmark represented in one or both of the first authentication image and the second authentication image.

25. The device of claim 22,

wherein at least one of the first authentication image and the second authentication image comprises a frame included in video data received by the device.

26. The device of claim 22, wherein the one or more programmable processors are further configured to:

perform facial recognition analysis on each of the first authentication image and the second authentication image;

if at least one of the first authentication image and second authentication image fails the facial recognition analysis, deny authentication with respect to accessing the one or more functionalities controlled by the device; and

if both of the first authentication image and the second authentication image pass the facial recognition analysis and the detected liveness gesture matches the predetermined liveness gesture, grant authentication with respect to accessing the one or more functionalities controlled by the device.

27. The device of claim 22, wherein the one or more programmable processors are further configured to:

determine whether the first authentication image and second authentication image are associated with a consistent user identity, at least in part by determining whether the first authentication image displays at least a threshold similarity relative to the second authentication image.