USING FACIAL DATA FOR DEVICE AUTHENTICATION OR SUBJECT IDENTIFICATION

• US 20130015946A1
• Filed: 07/12/2011
• Published: 01/17/2013
• Est. Priority Date: 07/12/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lau, James Kai Yu, Kaheel, Ayman, El-Saban, Motaz, Shawky, Mohamed, Gonzalez, Monica, Baz, Ahmed El, Deif, Tamer, Aly, Alaa Abdel-Hakim
• Granted Patent: US 9,082,235 B2
• Time in Patent Office: Days
• US Class Current: 340/5.2
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06V 10/422 for representing the structure of the pattern or shape of an object therefor
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/169 Holistic features and representations, i.e. based on the facial image taken as a whole
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/40 Spoof detection, e.g. liveness detection
• CPC Class Codes: G07C 9/00 Individual registration on entry or exit
• CPC Class Codes: G09C 5/00 Ciphering apparatus or methods not provided for in the preceding groups, e.g. involving the concealment or deformation of graphic data such as designs, written or printed messages
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

Exemplary methods, apparatus, and systems are disclosed for authenticating a user to computing device. In one exemplary embodiment, an indication of a request by a user to unlock a mobile device in a locked state is received. One or more images of the face of the user are captured. Facial components of the user from the one or more captured images are extracted. A determination is made as to whether the user is an authorized user or a non-authorized user based at least in part on a comparison of the facial components of the user extracted from the one or more captured images to facial components of the authorized user from one or more authentication images of the authorized stored on the mobile device. If the user is determined to be the authorized user, the mobile device unlocked; otherwise, the mobile device is maintained in its locked state.

Claims

1. A method, comprising:

receiving an indication of a request by a user to unlock a mobile device in a locked state;

capturing one or more images of the face of the user;

extracting facial components of the user from the one or more captured images;

determining whether the user is an authorized user based at least in part on a comparison of the facial components of the user extracted from the one or more captured images to facial components of the authorized user from one or more authentication images of the authorized user stored on the mobile device;

unlocking the mobile device if the user is determined to be the authorized user; and

maintaining the locked state of the mobile device if the user is determined not to be the authorized user.

2. The method of claim 1, wherein the one or more authentication images include a first authentication image of the authorized user in a first lighting condition and a second authentication image of the authorized user in a second lighting condition, the second lighting condition being different than the first light condition.

3. The method of claim 1, wherein the one or more authentication images include an authentication image of the authorized user where the face of the authorized user is illuminated primarily by a screen of the mobile device.

4. The method of claim 1, wherein the capturing includes prompting the user to make a facial expression or head motion, and

wherein the one or more authentication images include at least one authentication image of the user making the facial expression or head motion.

5. The method of claim 1, further comprising pre-processing the one or more captured images to compensate for lighting conditions in which the one or more images are captured, the pre-processing comprising performing one or more of white balancing, global exposure compensation, local exposure compensation, or downsampling of the one or more captured images.

6. The method of claim 1, wherein the comparison is performed by generating feature descriptors representative of the facial components of the user in the one or more captured images, and determining a difference between the feature descriptors of the user in the one or more captured images and respective facial descriptors of the authorized user generated from the one or more authentication images.

7. The method of claim 1, wherein the comparison is performed by generating a graph-based representation of the facial components of the user in the one or more captured images, and determining a difference between the graph-based representation of the facial components of the user and respective graph-based representations of the authorized user generated from the one or more authentication images,

the graph-based representation of the facial components of the user being indicative of appearances and relative geometric relationships of the facial components of the user.

8. The method of claim 1, wherein the determining comprises:

if the facial components of the user extracted from the one or more captured images match the facial components of the authorized user from the one or more authentication images, generating an indication that the user is authorized; and

if the facial components of the user extracted from the one or more captured images do not match the facial components of the authorized user from the one or more authentication images, causing a secondary authentication method to be performed on the mobile device.

9. The method of claim 1, wherein the capturing comprises capturing two or more images of the face of the user in succession, and

wherein the determining comprises determining whether the user is the authorized user based at least in part on a comparison of the facial components of the user extracted from the two or more captured images to the facial components of the authorized user from the two or more authentication images stored on the mobile device.

10. One or more computer-readable storage devices storing computer-executable instructions which when executed by a computer cause the computer to perform a method, the method comprising:

receiving an image of a user of a device;

identifying one or more facial descriptors of the user from the received image; and

evaluating whether the one or more identified facial descriptors of the user match one or more facial descriptors of a previous user extracted from multiple previously captured images of the previous user,

the multiple previously captured images of the previous user including at least one image of the previous user in a low-lighting condition.

11. The one or more computer-readable storage devices of claim 10, wherein the at least one image of the previous user in the low-lighting condition comprises an image of the user with light from a screen of the mobile device as the primary light source.

12. The one or more computer-readable storage devices of claim 10, wherein the method further comprises generating an indication that the user in the received image is an authorized user or is not the authorized user based at least in part on the evaluation.

13. The one or more computer-readable storage devices of claim 10, wherein the method further comprises finding that the one or more facial descriptors of the user in the received image match the one or more facial descriptors of the previous user if a smallest difference between the facial descriptors of the user in the received image and the facial descriptors of the previous user from the multiple images of the previous user satisfies a threshold value.

14. The one or more computer-readable storage devices of claim 10, wherein the method further comprises finding that the one or more facial descriptors of the user in the received image match the one or more facial descriptors of the previous user if (a) a difference between the facial descriptors of the user in the received image and the facial descriptors of the previous user from a plurality of images of the authorized user satisfies a first threshold value, and (b) the number of images in the plurality of images satisfies a second threshold.

15. The one or more computer-readable storage devices of claim 10, wherein the evaluating further comprises evaluating whether the one or more identified facial descriptors of the user match one or more facial descriptors of multiple other previous users.

16. The one or more computer-readable storage devices of claim 10, wherein a first of the facial descriptors is weighted higher than a second of the facial descriptors.

17. The one or more computer-readable storage devices of claim 10, wherein the method further comprises finding that the one or more facial descriptors of the user in the received image match the one or more facial descriptors of the previous user if a smallest difference between the facial descriptors of the user in the received image and the facial descriptors of the previous user from the multiple images of the previous user satisfies a first threshold value, and if a ratio between the smallest distance and a greatest distance between the facial descriptors of the user in the received image and the facial descriptors of the previous user in the multiple images of the previous user satisfies a second threshold value.

18. The one or more computer-readable storage devices of claim 10, wherein two or more images of the user of the device are received, and wherein the method further comprises:

computing an average of the one or more facial descriptors from the two or more images of the user; and

finding that the one or more facial descriptors of the user match the one or more facial descriptors of the previous user if a difference between the average of the one or more facial descriptors and the facial descriptors of the previous user in at least one of the multiple images of the previous user satisfies a threshold value.

19. A system, comprising:

an image capturing device;

a display device;

a memory or storage device storing a program;

a processing unit operable to execute the program, wherein said execution of the program causes the processing unit to;

display a first image capture screen on the display device, the first image capture screen including a first prompt requesting that a first image of a user be captured in a first lighting condition;

receive a first signal from the user to trigger the image capture device;

capture the first image of the user with the image capture device in response to the first signal;

display a second image capture screen on the display device, the second image capture screen including a second prompt requesting that a second image of the user be captured in a second lighting condition, the second lighting condition being different than the first lighting condition;

receive a second signal from the user to trigger the image capture device; and

capture the second image of the user with the image capture device in response to the second signal.

20. The system of claim 19, wherein at least one of the first image capture screen or the second image screen prompts the user to make a facial expression during image capture.