Image analysis for user authentication

• US 10,108,961 B2
• Filed: 03/27/2018
• Issued: 10/23/2018
• Est. Priority Date: 01/13/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wang, Yicong, Xu, Haizhi
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US15/937,658
• Publication Number:
  US 20180218371A1
• Time in Patent Office: 210 Days
• Field of Search: 382100, 382115
• CPC Class Codes: A61K 48/00 Medicinal preparations containing genetic material which is inserted into cells of the living body to treat genetic diseases; Gene therapy
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06Q 20/12 specially adapted for electronic shopping systems
• CPC Class Codes: G06Q 20/3674 involving authentication
• CPC Class Codes: G06Q 20/40 Authorisation, e.g. identification of payer or payee, verification of customer or shop credentials; Review and approval of payers, e.g. check credit lines or negative lists
• CPC Class Codes: G06Q 20/4014 Identity check for transactions
• CPC Class Codes: G06Q 20/40145 Biometric identity checks
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/40 Spoof detection, e.g. liveness detection
• CPC Class Codes: G06V 40/45 Detection of the body part being alive
• CPC Class Codes: H04L 63/0861 using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59

Abstract

A user can be authenticated to any of a number of computing devices using an authentication process that recognizes the user and verifies that an actual human being is attempting to be authenticated, in order to minimize the ability of another person to spoof the authentication process. A model of a user can be generated and stored in the cloud, enabling that model to be synchronized across various devices. A user accessing one of these devices can have image information captured, which can be used with a facial recognition process, to recognize the user, and with a human verification process, to verify that the facial information corresponds to a human user. Various approaches such as visual analysis, three-dimensional imaging, and thermal imaging can be used to verify that the human user being recognized is interactive with the device.

Claims

1. A mobile phone, comprising:

a processor; and

memory storing instructions that, when executed by the processor, cause the mobile phone to;

obtain a request to authorize a transaction;

capture, using a camera on the mobile phone, a two dimensional infrared image of at least a portion of a face of a user interacting with the mobile phone;

project a pattern of infrared dots from the mobile phone onto a face of the user;

obtain a reflection of at least a subset of the infrared dots using an infrared detector;

analyze the reflection of the infrared dots to verify that the user is in proximity of the mobile phone;

identify the user based at least in part on the two dimensional infrared image and the reflection of at least a subset of the infrared dots;

authenticate the user as a result of the identity of the user being recognized as an authorized user and as a result of the user'"'"'s face being in the proximity of the mobile phone; and

allow a transaction to be performed as a result of the user being authenticated.

2. The mobile phone of claim 1, wherein the reflection of the infrared dots is used to generate a three-dimensional model of a face of the user.

3. The mobile phone of claim 1, wherein the instructions further cause the mobile phone to:

obtain identifying information for an authorized user in association with enrollment in a visual authentication process; and

authenticate the user as a result of the identity of the user being recognized as the authorized user associated with the identifying information.

4. The mobile phone of claim 1, wherein an infrared LED on the mobile phone illuminates the user.

5. The mobile phone of claim 1, wherein the instructions further cause the mobile phone to direct the user to look at the mobile phone to facilitate an authentication process.

6. A computer-implemented method of authenticating a user of a mobile device, comprising:

capturing, using an image sensor associated with the mobile device, an image of at least a portion of a face of the user of the mobile device;

projecting a pattern of infrared dots from the mobile device onto the face of the user;

obtaining a reflection of a subset of the infrared dots using an infrared detector;

determining, based at least in part on the reflection, that the user corresponds to a physical being within proximity of the mobile device;

determining an identity of the user by at least analyzing a portion of the image and the reflection; and

authenticating the user as a result of the identity of the user being recognized as an authorized user and as a result of the user corresponding to a physical being in the proximity of the mobile device.

7. The computer-implemented method of claim 6, wherein:

the portion of the image is analyzed using a facial recognition algorithm;

facial data of the authorized user is collected as part of a visual authentication enrollment process; and

the identity of the user is determined based at least in part on the facial data.

8. The computer-implemented method of claim 6, further comprising:

generating a three-dimensional model of the face of the user by analyzing the reflection of a subset of the infrared dots; and

storing the three-dimensional model as part of a visual authentication enrolment process.

9. The computer-implemented method of claim 6, wherein the image sensor is operable to capture light in at least the infrared spectrum.

10. The computer-implemented method of claim 9, further comprising:

illuminating the user with an infrared light source.

11. The computer-implemented method of claim 9, wherein the image is an infrared image.

12. The computer-implemented method of claim 11, further comprising:

detecting that the user is looking at the image sensor; and

initiating an authentication process as a result of detecting that the user is looking at the image sensor.

13. The computer-implemented method of claim 6, wherein at least a second verification process is utilized when a first verification process is unable to verify that the user contained in the image corresponds to a physical being with at least a minimum level of confidence.

14. The computer-implemented method of claim 6, further comprising presenting a signal to the user indicating that that the user should look at the device to perform an authentication process.

15. The computer-implemented method of claim 6, further comprising:

receiving a request from a user to utilize an authentication process;

obtaining image information of the face of the user;

generating a three-dimensional model corresponding to the face of the user; and

storing the model to a memory on the mobile device.

16. The computer-implemented method of claim 15, wherein the three-dimensional model and the image information is further made available to at least one other computing device accessible by the user.

17. A non-transitory computer-readable storage medium storing instructions that, as a result of being executed by a processor on a mobile computing device cause the mobile computing device to:

capture, using an infrared camera on the mobile device, an infrared image of at least a portion of a face of a user of the mobile device;

project a pattern of infrared dots from the mobile device onto the face of the user;

obtain a reflection of a subset of the projected infrared dots using the infrared camera on the mobile device;

determine, using the reflection of the subset of the projected infrared dots, that the user represented in the image matches a three-dimensional model of an authorized user stored in a memory on the mobile device; and

authorize a request as a result of having determined that the user is the authorized user.

18. The non-transitory computer-readable storage medium of claim 17, wherein the instructions when executed further cause the mobile device to:

as part of a visual authentication enrolment process, project an additional set of infrared dots on to a face of an authorized user;

capture a reflection of the additional set of infrared dots using the infrared camera; and

generate a depth model of the authorized user based at least in part on the reflection of the additional set of infrared dots.

19. The non-transitory computer-readable storage medium of claim 17, wherein the instructions when executed further cause the mobile device to determine that the gaze of the user is directed at the device.

20. The non-transitory computer-readable storage medium of claim 17, wherein the instructions when executed further cause the mobile device to illuminate the face of the user with an infrared light emitting diode on the mobile device.