User authentication and verification through video analysis

• US 9,607,138 B1
• Filed: 12/18/2013
• Issued: 03/28/2017
• Est. Priority Date: 12/18/2013
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Baldwin, Leo Benedict, Gilmour, Nicholas Ryan
• Primary Examiner(s): Herzog, Madhuri
• Application Number:
  US14/132,885
• Time in Patent Office: 1,196 Days
• Field of Search: 713/168, 713182-186, 726 2- 4, 726 16- 21, 726 26- 30
• US Class Current: 1/1
• CPC Class Codes: G06F 21/30 Authentication, i.e. establishing the identity or authorisation of security principals
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: H04L 63/0861 using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04W 12/00 Security arrangements; Authentication; Protecting privacy or anonymity
• CPC Class Codes: H04W 12/06 Authentication

Abstract

Approaches utilize image information to not only identify a user, but also verify that the user is actually a physical person and not a false representation of that user. For example, a computing device can utilize image information with a facial recognition process in order to verify an identity of a current user of the device. A fingerprint or other verification metric can be generated from the image information and can be used to verify that the user is actually a physical human user instead of a representation (e.g., photo) of a human user. The fingerprint can include a number of cues, such as a shape of the face, a stereo disparity signature of the face, facial feature movement, as well as a presence of a human pulse. The cues can be combined and utilized for authenticating a user into the device as well as for verifying that the user is actually a physical person and not a false representation of that user.

Claims

1. A computing device, comprising:

at least one processor;

a first camera having a first field of view;

a second camera separated by a distance from the first camera, the second camera having a second field of view that at least partially overlaps the first field of view; and

memory including instructions that, when executed by the computing device, cause the computing device to;

acquire a first image by the first camera;

acquire a second image by the second camera, the first image and the second image being captured over a period of time;

analyze the first image to locate a first representation of a head portion;

analyze the second image to locate a second representation corresponding to the head portion in the second image;

determine, from the first representation and the second representation, at least one of depth or distance information associated with one or more facial features of the head portion;

determine the at least one of depth or distance information corresponds to a physical person within a detectable range of the computing device, and not a representation of the physical person;

detect changes in a red chroma color component of the head portion relative to at least one other chroma color component or luminance value over the period of time;

determine the changes correspond to a signature representative of the physical person, and not a representation of the physical person; and

authenticate the head portion as being that of the physical person in response to determining the at least one of depth or distance information corresponds to the physical person and the red chroma color component relative to the at least one other chroma color component or luminance value corresponds to the signature representative of the physical person.

2. The computing device of claim 1, wherein the signature is representative of the physical person when the changes vary with a frequency and amplitude of red chroma color changes consistent with a living human face or an amount of disparity is determined to be below a predetermined threshold, wherein changes in the red chroma color component correspond to changes in coloration of the head portion due to periodic blood flow through human skin, the periodic blood flow corresponding to a human pulse.

3. The computing device of claim 1, wherein the instructions further cause the computing device to:

analyze the first representation and the second representation using a facial recognition algorithm to determine at least one facial feature contained in both the first representation and the second representation;

compare the at least one facial feature to one or more facial features of previously authenticated faces; and

identify one of the previously authenticated faces that matches the head portion.

4. The computing device of claim 1, wherein the instructions causing the computing device to determine the at least one of depth or distance information further cause the computing device to:

determine a first location of a first feature point in the first image corresponding to at least one facial feature represented in the first representation;

determine a second location of a second feature point in the second image corresponding to the at least one facial feature represented in the second representation; and

determine an amount of offset between the first location and the second location along at least one axis.

5. A computer-implemented method, comprising:

acquiring, by cameras of a computing device having at least partially overlapping fields of view, a set of images including a representation of a face, the set of images comprising a first image and a second image;

determining at least one of depth or distance information of one or more features of the face represented in the first image and the second image;

determining the at least one of depth or distance information corresponds to a physical person within a detectable range of the computing device, and not a representation of the physical person;

determining, in at least a subset of the set of images, a change in a red chroma color component of the face relative to at least one other chroma color component or luminance value over a period of time;

determining the change corresponds to a signature of the physical person, and not a representation of the physical person; and

authenticating the face as being that of the physical person, and not a representation of the face, in response to determining the at least one of depth or distance information corresponds to the physical person and the change corresponds to the signature of the physical person.

6. The computer-implemented method of claim 5, wherein

determining the at least one of depth or distance information includes:

locating a first representation of the face in the first image;

locating a second representation of the face in the second image;

analyzing the first representation and the second representation using a facial recognition algorithm to determine a facial feature represented in both the first representation and the second representation;

determining a first location of one or more feature points corresponding to the facial feature represented in the first representation;

determining a second location of the one or more feature points in the second representation; and

determining an amount of offset between the first location and the second location along at least one axis.

7. The computer-implemented method of claim 5, further comprising:

locating a first representation of the face in the first image;

analyzing the first representation using a facial recognition algorithm;

determining, in response to analyzing the first representation, a facial feature represented in the first representation

identifying a previously authenticated face that matches the first representation.

8. The computer-implemented method of claim 5, further comprising:

generating a three-dimensional (3D) model of the representation based at least in part upon the set of images, wherein generating the 3D model utilizes at least in part determining one or more depths of one or more features on the face; and

identifying a previously authenticated face model that matches the 3D model.

9. The computer-implemented method of claim of claim 5, wherein the signature includes at least one of a movement of at least one facial feature, a movement of a head, or a representation of a retina.

10. The computer-implemented method of claim 5, wherein the period of time encompasses a plurality of human heartbeat cycles.

11. The computer-implemented method of claim 5, wherein the signature represents that of the physical person when the change includes a frequency and amplitude corresponding to a change in coloration of the face due to periodic blood flow through human skin, the periodic blood flow corresponding to a human pulse.

12. The computer-implemented method of claim 5, wherein determining the change comprises:

determining at least a normalized red chroma color component varies with a frequency and amplitude consistent with a living human face.

13. The computer-implemented method of claim 5, further comprising:

granting access to functionality on the computing device in response to authenticating the face; and

denying access to at least one level of functionality on the computing device in response to being unable to authenticate the face.

14. The computer-implemented method of claim 6, further comprising:

detecting, in at least a portion of the set of images, at least one of blinking, a making of an expression, or a moving of a head; and

including information indicative of the at least one of the blinking, the making of the expression, or the moving of the head in the signature.

15. The computer-implemented method of claim 5, further comprising:

locating a representation of a head in at least one captured image frame before analyzing one of the first set of images.

16. A non-transitory computer readable storage medium storing one or more sequences of instructions executable by one or more processors to perform a set of operations comprising:

acquiring a first set of images and a second set of images that include at least a representation of a face, the first set of images captured using a first camera and the second set of images captured using a second camera of a computing device, wherein the first camera and the second camera have partially overlapping fields of view;

determining at least one of depth or distance information for each of a plurality of corresponding features of the face in the first set of images and the second set of images;

determining the at least one of depth or distance information corresponds to a physical person within a detectable range of the computing device, and not a representation of the person;

determining, in at least a subset of the first set of images and the second set of images, changes in a red chroma color component of the face relative to at least one other chroma color component or luminance value over a determined period of time;

determining the changes correspond to a signature of the physical person, and not a representation of the physical person; and

authenticating the face as being that of the physical person, and not a representation of the face of the physical person, when the at least one of depth or distance information corresponds to the physical person and the changes correspond to the signature of the physical person.

17. The non-transitory computer readable medium of claim 16, further comprising instructions executed by the one or more processors to perform the operations of:

processing at least one image of the first set of images or the second set of images using a facial recognition algorithm; and

determining the representation of the face corresponds to that of an authorized user in response to matching information for the authorized user within at least a minimum level of confidence.

18. The non-transitory computer readable medium of claim 16, further comprising instructions executed by the one or more processors to perform the operations of:

flashing an illumination element of the computing device; and

analyzing, in response to the flashing, a reaction of a pupil of the representation of the face in at least one captured image of the first set of images or the second set of images.

19. The non-transitory computer readable medium of claim 16, further comprising instructions executed by the one or more processors to perform the operations of:

prompting performance of an action with a head in response to determining that the face is unable to be authenticated with at least a minimum level of confidence, at least the first set of images being monitored to detect performance of the action.