User identification by gesture recognition

• US 9,465,982 B2
• Filed: 04/07/2014
• Issued: 10/11/2016
• Est. Priority Date: 06/29/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Karakotsios, Kenneth Mark, Ivanchenko, Volodymyr V.
• Primary Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US14/247,013
• Publication Number:
  US 20140219515A1
• Time in Patent Office: 918 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G06V 40/37 based only on signature signals such as velocity or pressure, e.g. dynamic signature recognition

Abstract

A user can be identified and/or authenticated to an electronic device by analyzing aspects of a motion or gesture made by that user. At least one imaging element of the device can capture image information including the motion or gesture, and can determine time-dependent information about that motion or gesture in two or three dimensions of space. The time-dependent information can be used to identify varying speeds, motions, and other such aspects that are indicative of a particular user. The way in which a gesture or motion is made, in addition to the motion or gesture itself, can be used to authenticate an individual user. While other persons can learn the basic gesture or motion, the way in which each person makes that gesture or motion will generally be at least slightly different, which can be used to prevent unauthorized access to sensitive information, protected functionality, or other such content.

Claims

1. A computing device, comprising:

a processor;

a first camera;

a second camera; and

memory including instructions that, upon being executed by the processor, cause the computing device to;

capture within a first time period, using the first camera, ambient light image data including a first representation of at least one object corresponding to a user;

capture within the first time period, using the second camera, reflected infrared light image data including a second representation of the at least one object;

identify a plurality of three-dimensional positions of the at least one object within the first time period using the ambient light image data and the reflected infrared light image data;

compare the plurality of three-dimensional positions of the at least one object and a respective time within the first time period for each three-dimensional position to one or more user-specific measurements corresponding to a manner of the user performing a gesture, the one or more user-specific measurements corresponding to at least one of;

(a) a path of movement of the gesture, (b) one or more of relative timing, acceleration, or velocity information corresponding to the gesture, or (c) measurements of at least one feature of the user that is used to perform the gesture; and

determine that the plurality of three-dimensional positions of the at least one object and the respective time within the first time period for each three-dimensional position matches the one or more user-specific measurements with at least a minimum level of confidence; and

provide the user with access to at least some content or functionality of the computing device.

2. The computing device of claim 1, wherein the instructions, upon being executed, further cause the computing device to:

determine that a second plurality of three-dimensional positions and a respective time within a second time period for each three-dimensional position of the second plurality of three-dimensional positions do not match the one or more user-specific measurements with at least the minimum level of confidence; and

restrict access to at least some of the content or the functionality of the computing device.

3. The computing device of claim 1, wherein the instructions, upon being executed, further cause the computing device to:

substantially remove background information from the ambient light image data using the reflected infrared light image data.

4. A computer-implemented method, comprising:

obtaining ambient light image data, captured within a first time period, including a first representation of at least one object;

obtaining reflected infrared image data, captured within the first time period, including a second representation of the at least one object;

identifying a plurality of three-dimensional points corresponding to the at least one object within the first time period using the ambient light image data and the reflected infrared image data;

comparing the plurality of three-dimensional points and a respective time within the first time period for each three-dimensional point to measurements corresponding to the at least one object used in performing a gesture;

determining that the plurality of three-dimensional points and the respective time for each three-dimensional point corresponds to the measurements with at least a minimum level of confidence; and

providing access to at least some content or functionality of a computing device.

5. The computer-implemented method of claim 4, wherein the measurements relate to at least one of:

(a) a path of movement of the gesture, (b) one or more of relative timing, acceleration, or velocity information corresponding to the gesture, or (c) the at least one object used to perform the gesture.

6. The computer-implemented method of claim 4, wherein identifying the plurality of three-dimensional points is based at least in part upon one of image recognition, proximity detection, or intensity analysis.

7. The computer-implemented method of claim 4, further comprising:

substantially removing background information in the ambient light image data using the reflected infrared image data.

8. The computer-implemented method of claim 4, further comprising:

detecting motion within a field of view of a camera of the computing device; and

capturing the ambient light image data using the camera.

9. The computer-implemented method of claim 4, further comprising:

prompting for a second gesture;

capturing, within a second period of time, image data including a representation of at least one object used in performing the second gesture; and

providing an indication of a degree of difficulty of replicating the second gesture.

10. The computer-implemented method of claim 4, further comprising:

causing at least one source of illumination to be activated within the first time period.

11. The computer-implemented method of claim 10, wherein the at least one source of illumination includes at least one of a white light or an infrared source of illumination, the at least one of the white light or the infrared source of illumination being activated continuously or periodically during the first time period.

12. The computer-implemented method of claim 4, further comprising:

storing at least one portion of the ambient light image data in a rolling buffer,

wherein the plurality of three-dimensional points are identified after a fixed period of time of storing the at least one portion of the ambient light image data in the rolling buffer.

13. The computer-implemented method of claim 4, further comprising:

fitting a curve or a function to the plurality of three-dimensional points.

14. The computer-implemented method of claim 4, further comprising:

determining that a second plurality of three-dimensional points and a respective time within a second period of time for each three-dimensional point of the second plurality of three-dimensional points do not correspond to the one or more measurements with at least the minimum level of confidence; and

restricting access to at least some of the content or the functionality of the computing device.

15. The computer-implemented method of claim 4, further comprising:

detecting no gesture for a specified period of inactivity; and

deactivating a gesture input mode of the computing device.

16. A non-transitory computer-readable storage medium storing instructions that, upon being executed by a processor, cause the processor to:

obtain ambient light image data, captured within a first time period, including a first representation of at least one object;

obtain reflected infrared image data, captured within the first time period, including a second representation of the at least one object;

identify a plurality of three-dimensional points corresponding to the at least one object within the first time period using the ambient light image data and the reflected infrared image data;

compare the plurality of three-dimensional points and a respective time within the first time period for each three-dimensional point to measurements corresponding to the at least one object used in performing a gesture, the measurements relating to at least one of;

(a) a path of movement of the gesture, (b) one or more of relative timing, acceleration, or velocity information corresponding to the gesture, or (c) the at least one object used to perform the gesture;

determine that the plurality of three-dimensional points and the respective time for each three-dimensional point corresponds to the measurements with at least a minimum level of confidence; and

provide access to at least some content or functionality of a computing device.

17. The non-transitory computer-readable storage medium of claim 16, wherein the instructions, upon being executed, further cause the processor to:

prompt for a second gesture;

capture, within a second period of time, image data including a representation of at least one object used in performing the second gesture; and

provide an indication of a degree of difficulty of replicating the second gesture.

18. The non-transitory computer-readable storage medium of claim 16, wherein the instructions, upon being executed, further cause the processor to:

store at least one portion of the ambient light image data in a rolling buffer,

wherein the plurality of three-dimensional points are identified after a fixed period of time of storing the at least one portion of the ambient light image data in the rolling buffer.