Gesture recognition for device input

• US 8,819,812 B1
• Filed: 08/16/2012
• Issued: 08/26/2014
• Est. Priority Date: 08/16/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Noble, Isaac S., Zhou, Dong, Weber, Jason R., Polansky, Stephen M., Strutt, Guenael T., Bell, Matthew P.
• Primary Examiner(s): Zand, Kambiz
• Assistant Examiner(s): Jamshidi, Ghodrat
• Application Number:
  US13/587,586
• Time in Patent Office: 740 Days
• US Class Current: 726/19
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A user can make a symbol with their hand, or other such gesture, at a distance from a computing device that can be captured by at least one imaging element of the device. The captured information can be analyzed to attempt to determine the location of distinguishing features of the symbol in the image information. The image information is then compared to hand gesture information stored in, for example, a library of hand gestures for the user. Upon identifying a match, an input to an application executing on the computing device is provided when the image information contains information matching at least one hand gesture with at least a minimum level of certainty. The hand gesture could include a single “static” gesture, such as a specific letter in sign language, for example, or include two or more “static” gestures. The gesture could also include motion, such as hand movement.

Claims

1. A computer-implemented method, comprising:

under control of one or more computing devices including executable instructions,

acquiring first image information using at least one camera of a computing device;

storing, for at least a minimum period of time, the first image information in a rolling buffer;

acquiring second image information using the at least one camera;

detecting, based at least in part on the second image information, one or more fingers of a user within a specified distance of the computing device;

determining, using the second image information, movement of the one or more fingers, the movement corresponding to a first portion of a candidate hand gesture;

analyzing the first image information stored in the rolling buffer to detect a second portion of the candidate hand gesture, the second portion having been performed before the first portion of the candidate hand gesture;

based at least in part on the first portion of the candidate hand gesture and the second portion of the candidate hand gesture, determining a hand gesture performed by the user;

comparing the performed hand gesture to a gesture pattern stored on the computing device; and

authenticating an identity of the user as an authorized user, and providing an input to unlock the computing device from a locked state, in response to the hand gesture, performed by the user, being determined to correspond to the gesture stored on the computing device.

2. The computer-implemented method of claim 1, wherein the acquiring of the first image information is performed in response to identifying a condition indicative of a hand within a predetermined distance of the computing device.

3. The computer-implemented method of claim 1, further comprising:

denying the user access to one or more of functionality or data on the computing device when one or more of the identity of the user is unable to be authenticated to correspond to an authorized user.

4. The computer-implemented method of claim 1, wherein the performed hand gesture is a static gesture made with the one or more fingers of the user.

5. The computer-implemented method of claim 1, wherein the movement of the one or more fingers is associated with a shape drawn in view of the at least one image capturing element of the computing device with one or more fingers of the user.

6. The computer-implemented method of claim 1, wherein the computing device captures images and stores corresponding image information on a rolling basis for a duration of time in the rolling buffer, enabling the computing device to utilize the stored image information for at least one of the detecting or the determining the movement of the one or more fingers to determine the first portion of the candidate hand gesture.

7. A computer-implemented method, comprising:

under control of one or more computing devices including executable instructions,

acquiring image information from at least one image captured using at least one image sensor of a computing device, the image information being stored for a duration of time on a rolling basis;

analyzing the image information to detect a hand gesture performed by a user;

determining, in the image information, movement of one or more fingers, the movement corresponding to a first portion of a candidate hand gesture;

analyzing the image information stored for the duration of time to detect a second portion of the candidate hand gesture;

comparing the first portion of the candidate hand gesture and the second portion of the candidate hand gesture to at least one gesture associated with the user, the second portion of the candidate hand gesture having been performed before the first portion; and

providing an input to an application executing on the computing device in response to the first portion of the candidate hand gesture and the second portion of the candidate hand gesture being determined to correspond to the at least one gesture associated with the user, the at least one gesture associated with the user being specific to the user for providing the input to the application.

8. The computer-implemented method of claim 7, wherein the at least one gesture associated with the user is one of a plurality of gestures stored on the computing device.

9. The computer-implemented method of claim 7, wherein the user is denied access to at least some content or functionality on the computing device when the performed gesture does not match the at least one gesture associated with the user with at least a minimum level of certainty.

10. The computer-implemented method of claim 7, further comprising:

enabling the user to assign a specific gesture as the input for performing a particular function by the application.

11. The computer-implemented method of claim 10, wherein assigning the specific gesture includes training the computing device to recognize a new gesture.

12. The computer-implemented method of claim 10, wherein the function is at least one of dialing a specific telephone number, or unlocking the computing device.

13. The computer-implemented method of claim 7, wherein the performed gesture is performed with a hand of the user and the at least one gesture associated with the user is stored on the computing device and includes information for recognizing at least one symbol made with the hand of the user.

14. The computer-implemented method of claim 7, wherein comparing the first portion of the candidate hand gesture and the second portion of the candidate includes comparing timing information of the first portion of the candidate hand gesture and the second portion of the candidate to timing information of the at least one gesture.

15. The computer-implemented method of claim 7, further comprising:

authenticating an identity of the user when the performed hand gesture at least partially matches the at least one gesture associated with the user; and

granting the user access to one or more of functionality or data on the computing device when the input is provided to the application.

16. The computer-implemented method of claim 7, wherein the image information is analyzed to determine a location of the one or more fingers of the user using at least one of image recognition, proximity detection, or intensity analysis.

17. The computer-implemented method of claim 7, wherein the performed gesture is a static gesture made with a hand or a finger of the user for a period of time.

18. The computer-implemented method of claim 7, wherein the performed gesture includes movement of a finger or a hand of the user over a period of time.

19. The computer-implemented method of claim 7, further comprising:

deactivating a gesture input mode of the computing device if no gesture is detected within a period of inactivity.

20. A computing device, comprising:

a device processor;

at least one image capture element; and

a memory device including instructions operable to be executed by the processor to perform a set of actions, enabling the computing device to;

acquire image information from at least one image captured using at least one image sensor of a computing device, the image information being stored for a duration of time on a rolling basis;

analyze the image information to detect a hand gesture performed by a user;

determine, in the image information, movement of one or more fingers, the movement corresponding to a first portion of a candidate hand gesture;

analyze the image information stored for the duration of time to detect a second portion of the candidate hand gesture;

compare the first portion of the candidate hand gesture and the second portion of the candidate hand gesture at least one gesture associated with the user, the second portion of the candidate hand gesture having been performed before the first portion; and

provide an input to an application executing on the computing device in response to the first portion of the candidate hand gesture and the second portion of the candidate hand gesture being determined to correspond to the at least one gesture associated with the user, the at least one gesture associated with the user being specific to the user for providing the input to the application.

21. The computing device of claim 20, wherein the application executing on the computing device is associated with unlocking the computing device while in a locked screen mode.

22. The computing device of claim 20, further comprising:

enable the user to assign a specific hand gesture to a particular function executable by the application.

23. The computing device of claim 22, wherein the function is at least one of dialing a telephone number, or unlocking the computing device.

24. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to:

acquire image information from at least one image captured using at least one image sensor of a computing device, the image information being stored for a duration of time on a rolling basis;

analyze the image information to detect a hand gesture performed by a user;

determine, in the image information, movement of one or more fingers, the movement corresponding to a first portion of a candidate hand gesture;

analyze the image information stored for the duration of time to detect a second portion of the candidate hand gesture;

compare the first portion of the candidate hand gesture and the second portion of the candidate hand gesture to at least one gesture associated with the user, the second portion of the candidate hand gesture having been performed before the first portion; and

provide an input to an application executing on the computing device in response to the first portion of the candidate hand gesture and the second portion of the candidate hand gesture being determined to correspond to the at least one gesture associated with the user, the at least one gesture associated with the user being specific to the user for providing the input to the application.

25. The non-transitory computer-readable storage medium of claim 24, further comprising:

training the computing device to recognize a new gesture and assign the new gesture to a particular function executable by the application.

26. The non-transitory computer-readable storage medium of claim 24, further comprising:

authenticate an identity of the user when the first portion of the candidate hand gesture and the second portion of the candidate at least partially match the at least one stored gesture; and

grant the user access to one or more of functionality or data on the computing device when the input is provided to the application.