GESTURE RECOGNITION FOR DEVICE INPUT

• US 20150049017A1
• Filed: 08/25/2014
• Published: 02/19/2015
• Est. Priority Date: 08/16/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Weber, Jason R., Noble, Isaac S., Zhou, Dong, Polansky, Stephen M., Strutt, Guenael T., Bell, Matthew P.
• Granted Patent: US 9,921,659 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A user can make a symbol with their hand, or other such gesture, at a distance from a computing device that can be captured by at least one imaging element of the device. The captured information can be analyzed to attempt to determine the location of distinguishing features of the symbol in the image information. The image information is then compared to hand gesture information stored in, for example, a library of hand gestures for the user. Upon identifying a match, an input to an application executing on the computing device is provided when the image information contains information matching at least one hand gesture with at least a minimum level of certainty. The hand gesture could include a single “static” gesture, such as a specific letter in sign language, for example, or include two or more “static” gestures. The gesture could also include motion, such as hand movement.

Claims

1. A computer-implemented method, comprising:

acquiring, by a computing device, image data;

analyzing a first subset of the image data acquired over a first period to identify a gesture;

determining a first path corresponding to movement of a representation of one or more fingers in the first subset of the image data;

comparing the first path to a gesture library;

analyzing a second subset of the image data acquired over a second period to identify the gesture in response to being unable to identify a first path matching gesture;

determining a second path corresponding to movement of the representation of one or more fingers in the second subset of the image data;

determining, from the first path and the second path, a combined path corresponding to the movement of the representation of the one or more fingers ;

determining a combined path matching gesture by comparing the combined path to the gesture library; and

sending an input to an application executing on the computing device.

2. The computer-implemented method of claim 1, wherein the first subset of the image data and the second subset of the image data correspond to at least one of overlapping or adjacent periods in time.

3. The computer-implemented method of claim 1, wherein the first path and the second path correspond to at least one of overlapping or adjacent paths represented by the movement of the representation of one or more fingers.

4. The computer-implemented method of claim 1, wherein the application performs a function in response to receiving the input, the function being at least one of dialing a specific telephone number, or unlocking the computing device.

5. The computer-implemented method of claim 1, wherein analyzing the image data includes:

determining, for each of the first subset of the image data and the second subset of the image data, a position of the representation of the one or more fingers using at least one of image recognition, proximity detection, or intensity analysis.

6. The computer-implemented method of claim 1, further comprising:

receiving a request to assign a gesture input corresponding to the combined path matching gesture as the input for performing a function associated with the application;

detecting the movement corresponding to the representation of one or more fingers from the image data; and

storing the movement as the gesture input in the gesture library.

7. The computer-implemented method of claim 6, wherein the gesture input is stored as a sequence of points in time corresponding to the movement.

8. The computer-implemented method of claim 7, further comprising:

comparing time information associated with the combined path to the time stored for the sequence of points associated with the gesture input.

9. A computing device, comprising:

a device processor;

at least one image capture element; and

a memory device including instructions operable to be executed by the processor to perform a set of actions, enabling the computing device to;

acquire image data from at least one image captured using the at least one image capture element;

analyze a first subset of the image data acquired over a first period to identify a gesture;

determine first path corresponding to movement of a representation of one or more fingers in the first subset of the image data;

compare the first path to a gesture library;

analyze a second subset of the image data acquired over a second period to identify the gesture in response to being unable to identify a first path matching gesture;

determine a second path corresponding to movement of the representation of one or more fingers in the second subset of the image data;

determine, from the first path and the second path, a combined path corresponding to the movement of the representation of a portion of the user in the first subset of the image data and the second subset of the image data;

determine a combined path matching gesture by comparing the combined path to the gesture library; and

send an input to an application executing on the computing device.

10. The computing device of claim 9, wherein the instructions that, when executed by the processor, further enabling the computing device to:

deactivate a gesture input mode of the computing device in response to a period of inactivity.

11. The computing device of claim 9, wherein the representation of the portion of the user is a representation of one or more fingers of the user.

12. The computing device of claim 9, wherein the first path and the second path correspond to at least one of overlapping or adjacent paths represented by the movement of the representation of the portion of the user.

13. The computing device of claim 9, wherein the application performs a function in response to receiving the input, the function being at least one of dialing a specific telephone number, or unlocking the computing device.

14. The computing device of claim 9, wherein the first subset of the image data and the second subset of the image data correspond to at least one of overlapping or adjacent periods in time.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to:

acquire, by a computing device, image data from at least one image;

analyze a first subset of the image data acquired over a first period to identify a gesture;

determine first path corresponding to movement of a representation of a portion of a user in the first subset of the image data;

compare the first path to a gesture library;

analyze a second subset of the image data acquired over a second period to identify the gesture in response to being unable to identify a first path matching gesture;

determine a second path corresponding to movement of the representation of one or more fingers in the second subset of the image data;

determine, from the first path and the second path, a combined path corresponding to the movement of the representation of a portion of the user in the first subset of the image data and the second subset of the image data;

determine a combined path matching gesture by comparing the combined path to the gesture library; and

send an input to an application executing on the computing device.

16. The non-transitory computer-readable medium of claim 15, wherein the first subset of the image data and the second subset of the image data correspond to at least one of overlapping or adjacent periods in time, and the first path and the second path corresponding to at least one of overlapping or adjacent paths represented by the movement of the representation of one or more fingers.

17. The non-transitory computer-readable medium of claim 15, wherein analyzing the image data includes:

determining, for each of the first subset of the image data and the second subset of the image data, a position of the representation of the one or more fingers using at least one of image recognition, proximity detection, or intensity analysis.

18. The non-transitory computer-readable medium of claim 17, wherein the instructions that, when executed by the processor, further cause the processor to:

receive a request to assign a gesture input corresponding to the combined path matching gesture as the input for performing a function associated with the application;

detect the movement corresponding to the representation of one or more fingers from the image data; and

store the movement as the gesture input in the gesture library.

19. The non-transitory computer-readable medium of claim 18, wherein the gesture input is stored as a sequence of points in time corresponding to the movement.

20. The non-transitory computer-readable medium of claim 19, wherein the instructions that, when executed by the processor, further cause the processor to:

compare combined time information associated with the combined path to the sequence of points in time associated with the gesture input.