Gesture recognition for device input

• US 9,921,659 B2
• Filed: 08/25/2014
• Issued: 03/20/2018
• Est. Priority Date: 08/16/2012
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Weber, Jason R., Noble, Isaac S., Zhou, Dong, Polansky, Stephen M., Strutt, Guenael T., Bell, Matthew P.
• Primary Examiner(s): Jamshidi, Ghodrat
• Application Number:
  US14/468,206
• Publication Number:
  US 20150049017A1
• Time in Patent Office: 1,303 Days
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A user can make a symbol with their hand, or other such gesture, at a distance from a computing device that can be captured by at least one imaging element of the device. The captured information can be analyzed to attempt to determine the location of distinguishing features of the symbol in the image information. The image information is then compared to hand gesture information stored in, for example, a library of hand gestures for the user. Upon identifying a match, an input to an application executing on the computing device is provided when the image information contains information matching at least one hand gesture with at least a minimum level of certainty. The hand gesture could include a single “static” gesture, such as a specific letter in sign language, for example, or include two or more “static” gestures. The gesture could also include motion, such as hand movement.

Claims

1. A computer-implemented method, comprising:

acquiring, by a computing device, image data over a period of time;

storing the image data in a buffer;

determining a first movement of at least a portion of a hand represented in a first portion of the image data;

determining the first movement does not match gesture information in a gesture library;

determining, after determining the first movement does not match the gesture information, a second movement of at least a portion of the hand represented in a second portion of the image data stored in the buffer, the second portion of the image data being acquired previous to the first portion of the image data in the period of time;

determining, from the first movement and the second movement, a combined movement;

determining gesture information in the gesture library that matches the combined movement; and

sending an input, corresponding to the gesture information, to an application executing on the computing device.

2. The computer-implemented method of claim 1, wherein the first portion of the image data and the second portion of the image data correspond to at least one of partially overlapping or adjacent periods in time.

3. The computer-implemented method of claim 1, wherein the first movement and the second movement correspond to at least one of partially overlapping or adjacent movements in time.

4. The computer-implemented method of claim 1, wherein the application performs a function in response to receiving the input, the function being at least one of dialing a telephone number or unlocking the computing device.

5. The computer-implemented method of claim 1, further comprising:

determining, in the first portion of the image data, a first position of at least one finger using at least one of image recognition, proximity detection, or intensity analysis; and

determining, in the second portion of the image data, a second position of the at least one finger using at least one of image recognition, proximity detection, or intensity analysis.

6. The computer-implemented method of claim 1, further comprising:

receiving a request to assign a gesture as the input;

detecting movement of at least a portion of the hand in the image data; and

storing the movement as the input in the gesture library.

7. The computer-implemented method of claim 6, wherein the input is stored as a sequence of points in time corresponding to the movement.

8. The computer-implemented method of claim 7, further comprising:

comparing time information associated with the combined movement to the time stored for the sequence of points.

9. The computer-implemented method of claim 1, further comprising:

determining a combined movement speed corresponding to the combined movement, the combined movement speed including a first movement speed associated with the first movement and a second movement speed associated with the second movement; and

comparing the combined movement speed to a speed associated with the gesture information.

10. A computing device, comprising:

at least one processor;

at least one image capture element; and

at least one memory including instructions that, when executed, cause the computing device to;

acquire image data over a period of time using the at least one image capture element;

store the image data in a buffer;

determine a first movement of a hand represented in a first portion of the image data;

determine the first movement does not match gesture information in a gesture library;

determine, after determining the first movement does not match the gesture information, a second movement of the hand represented in a second portion of the image data stored in the buffer, the second portion of the image data being acquired previous to the first portion of the image data in the period of time;

determine, from the first movement and the second movement, a combined movement;

determine gesture information in the gesture library that matches the combined movement; and

send an input, corresponding to the gesture information, to an application executing on the computing device.

11. The computing device of claim 10, wherein the instructions further cause the computing device to:

after sending the input, deactivate a gesture input mode of the computing device in response to a period of inactivity.

12. The computing device of claim 10, wherein the first movement and the second movement correspond to at least one of partially overlapping or adjacent movements in time.

13. The computing device of claim 10, wherein the application performs a function in response to receiving the input, the function being at least one of dialing a telephone number or unlocking the computing device.

14. The computing device of claim 10, wherein the first portion of the image data and the second portion of the image data correspond to at least one of partially overlapping or adjacent periods in time.

15. A computing system, comprising:

at least one processor;

at least one image capture element; and

at least one memory including instructions that, when executed, cause the computing device to;

acquire, by a computing device, image data over a period of time using the at least one image capture element;

store the image data in a buffer;

determine a first movement of at least a portion of a user represented in a first portion of the image data;

determine the first movement does not match gesture information in a gesture library;

determine, after determining the first movement does not match the gesture information, a second movement of at least a portion of the user represented in a second portion of the image data stored in the buffer, the second portion of the image data being acquired previous to the first portion of the image data in the period of time;

determine, from the first movement and the second movement, a combined movement;

determine gesture information in the gesture library that matches the combined movement; and

send an input, corresponding to the gesture information, to an application executing on the computing device.

16. The computing system of claim 15, wherein the first portion of the image data and the second portion of the image data correspond to at least one of partially overlapping or adjacent periods in time, and the first movement and the second movement correspond to at least one of partially overlapping or adjacent movements in time.

17. The computing system of claim 15, wherein the instructions further cause the computing system to:

determine, in the first portion of the image data, a first position of at least one finger using at least one of image recognition, proximity detection, or intensity analysis; and

determine, in the second portion of the image data, a second portion of the at least one finger using at least one of image recognition, proximity detection, or intensity analysis.

18. The computing system of claim 15, wherein the instructions further cause the computing system to:

receive a request to assign a gesture as the input;

detect movement of at least a portion of the user in the image data; and

store the movement as the input in the gesture library.

19. The computing system of claim 18, wherein the input is stored as a sequence of points in time corresponding to the movement.

20. The computing system of claim 19, wherein the instructions further cause the computing system to:

compare time information associated with the combined movement to the time stored for the sequence of points.