Gesture recognition

• US 9,704,027 B1
• Filed: 02/27/2012
• Issued: 07/11/2017
• Est. Priority Date: 02/27/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Chang, Samuel Henry, Gopalan, Sowmya, Yao, Ning, Zhu, Yuhua
• Primary Examiner(s): Fujita, Katrina
• Application Number:
  US13/405,621
• Time in Patent Office: 1,961 Days
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 2201/07 Target detection
• CPC Class Codes: G06V 2201/121 using special illumination
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A hand gesture may be characterized mathematically as a set of motion parameters applied to a dynamic motion model. Training may be conducted to compile a library of motion parameter sets corresponding to various gestures. Motion parameters corresponding to observed gestures may than be compared to the library of motion parameter sets to classify the observed gestures.

Claims

1. A system comprising:

one or more processors; and

one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;

analyzing a sequence of motions represented by images, the analyzing based at least in part on Gaussian distribution of positional coordinates associated with the sequence of motions;

determining one or more training sequences based at least in part on the sequence of motions;

determining one or more reference gestures based at least in part on the one or more training sequences;

determining, from an observed gesture made by a hand of a user, a first angular orientation of the hand associated with a first pose of the hand;

determining a second angular orientation of the hand associated with a second pose of the hand;

based at least in part on the first angular orientation of the first pose and the second angular orientation of the second pose of the hand, determining motion parameters of the observed gesture, wherein the motion parameters apply to a dynamic motion model;

analyzing the motion parameters based at least in part on the one or more reference gestures; and

classifying the observed gesture as one of the one or more reference gestures.

2. The system of claim 1, wherein the dynamic motion model comprises one or more dynamic state equations.

3. The system of claim 1, wherein determining the motion parameters comprises recursive least squares parameter identification.

4. The system of claim 1, wherein determining the first pose of the hand and the second pose of the hand are based at least in part on analyzing a reflection of a pattern projected onto the hand.

5. The system of claim 1, wherein determining the first pose of the hand and the second pose of the hand are based at least in part on a structured pattern reflected from the hand.

6. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

analyzing a sequence of motions represented by images, the analyzing based at least in part on Gaussian distribution of positional coordinates associated with the sequence of motions;

determining one or more training sequences based at least in part on the sequence of motions;

determining one or more reference gestures based at least in part on the one or more training sequences;

observing a hand gesture of a user;

determining motion parameters of the hand gesture of the user, the motion parameters based on at least a change from a first angular orientation of a hand of the user to a second angular orientation of the hand;

analyzing the motion parameters of the hand gesture of the user with reference to the one or more reference gestures; and

associating an action with the hand gesture based at least in part on the one or more reference gestures.

7. The one or more non-transitory computer-readable media of claim 6, wherein the motion parameters of the hand gesture comprise a dynamic motion model including one or more state equations.

8. The one or more non-transitory computer-readable media of claim 6, wherein determining the motion parameters comprises recursive least squares parameter identification.

9. The one or more non-transitory computer-readable media of claim 6, wherein determining the motion parameters of the hand gesture of the user is based at least in part on structured light analysis of a physical environment of the user.

10. A method comprising:

analyzing a sequence of motions represented by images, the analyzing based at least in part on Gaussian distribution of positional coordinates associated with the sequence of motions;

determining one or more training sequences based at least in part on the sequence of motions;

determining one or more reference gestures based at least in part on the one or more training sequences;

determining motion parameters of an observed hand gesture, wherein the motion parameters of the observed hand gesture are based on a change from a first angular orientation of a hand of a user to a second angular orientation of the hand;

analyzing the motion parameters with the one or more reference gestures; and

classifying the observed hand gesture.

11. The method of claim 10, wherein the motion parameters apply to a dynamic motion model including one or more state equations.

12. The method of claim 10, wherein determining the motion parameters comprises recursive least squares parameter identification.

13. The method of claim 10, wherein determining the motion parameters is based at least in part on structured light analysis of a sequence of images.

14. The method of claim 10, wherein the determining the motion parameters comprises determining the motion parameters using sequential images captured from different perspectives relative to the user.

15. The method of claim 10, further comprising:

receiving, from one or more microphones, an audio signal representing speech from the user; and

characterizing the speech as a command from the user to perform an action or to initiate attention on the user.

16. The method of claim 10, further comprising:

receiving, from a portable signaling device, ultrasonic signals from the user; and

characterizing the ultrasonic signals as a command from the user to perform an action or to initiate attention on the user.

17. The method of claim 10, wherein analyzing the sequence of motions comprises averaging parameters corresponding to the sequence of motions.