Hand gesture detection

• US 8,970,479 B1
• Filed: 07/31/2012
• Issued: 03/03/2015
• Est. Priority Date: 07/31/2012
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Rawles, LLC (Amazon.com, Inc.)
• Inventors: Chang, Samuel Henry, Yao, Ning, Puri, Manika, Wang, Weijian, Gopalan, Sowmya, Zhu, Yuhua
• Primary Examiner(s): SHERMAN, STEPHEN G
• Application Number:
  US13/562,734
• Time in Patent Office: 945 Days
• Field of Search: 345156-184
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Techniques are described for detecting a hand gesture made by a user. Fingertips of a hand may be identified and tracked over time. When a user contracts the fingertips from an extended position, hand spread may be calculated based on the area of the hand and fingers. The hand spread over time may be compared to a Gaussian function to evaluate whether the observed motion represents a grasping motion.

Claims

1. A system comprising:

one or more processors;

an image sensor; and

one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;

capturing a series of images of a hand using the image sensor;

analyzing, at least in part, the images to detect fingertips of the hand;

subsequent to detecting the fingertips, further analyzing the images to determine a spread of the hand over time; and

identifying a hand grasp gesture based at least in part on (a) whether the spread of the hand over time corresponds to a Gaussian function and (b) whether the spread of the hand decreases by a threshold amount subsequent to detecting the fingertips.

2. The system of claim 1, wherein identifying the hand grasp gesture is further based at least in part on whether the spread of the hand decreases by the threshold amount within a predefined time subsequent to detecting the fingertips.

3. The system of claim 1, wherein identifying the hand grasp gesture is further based at least in part on whether the fingertips move toward a center of the hand subsequent to detecting the fingertips.

4. The system of claim 1, the acts further comprising:

further analyzing the images to determine a contour of the hand within each of the images; and

fitting a closed shape to the determined contour of the hand within each of the images to determine the spread of the hand over time.

5. The system of claim 1, the acts further comprising:

determining positions of the detected fingertips in each of the images; and

fitting a radially symmetric closed shape to the determined positions of the fingertips of the hand in each of the images to determine the spread of the hand over time.

6. The system of claim 1, the acts further comprising:

further analyzing the images to determine a contour of the hand within each of the images; and

maximizing a size of a predefined shape within the determined contour of the hand to determine the spread of the hand over time.

7. A method, comprising:

determining a spread of a hand over time;

comparing the spread of the hand over time with a Gaussian function, the Gaussian function associated with a transition of the spread of the hand over time from an open hand, where fingers are spread outwards respective to a palm of the hand, to a closed hand, where the fingers have converged towards a palm of the hand; and

identifying a hand gesture based at least in part on the comparing.

8. The method of claim 7, wherein identifying the hand gesture is further based at least in part on whether the spread of the hand decreases by a threshold amount over time.

9. The method of claim 7, wherein identifying the hand gesture is further based at least in part on whether the spread of the hand decreases by a threshold amount within a time period of a predefined length.

10. The method of claim 7, further comprising:

determining a contour of the hand over time; and

fitting a closed shape to the determined contour of the hand to determine the spread of the hand over time.

11. The method of claim 7, further comprising:

determining positions of fingertips of the hand over time; and

fitting a closed shape to the determined positions of the fingertips determine the spread of the hand over time.

12. The method of claim 7, further comprising:

determining a contour of the hand over time; and

maximizing a size of a predefined shape within the determined contour of the hand to determine the spread of the hand over time.

13. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

detecting fingertips of a hand;

subsequent to detecting the fingertips, determining a spread of the hand over time;

comparing the spread of the hand over time with a Gaussian function, the Gaussian function associated with a range of the spread of the hand over time; and

identifying a hand gesture based at least in part on the comparing.

14. The one or more non-transitory computer-readable media of claim 13, wherein the spread of the hand is defined at least in part by the detected fingertips.

15. The one or more non-transitory computer-readable media of claim 13, wherein identifying the hand gesture is further based at least in part on whether the spread of the hand decreases by a threshold amount over time.

16. The one or more non-transitory computer-readable media of claim 13, wherein identifying the hand gesture is further based at least in part on whether the spread of the hand decreases by a threshold amount within a time period of a predefined length.

17. The one or more non-transitory computer-readable media of claim 13, the acts further comprising:

determining a contour of the hand over time; and

fitting a closed shape to the determined contour of the hand to determine the spread of the hand over time.

18. The one or more non-transitory computer-readable media of claim 13, the acts further comprising:

determining positions of the fingertips over time; and

fitting a closed shape to the determined positions of fingertips to determine the spread of the hand over time.

19. The one or more non-transitory computer-readable media of claim 13, the acts further comprising:

determining a contour of the hand over time; and

maximizing a size of a predefined shape within the determined contour of the hand to determine the spread of the hand over time.