Enhanced detection of gesture

• US 8,605,941 B2
• Filed: 07/24/2009
• Issued: 12/10/2013
• Est. Priority Date: 07/25/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Clarkson, Ian
• Primary Examiner(s): Johns, Andrew W
• Application Number:
  US12/508,645
• Publication Number:
  US 20100040292A1
• Time in Patent Office: 1,600 Days
• Field of Search: 382/100, 382/103, 382/201, 382/187, 382/107, 715/863
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

The enhanced detection of a waving engagement gesture, in which a shape is defined within motion data, the motion data is sampled at points that are aligned with the defined shape, and, based on the sampled motion data, positions of a moving object along the defined shape are determined over time. It is determined whether the moving object is performing a gesture based on a pattern exhibited by the determined positions, and an application is controlled if determining that the moving object is performing the gesture.

Claims

1. A non-transitory computer-readable storage medium encoded with instructions that, when executed, operate to cause a computer to perform operations comprising:

defining a shape corresponding to a gesture within motion data;

sampling the motion data at points that are aligned with the defined shape;

determining, based on the sampled motion data, positions of a moving object along the defined shape, over time, wherein determining the positions occurs subsequent to defining the shape;

determining whether the moving object is performing the gesture based on a pattern exhibited by the determined positions; and

controlling an application if determining that the moving object is performing the gesture.

2. The computer-readable storage medium of claim 1, wherein:

the motion data comprises a motion history map further comprising motion history data values that provide, for each point of an image, an indication of time since the moving object was detected at the point.

3. The computer-readable storage medium of claim 2, wherein determining the positions of the moving object along the defined shape, over time, further comprises:

at first and second times;

selecting points that are aligned with the defined shape and that comprise sampled motion history data values which satisfy a predetermined threshold, and

selecting one of the selected points; and

outputting, as first and second positions of the moving object, the one points respectively selected at the first and second times.

4. The computer-readable storage medium of claim 3, wherein the one point comprises a median, mean, or random point of the selected points.

5. The computer-readable storage medium of claim 4, wherein the motion history map is generated using optical flow.

6. The computer-readable storage medium of claim 2, further comprising instructions that, when executed, operate to cause the computer to perform operations comprising:

accessing the image; and

generating the motion history map based on the accessed image.

7. The computer-readable storage medium of claim 2, wherein the pattern comprises a shape of one period of a sinusoid on a graph of the determined positions over time, the determined positions expressed as a proportion of a single dimension of the shape.

8. The computer-readable storage medium of claim 2, wherein the pattern comprises a shape of one period of a stepped sinusoid on a graph of the determined positions over time, the determined positions expressed as a proportion of a single dimension of the shape.

9. The computer-readable storage medium of claim 2, further comprising instructions that, when executed, operate to cause the computer to perform operations comprising:

determining, for each point, whether the moving object has been detected within a predetermined threshold; and

grouping adjacent points determined to have detected motion of the moving object within the predetermined threshold,

wherein the motion data is sampled at a subset of the grouped points that are aligned with the defined shape.

10. The computer-readable storage medium of claim 9, further comprising instructions that, when executed, operate to cause the computer to perform operations comprising:

defining a bounding box around the grouped points,

wherein a size and a location of the shape within the motion data are defined with respect to the bounding box.

11. The computer-readable storage medium of claim 10, wherein:

the shape comprises a line segment or a chord.

12. The computer-readable storage medium of claim 10, wherein the shape comprises a longest line segment capable of fitting within the grouped points.

13. The computer-readable storage medium of claim 1, further comprising instructions that, when executed, operate to cause the computer to perform operations comprising:

detecting groups of points within the motion data; and

selecting one of the groups of points,

wherein the shape is defined within the one selected group.

14. The computer-readable storage medium of claim 13, wherein the one group is selected based on relative size.

15. The computer-readable storage medium of claim 1, wherein:

the motion data is sampled at a sampled quantity of points that are aligned with the defined shape, and

the sampled quantity comprises a fixed quantity or is based on a size of the defined shape or an aligned quantity of points that are aligned with the defined shape within the motion data.

16. The computer-readable storage medium of claim 1, wherein determining whether the moving object is performing the gesture based on the pattern exhibited by the determined positions further comprises comparing the pattern to upper and lower threshold criteria and to timing criteria.

17. The computer-readable storage medium of claim 1, wherein the gesture comprises a swiping or waving, hand or finger gesture.

18. The computer-readable storage medium of claim 1, further comprising instructions that, when executed, operate to cause the computer to perform operations comprising:

adding the determined positions to a motion history, and

detecting whether the pattern exists within the motion history.

19. The computer-readable storage medium of claim 1, further comprising instructions that, when executed, operate to cause the computer to perform operations comprising:

counting a quantity of performances of the gesture.

20. The computer-readable storage medium of claim 1, wherein at least one dimension of the shape is defined based on detected positions of the moving object at two or more different times.

21. The computer-readable storage medium of claim 1, wherein the shape comprises a first shape of a plurality of shapes, wherein each shape of the plurality of shapes corresponds to a gesture.

22. A computer-implemented method comprising:

defining a shape corresponding to a gesture within motion data;

sampling the motion data at points that are aligned with the defined shape;

determining, based on the sampled motion data, positions of a moving object along the defined shape, over time, wherein determining the positions occurs subsequent to defining the shape;

determining whether the moving object is performing the gesture based on a pattern exhibited by the determined positions; and

controlling an application if determining that the moving object is performing the gesture.

23. A device comprising a processor configured to:

define a shape corresponding to a gesture within motion data;

sample the motion data at points that are aligned with the defined shape;

determine, based on the sampled motion data, positions of a moving object along the defined shape, over time, wherein the positions are determined subsequent to the shape being defined;

determine whether the moving object is performing the gesture based on a pattern exhibited by the determined positions; and

control an application if determining that the moving object is performing the gesture.

24. A system comprising:

means for defining a shape corresponding to a gesture within motion data;

means for sampling the motion data at points that are aligned with the defined shape;

means for determining, based on the sampled motion data, positions of a moving object along the defined shape, over time, wherein the means for determining comprise means for determining the positions subsequent to the shape being defined;

means for determining whether the moving object is performing the gesture based on a pattern exhibited by the determined positions; and

means for controlling an application if determining that the moving object is performing the gesture.