Enhanced detection of circular engagement gesture

• US 9,377,859 B2
• Filed: 02/17/2012
• Issued: 06/28/2016
• Est. Priority Date: 07/24/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Clarkson, Ian
• Primary Examiner(s): Levy, Amy M
• Application Number:
  US13/399,757
• Publication Number:
  US 20120151421A1
• Time in Patent Office: 1,593 Days
• Field of Search: 715/863, 382/103, 382/154, 382/187
• US Class Current: 1/1
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/1335 Combining adjacent partial images (e.g. slices) to create a composite input or reference pattern; Tracking a sweeping finger movement
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

The enhanced detection of a circular engagement gesture, in which a shape is defined within motion data, and the motion data is sampled at points that are aligned with the defined shape. It is determined whether a moving object is performing a gesture correlating to the defined shape based on a pattern exhibited by the sampled motion data. An application is controlled if determining that the moving object is performing the gesture.

Claims

1. A computer-implemented method, comprising:

receiving motion data, wherein the motion data comprises multiple images captured of an object over a period of time;

generating a motion history map from the received motion data;

defining a plurality of points within the motion history map, wherein;

each point of the plurality of points corresponds to a point in time,

the plurality of points are positioned within the motion history map and aligned with a shape stored prior to when the multiple images are captured, the shape inscribed within the boundaries of the motion data, and

a quantity of the plurality of points is based, at least in part, on a size of the shape;

sampling the motion history map at the plurality of points;

determining that the object is performing a gesture corresponding to the shape based on the sampled motion history map by, for each point of the plurality of points, comparing a time associated with the sampled motion history map at that point with an expected time for that point; and

controlling an application at least partially based on determining that the object is performing the gesture.

2. The computer-implemented method of claim 1, wherein the determining is based on a relative time at which motion occurred at each of the plurality of points.

3. The computer-implemented method of claim 1, wherein samples of the sampled motion history map are distributed among a plurality of regions of the motion history map according to how recently detected motion occurred at each of the samples, and wherein the determining is based on a sequence of the regions in which the samples are distributed.

4. The computer-implemented method of claim 1, wherein:

the motion history map comprises motion history data values that provide an indication of time since the object was detected at each of multiple points.

5. The computer-implemented method of claim 4, wherein generating the motion history map from the received motion data comprises:

accessing each image of the multiple images; and

generating the motion history map based on the accessed images.

6. The computer-implemented method of claim 4, wherein:

the determining that the object is performing the gesture corresponding to the shape based on the sampled motion history map is based on a pattern exhibited by the sampled motion history map; and

the pattern comprises an upward or downward sloping shape on a graph of the motion history data values for consecutive points that correspond to the shape.

7. The computer-implemented method of claim 4, wherein:

the determining that the object is performing the gesture corresponding to the shape based on the sampled motion history map is based on a pattern exhibited by the sampled motion history map, and

the pattern comprises an upward or downward stepped shape on a graph of the motion history data values for consecutive points that correspond to the shape.

8. The computer-implemented method of claim 4, wherein generating the motion history map from the received motion data comprises determining, for each point of the multiple points, whether the object has been detected within a predetermined period of time and grouping adjacent points determined to have detected motion of the object within the predetermined period of time;

and

wherein the motion history map is sampled at a subset of the grouped points that are aligned with the shape.

9. The computer-implemented method of claim 8, wherein the plurality of points are sampled such that a center of the shape corresponds to a centroid, a center of mass, or a median pixel of the grouped points.

10. The computer-implemented method of claim 8, further comprising:

defining a bounding box around the grouped points in the motion history map, wherein

the size and a location of the shape defined by the plurality of points within the motion history map are defined with respect to the bounding box.

11. The computer-implemented method of claim 10, wherein

the shape comprises a circle, and

a diameter of the circle is less than a smaller dimension of the bounding box.

12. The computer-implemented method of claim 10, wherein

a center of the shape is aligned within the bounding box according to a function of a dimension or an aspect ratio of the bounding box.

13. The computer-implemented method of claim 1, wherein the shape comprises a circle shape, a figure-eight-shape, or an infinity-shape.

14. A system comprising:

a processor; and

a memory communicatively coupled with and readable by the processor and having stored therein processor-readable instructions which, when executed by the processor, cause the processor to;

receive motion data, wherein the motion data comprises multiple images captured of an object over a period of time;

generate a motion history map from the received motion data;

define a plurality of points within the motion history map, wherein;

each point of the plurality of points corresponds to a point in time,

the plurality of points are positioned within the motion history map and aligned with a shape stored prior to when the multiple images are captured, the shape inscribed within the boundaries of the motion data, and

a quantity of the plurality of points is based, at least in part, on a size of the shape;

sample the motion history map at the plurality of points;

determine that the object is performing a gesture correlated to the shape based on the sampled motion history map by, for each point of the plurality of points, comparing a time associated with the sampled motion history map at that point with an expected time for that point; and

control an application at least partially based on determining that the object is performing the gesture.

15. The system of claim 14, wherein the processor-readable instructions which cause the processor to determine that the object is performing the gesture correlated to the shape based on the sampled motion history map is based on a relative time at which motion occurred at each of the plurality of points.

16. The system of claim 14, wherein:

samples of the sampled motion history map are distributed among a plurality of regions of the motion history map according to how recently detected motion occurred at each of the samples, and

the processor-readable instructions which cause the processor to determine that the object is performing the gesture correlated to the shape based on the sampled motion history map is based on a sequence of the regions in which the samples are distributed.

17. The system of claim 14, wherein:

the motion history map comprises motion history data values that provide, for multiple points, an indication of time since the object was detected at each of the multiple points.

18. The system of claim 17, wherein the processor-readable instructions which, when executed by the processor, cause the processor to generate the motion history map from the received motion data, include instructions for:

accessing the multiple images; and

generating the motion history map based on the accessed images.

19. The system of claim 17, wherein:

the processor-readable instructions that cause the processor to determine that the object is performing the gesture corresponding to the shape based on the sampled motion history map is further based on a pattern exhibited by the sampled motion history map; and

the pattern comprises an upward or downward sloping shape on a graph of the motion history data values for consecutive points that are aligned with the shape.

20. The system of claim 17, wherein:

the processor-readable instructions that cause the processor to determine that the object is performing the gesture corresponding to the shape based on the sampled motion history map is further based on a pattern exhibited by the sampled motion history map; and

the pattern comprises an upward or downward stepped shape on a graph of the motion history data values for consecutive points that are aligned with the shape.

21. The system of claim 17, wherein the processor-readable instructions which, when executed by the processor, cause the processor to generate the motion history map from the received motion data, include instructions for:

determining, for each point of the multiple points, whether the object has been detected within a predetermined period of time; and

grouping adjacent points determined to have detected motion of the object within the predetermined period of time, wherein

the motion history map is sampled at a subset of the grouped points that are aligned with the shape.

22. The system of claim 21, wherein the plurality of points are sampled such that a center of the shape corresponds to a centroid, a center of mass, or a median pixel of the grouped points.

23. The system of claim 21, further comprising processor-readable instructions which, when executed by the processor, cause the processor to:

define a bounding box around the grouped points in the motion history map, wherein the size and a location of the shape defined by the plurality of points within the motion history map are defined with respect to the bounding box.

24. The system of claim 23, wherein:

the shape comprises a circle, and

a diameter of the circle is less than a smaller dimension of the bounding box.

25. The system of claim 23, wherein a center of the shape is aligned within the bounding box according to a function of a dimension or an aspect ratio of the bounding box.

26. The system of claim 14, wherein the shape comprises a circle shape, a figure-eight-shape, or an infinity-shape.

27. An apparatus comprising:

means for receiving motion data, wherein

the motion data comprises multiple images captured of an object over a period of time;

means for generating a motion history map from the received motion data;

means for defining a plurality of points within the motion history map, wherein;

each point of the plurality of points corresponds to a point in time,

the plurality of points are positioned within the motion history map and aligned with a shape stored prior to when the multiple images are captured, the shape inscribed within the boundaries of the motion data, and

a quantity of the plurality of points is based, at least in part, on a size of the shape;

means for sampling the motion history map at the plurality of points;

means for determining that the object is performing a gesture corresponding to the shape based on the sampled motion history map by, for each point of the plurality of points, comparing a time associated with the sampled motion history map at that point with an expected time for that point; and

means for controlling an application at least partially based on determining that the object is performing the gesture.

28. The apparatus of claim 27, wherein

the means for determining that the object is performing the gesture uses a relative time at which motion occurred at each of the plurality of points.

29. The apparatus of claim 27, wherein

samples of the sampled motion history map are distributed among a plurality of regions of the motion history map according to how recently detected motion occurred at each of the samples, and wherein the means for determining is based on a sequence of the regions in which the samples are distributed.

30. The apparatus of claim 27, wherein:

the motion history map comprises motion history data values that provide an indication of time since the object was detected at multiple points.

31. A non-transitory computer-readable storage medium encoded with processor-readable instructions that, when executed, cause a processing device to perform operations comprising:

receiving motion data, wherein the motion data comprises multiple images captured of an object over a period of time;

generating a motion history map from the received motion data;

defining a plurality of points within the motion history map, wherein;

each point of the plurality of points corresponds to a point in time,

the plurality of points are positioned within the motion history map and aligned with a shape stored prior to when the multiple images are captured, the shape inscribed within the boundaries of the motion data, and

a quantity of the plurality of points is based, at least in part, on a size of the shape;

sampling the motion history map at the plurality of points;

determining that the object is performing a gesture corresponding to the shape based on the sampled motion history map by, for each point of the plurality of points, comparing a time associated with the sampled motion history map at that point with an expected time for that point; and

controlling an application at least partially based on determining that the object is performing the gesture.