Predictive Determination

• US 20110234490A1
• Filed: 06/06/2011
• Published: 09/29/2011
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Markovic, Relja, McBride, Justin, Snook, Gregory N.
• Granted Patent: US 8,578,302 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Systems, methods and computer readable media are disclosed for a gesture recognizer system architecture. A recognizer engine is provided, which receives user motion data and provides that data to a plurality of filters. A filter corresponds to a gesture, that may then be tuned by an application receiving information from the gesture recognizer so that the specific parameters of the gesture—such as an arm acceleration for a throwing gesture—may be set on a per-application level, or multiple times within a single application. Each filter may output to an application using it a confidence level that the corresponding gesture occurred, as well as further details about the user motion data.

Claims

1. A method for predicting a gesture made by a user to a first application, comprising:

receiving image data captured by a depth camera, wherein the image data comprises depth information and is representative of a motion or pose performed by the user;

applying a filter to the image data to interpret a gesture, wherein the filter comprises a first parameter about the gesture and a second parameter about the gesture, the first parameter corresponding to an earlier part of the gesture than the second parameter;

determining, from the applied filter, an output corresponding to the gesture being performed, wherein determining the output includes determining the output corresponds to a high confidence level when the first parameter corresponds to a high confidence level and the second parameter does not correspond to a high confidence level; and

sending the first application the output.

2. The method of claim 1, wherein determining, from the applied filter, an output corresponding to the gesture being performed includes:

determining that future received data will correspond to the gesture being performed based on a correlation between the image data and a prior data that corresponds to the gesture being completed.

3. The method of claim 1, wherein determining, from the applied filter, an output corresponding to the gesture being performed includes:

determining that future received data will correspond to the gesture being performed based on a physiology of the user.

4. The method of claim 1, wherein the output comprises a confidence level that the data is indicative of the gesture being performed.

5. The method of claim 1, wherein the output comprises an indication that the filter is being predicted.

6. The method of claim 1, further comprising:

receiving second image data;

applying the filter to the second image data to interpret the gesture;

determining from the applied filter that the gesture has been performed;

determining a second output based on the applied filter; and

sending the first application the second output.

7. The method of claim 6, wherein the output comprises an indication that the filter is being predicted and the second output comprises an indication that the filter has been completed.

8. The method of claim 1, further comprising:

receiving second image data;

applying the filter to the second image data to interpret the gesture;

determining from the applied filter that the gesture was not performed;

determining a second output based on the applied filter, the second output indicating that the gesture was not performed; and

sending the first application the second output.

9. The method of claim 1, further comprising:

receiving a parameter from the first application, the parameter defining a threshold of one of a volume of space, a velocity, a direction of movement, an angle, or a place where a movement occurs; and

wherein

applying a filter to the image data to interpret the gesture includes applying the parameter to the image data to interpret the gesture.

10. The method of claim 1, wherein the output corresponds to an output that would be determined when the gesture was completed.

11. The method of claim 1, wherein the filter is linked to a previous filter, and determining, from the applied filter, an output corresponding to the gesture being performed, includes:

determining the output based on a prior output of the previous filter.

12. The method of claim 1, further comprising:

applying a second filter to the image data to interpret the gesture, the second filter representing a second gesture and comprising base information about the second gesture;

determining, from the applied second filter, a second output corresponding to the second gesture being performed and a context, the second output being indicative of a greater confidence level than the output; and

sending the first application the second output.

13. A method for processing a gesture made by a user to a first application, comprising:

receiving image data captured by a depth camera, wherein the image data comprises depth information and is representative of a motion or pose performed by the user;

determining an output from the filter based on the image data;

sending the application the output

applying a second filter to the image data to interpret the image data, the second filter representing a second gesture and comprising base information about the second gesture;

determining, from the applied second filter, a second output corresponding to the second gesture being performed and a context, the second output being indicative of a greater confidence level than the output; and

sending the application the second output.

14. The method of claim 13, further comprising:

sending the application at least one selected from the group consisting of a velocity of movement, a release point, a distance, and a body part that made the gesture.

15. The method of claim 13, further comprising:

sending the application a plurality of outputs, each output corresponding to a filter of a plurality of filters.

16. A computer readable storage medium, comprising computer readable instructions that when executed on a computer, cause the computer to perform operations comprising:

receiving from a first application of the a plurality of applications a value for at least one parameter;

receiving image data captured by a depth camera, wherein the image data comprises depth information and is representative of a motion or pose performed by the user;

applying the filter to the image data to interpret the image data, wherein the filter comprises a first parameter and a second parameter about a gesture, the first parameter corresponding to an earlier part of the gesture than the second parameter;

determining a confidence level that the image data is indicative of the gesture, wherein determining the confidence level includes determining the confidence level corresponds to a high confidence level when the first parameter corresponds to a high confidence level and the second parameter does not correspond to a high confidence level; and

sending the first application an indication of the gesture with its associated confidence level.

17. The computer-readable storage medium of claim 16, further comprising:

applying a second filter to the image data;

determining, from the applied second filter, a confidence level that the image data is indicative of a second gesture corresponding to the second filter, the second confidence level being greater than the confidence level; and

sending the first application the second confidence level.

18. The computer-readable storage medium of claim 16, wherein determining a confidence level that the image data is indicative of at least one gesture includes:

determining that future received data will be indicative of at least one gesture based on a correlation between the image data and prior image data that is indicative of the gesture.

19. The computer-readable storage medium of claim 16, wherein determining a high confidence level that the image data is indicative of the at least one gesture based on the first parameter corresponding to a high confidence level and the second parameter not corresponding to a high confidence level comprises:

determining that future received data will correspond to the gesture being performed based on a correlation between the image data and a prior data that corresponds to the gesture being completed.

20. The computer-readable storage medium of claim 16, wherein determining a high confidence level that the image data is indicative of the at least one gesture based on the first parameter corresponding to a high confidence level and the second parameter not corresponding to a high confidence level comprises:

determining that future received data will correspond to the gesture being performed based on a physiology of the user.