Gesture Coach

• US 20100306712A1
• Filed: 05/29/2009
• Published: 12/02/2010
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Latta, Stephen, Stone Perez, Kathryn, Tsunoda, Kudo, Kipman, Alex, Geisner, Kevin, Bennett, Darren Alexander, Snook, Gregory N.
• Granted Patent: US 8,418,085 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G09B 19/0038 Sports
• CPC Class Codes: G16H 20/30 relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising

Abstract

A capture device may capture a user'"'"'s motion and a display device may display a model that maps to the user'"'"'s motion, including gestures that are applicable for control. A user may be unfamiliar with a system that maps the user'"'"'s motions or not know what gestures are applicable for an executing application. A user may not understand or know how to perform gestures that are applicable for the executing application. User motion data and/or outputs of filters corresponding to gestures may be analyzed to determine those cases where assistance to the user on performing the gesture is appropriate.

Claims

1. A method for providing assistance regarding a user-performed gesture, comprising:

receiving data captured by a capture device, the data corresponding to a user-performed gesture;

analyzing the data to produce an output corresponding to whether the data corresponds to a system-recognized gesture;

determining from the output that it is unlikely that the user correctly performed the system-recognized gesture;

determining from the output that the user likely intended to perform the system-recognized gesture; and

providing assistance regarding the user-performed gesture.

2. The method of claim 1, wherein providing assistance comprises:

adjusting the output; and

sending the adjusted output to an application corresponding to the system-recognized gesture.

3. The method of claim 2, wherein adjusting the output comprises:

increasing the likelihood that the output corresponds to the user correctly performing the system-performed gesture.

4. The method of claim 2, further comprising:

determining that that the output may be adjusted before adjusting the output.

5. The method of claim 1, wherein the output comprises a confidence level.

6. The method of claim 5, wherein it is unlikely that the user correctly performed the system-recognized gesture when the confidence level is below a threshold.

7. The method of claim 1, wherein analyzing the data to produce an output corresponding to whether the data corresponds to a system-recognized gesture further comprises

analyzing, by a filter, the data to produce an output corresponding to whether the data corresponds to a system-recognized gesture further comprises

providing assistance comprises:

replacing the filter with a second filter.

8. The method of claim 1 wherein providing assistance comprises:

suspending an application corresponding to the user-performed gesture.

9. The method of claim 1, wherein providing assistance comprises:

displaying the output to the user on a display device.

10. The method of claim 1, wherein providing assistance comprises:

displaying a representation of the user-performed gesture and a demonstration of the system-recognized gesture.

11. The method of claim 10 further comprising:

displaying a difference between the user-performed gesture and the demonstration of the system-recognized gesture.

12. The method of claim 1, wherein the assistance is sourced from another user.

13. The method of claim 12, wherein the sourced assistance is at least one from a set, the set comprising:

assistance that is at or near the top of a leader-board, assistance that has been rated highly, assistance that has been identified by a user that created it as being appropriate for assistance, assistance that a second user has identified as being appropriate for the user, assistance from a user of the same culture as the user, assistance from a user of the same language as the user, assistance from a user of a similar age as the user, and assistance from a user of a similar location as the user.

14. A system for providing assistance regarding a user-performed gesture, comprising:

a processor;

a component that receives data captured by a camera, the data corresponding to a user-performed gesture;

a component that analyzes the data to produce an output corresponding to whether the data corresponds to a system-recognized gesture;

a component that determines from the output that it is unlikely that the user correctly performed the system-recognized gesture;

a component that determines from the output that the user likely intended to perform the system-recognized gesture; and

a component that provides assistance regarding the user-performed gesture.

15. The system of claim 14, wherein the component that provides assistance further:

relaxes a tolerance level associated with the component that analyzes the data.

16. The system of claim 14, wherein the component that provides assistance further comprises:

a component that adjusts the output; and

a component sends the adjusted output to an application corresponding to the user-performed gesture.

17. The system of claim 16, wherein the component that adjusts the output comprises:

a component that increases a responsiveness of the component that analyzes the data.

18. The system of claim 14, wherein the output comprises a confidence level.

19. The system of claim 18, wherein it is unlikely that the user correctly performed the system-recognized gesture when the confidence level is below a threshold.

20. A computer readable storage medium, comprising computer readable instructions that when executed on a processor, cause the processor to perform the operations of:

receiving data captured by a depth camera, the data corresponding to a user-performed gesture;

analyzing the data to produce an output corresponding to whether the data corresponds to a system-recognized gesture;

determining from the output that it is unlikely that the user correctly performed the system-recognized gesture;

determining from the output that the user likely intended to perform the system-recognized gesture; and

displaying assistance on performing the system-recognized gesture on a display device.