SHOW BODY POSITION

• US 20100281432A1
• Filed: 05/01/2009
• Published: 11/04/2010
• Est. Priority Date: 05/01/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bennett, Darren, Geisner, Kevin, Markovic, Relja, Snook, Gregory Nelson, Latta, Stephen Gilchrist
• Granted Patent: US 9,377,857 B2
• Time in Patent Office: Days
• US Class Current: 715/849
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

A capture device may capture a user'"'"'s motion and a display device may display a model that maps to the user'"'"'s motion, including gestures that are applicable for control. A user may be unfamiliar with a system that maps the user'"'"'s motions or not know what gestures are applicable for an executing application. A user may not understand or know how to perform gestures that are applicable for the executing application. Providing visual feedback representing instructional gesture data to the user can teach the user how to properly gesture. The visual feedback may be provided in any number of suitable ways. For example, visual feedback may be provided via ghosted images, player avatars, or skeletal representations. The system can process prerecorded or live content for displaying visual feedback representing instructional gesture data. The feedback can portray the deltas between the user'"'"'s actual position and the ideal gesture position.

Claims

1. A method for providing feedback, the method comprising:

receiving a request for instructions for at least one gesture; and

rendering feedback representing instructional gesture data for the at least one gesture.

2. The method of claim 1 further comprising:

receiving a depth image of a physical space, wherein the depth image includes data representative of a gesture;

determining whether the depth image includes a human target;

scanning the human target if, based on the determination, the depth image includes a human target;

generating a model of the human target based on the scan; and

rendering a visual representation of the model that maps to the gesture of the human target.

3. The method in accordance with claim 1, wherein the instructional gesture data comprises at least one of prerecorded content or live content.

4. The method in accordance with claim 1, further comprising predicting an intended control from the data representative of the gesture, wherein the feedback rendered comprises instructional gesture data applicable to the prediction.

5. The method in accordance with claim 1, wherein the feedback is at least one of visual or auditory feedback.

6. The method in accordance with claim 1, wherein the feedback is a visual representation of the instructional gesture data in a form of a skeletal representation, a ghosted image, or a player avatar.

7. The method in accordance with claim 1, wherein the feedback is rendered by rendering at least one of:

a visual representation of the instructional gesture data superimposed by the visual representation of the gesture,

side-by-side images of the visual representation of the gesture and a visual representation of the instructional gesture data,

a visual representation of the gesture and corrected or highlighted feedback that indicates a delta between an actual position of the gesture and an ideal position, or

a visual representation of the instructional gesture data on a portion of a display space, wherein the portion is smaller than a total display space.

8. The method in accordance with claim 1, further comprising identifying a trigger that initiates the rendering of the feedback, wherein the trigger is one of an identification of a failure in an execution of the gesture, an initiation of an application, or a request for training of the gesture.

9. The method in accordance with claim 1, further comprising determining a control to perform in an application based on the gesture, wherein the gesture is a gesture of a user in a physical space and is mapped to a skeletal model.

10. A system for providing visual feedback, the system comprising:

a capture device, wherein the capture device receives a depth image of a physical space having data representative of a measured gesture;

a display device for rendering visual feedback representing instructional gesture data; and

a processor, wherein the processor executes computer executable instructions.

11. The system in accordance with claim 10, further comprising a networking component adapted to receive instructional gesture data over a network.

12. The system in accordance with claim 10,

wherein the computer executable instructions further comprise instructions for determining whether the depth image includes a human target, and

wherein the capture device, based on a determination that the depth image includes a human target, scans the human target, tracks a skeletal model, and renders a visual representation that maps to the skeletal model.

13. The system in accordance with claim 10, wherein the computer executable instructions further comprise instructions for selecting the visual feedback to render from at least one of prerecorded content or live content received over a network.

14. The system in accordance with claim 10, wherein the computer executable instructions further comprise instructions for predicting an intended control from the data representative of the measured gesture, wherein the visual feedback rendered comprises instructional gesture data applicable to the prediction.

15. The system in accordance with claim 10, wherein the display device renders the visual feedback in real-time or in slow motion.

16. The system in accordance with claim 10, wherein the display device renders a visual representation that maps to the measured gesture, wherein the visual representation is based on a skeletal representation.

17. The system in accordance with claim 10, wherein the visual feedback is a visual representation of the instructional gesture data in a form of a skeletal representation, a ghosted image, or a player avatar.

18. The system in accordance with claim 10, wherein the display device renders the visual feedback in by rendering at least one of:

a visual representation of the instructional gesture data superimposed by the visual representation of the measured gesture,

side-by-side images of the visual representation of the measured gesture and a visual representation of the instructional gesture data,

a visual representation of the measured gesture and corrected or highlighted feedback indicative of an error, such as a delta between an actual position and an ideal position, or

a visual representation of the instructional gesture data on a portion of a display space, wherein the portion is smaller than a total display space.

19. The system in accordance with claim 10, wherein the computer executable instructions further comprise instructions for identifying a trigger that initiates the rendering of the visual feedback, wherein the trigger is indicative of a failure in an execution of the measured gesture, an initiation of an application, or a request for training.

20. A method for determining whether to provide instructional gesture data, the method comprising:

receiving image data for a scene, wherein the image data includes data representative of a gesture;

comparing the data representative of the gesture to at least one output of a gesture filter;

detecting a variation between the data representative of the gesture and the at least one output of the gesture filter, wherein the variation is indicative of a failure in an execution of the gesture; and

determining whether to provide instructional gesture data based on said detected variation.