SHOW BODY POSITION

• US 20110035666A1
• Filed: 10/27/2010
• Published: 02/10/2011
• Est. Priority Date: 05/01/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bennett, Darren, Geisner, Kevin, Markovic, Relja, Snook, Gregory Nelson, Latta, Stephen Gilchrist
• Granted Patent: US 9,298,263 B2
• Time in Patent Office: Days
• US Class Current: 715/709
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

A capture device may capture a user'"'"'s motion and a display device may display a model that maps to the user'"'"'s motion, including gestures that are applicable for control. A user may be unfamiliar with a system that maps the user'"'"'s motions or not know what gestures are applicable for an executing application. A user may not understand or know how to perform gestures that are applicable for the executing application. Providing visual feedback representing instructional gesture data to the user can teach the user how to properly gesture. The visual feedback may be provided in any number of suitable ways. For example, visual feedback may be provided via ghosted images, player avatars, or skeletal representations. The system can process prerecorded or live content for displaying visual feedback representing instructional gesture data. The feedback can portray the deltas between the user'"'"'s actual position and the ideal gesture position.

Claims

1. A method for providing feedback, the method comprising:

capturing a depth image, wherein the depth image comprises a plurality of image layers;

evaluating the plurality of image layers;

identifying data representative of a target'"'"'s motion or position in a physical space;

comparing the identified data to a plurality of gesture filters, wherein each of said plurality of gesture filters comprises one or more parameters with associated thresholds and ranges;

identifying a confidence rating output for each of the comparisons to the plurality of gesture filters, wherein each of the plurality of confidence rating outputs indicates a likelihood that the identified data corresponds to a respective gesture filter;

predicting an intended gesture based on a comparison of the confidence rating outputs;

determining that a variation between the identified data and the one or more filter parameters for the intended gesture is below a threshold level; and

rendering feedback on a display representing instructional gesture data of the proper technique for performing the intended gesture, wherein the feedback is indicative of one or more variations between the identified data and the one or more parameters in a gesture filter associated with the intended gesture.

2. The method of claim 1 further comprising:

determining whether the depth image includes a human target;

scanning the human target if, based on the determination, the depth image includes a human target;

generating a model of the human target based on the scan; and

rendering a visual representation of the model that maps to the human target.

3. The method in accordance with claim 1, wherein the feedback comprises at least one of prerecorded content or live content.

4. The method in accordance with claim 1, further comprising predicting an intended control from the identified data, wherein the feedback rendered comprises feedback applicable to the prediction.

5. The method in accordance with claim 1, wherein the feedback is at least one of visual or auditory feedback.

6. The method in accordance with claim 1, wherein the feedback is a visual representation in a form of a skeletal representation, a ghosted image, or a player avatar.

7. The method in accordance with claim 1, wherein the feedback is rendered by rendering at least one of:

a visual representation of the feedback superimposed over the captured depth image of the target,

side-by-side images of the captured depth image of the target and a visual representation of the feedback,

a visual representation of the captured depth image of the target and corrected or highlighted feedback that indicates a delta between a position depicted in the captured depth image of the target and an ideal position, or

a visual representation of the proper method for performing the intended gesture on a portion of a display space, wherein the portion is smaller than the total display space.

8. The method in accordance with claim 1, further comprising identifying a trigger that initiates the rendering of the feedback, wherein the trigger is one of an identification of a failure in an execution of an intended gesture, an initiation of an application, or a request for training of an intended gesture.

9. The method in accordance with claim 1, further comprising determining a control to perform in an comparison application based on a gesture, wherein the gesture is a gesture of a user in a physical space and is mapped to a skeletal model.

10. A system for providing visual feedback, the system comprising:

a capture device for capturing a depth image of a physical space having data representative of a target'"'"'s motion or position in the physical space;

a processor for executing computer executable instructions for;

identifying data representative of a target'"'"'s motion or position in a physical space;

comparing the identified data to a plurality of gesture filters, wherein each of said plurality of gesture filters comprises one or more parameters with associated thresholds and ranges;

identifying a confidence rating output for each of the comparisons to the plurality of gesture filters, wherein each of the plurality of confidence rating outputs indicates a likelihood that the identified data corresponds to a respective gesture filter;

predicting an intended gesture based on a comparison of the confidence rating outputs;

determining that a variation between the captured data and the filter parameters for the intended gesture is below a threshold level; and

providing feedback representing a proper method for performing the intended gesture, wherein the feedback is indicative of one or more variations between the identified data and the one or more filter parameters that define the proper method performing the intended gesture;

a display device for rendering visual feedback representing the proper method for performing the intended gesture.

11. The system in accordance with claim 10, further comprising a networking component adapted to receive feedback representing the proper method for performing the intended gesture over a network.

12. The system in accordance with claim 10,

wherein the computer executable instructions further comprise instructions for determining whether the depth image includes a human target, and

wherein the capture device, based on a determination that the depth image includes a human target, scans the human target, tracks a skeletal model, and renders a visual representation that maps to the skeletal model.

13. The system in accordance with claim 10, wherein the computer executable instructions further comprise instructions for selecting the visual feedback to render from at least one of prerecorded content or live content received over a network.

14. The system in accordance with claim 10, wherein the computer executable instructions further comprise instructions for predicting an intended control from the identified data, wherein the visual feedback rendered represents the proper method for performing the intended gesture.

15. The system in accordance with claim 10, wherein the display device renders the visual feedback in real-time or in slow motion.

16. The system in accordance with claim 10, wherein the display device renders a visual representation that maps to the target'"'"'s motion or position in the physical space, wherein the visual representation is based on a skeletal representation.

17. The system in accordance with claim 10, wherein the visual feedback is a visual representation of the proper method for performing the intended gesture in a form of a skeletal representation, a ghosted image, or a player avatar.

18. The system in accordance with claim 10, wherein the display device renders the visual feedback by rendering at least one of:

a visual representation of the proper method for performing the intended gesture superimposed by a visual representation of the target'"'"'s motion or position in the physical space;

side-by-side images of a visual representation of the target'"'"'s motion or position in the physical space and a visual representation of the proper method for performing the intended gesture;

a visual representation of the target'"'"'s motion or position in the physical space, and corrected or highlighted feedback indicative of an error, such as a delta between an at least one gesture filter actual position and an ideal position;

or

a visual representation of the proper method for performing the intended gesture on a portion of a display space, wherein the portion is smaller than a total display space.

19. The system in accordance with claim 10, wherein the computer executable instructions further comprise instructions for identifying a trigger that initiates the rendering of the visual feedback, wherein the trigger is indicative of a failure in an execution of the target'"'"'s motion or position in the physical space, an initiation of an application, or a request for training.

20. A method for determining whether to provide feedback, the method comprising:

receiving image data for a scene, wherein the image data comprises depth data representative of a target'"'"'s motion or position in a physical space;

comparing the depth data representative of the target'"'"'s motion or position in the physical space to a plurality of gesture filters wherein each of said plurality of gesture filters comprises one or more parameters with associated thresholds and ranges;

identifying a confidence rating output for each of the comparisons to the plurality of gesture filters, wherein each of the plurality of confidence rating outputs indicates a likelihood that the depth data representative of the target'"'"'s motion or position corresponds to a respective gesture filter;

predicting an intended gesture based on a comparison of the confidence rating outputs;

detecting a variation between the depth data representative of the target'"'"'s motion or position to the parameters of the gesture filter associated with the intended gesture, wherein the variation is indicative of a failure in an execution of the intended gesture; and

determining whether to provide feedback based on said detected variation.