Directed Performance In Motion Capture System

• US 20110175801A1
• Filed: 01/15/2010
• Published: 07/21/2011
• Est. Priority Date: 01/15/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Murphy, Brian S., Bennett, Darren A., Wright, Shawn C., Geisner, Kevin A., Markovic, Relja, Latta, Stephen G., Vuchetich, Christopher
• Granted Patent: US 8,284,157 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/305 for providing a graphical or textual hint to the player
• CPC Class Codes: A63F 2300/61 Score computation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Techniques for enhancing the use of a motion capture system are provided. A motion capture system tracks movement and audio inputs from a person in a physical space, and provides the inputs to an application, which displays a virtual space on a display. Bodily movements can be used to define traits of an avatar in the virtual space. The person can be directed to perform the movements by a coaching avatar, or visual or audio cues in the virtual space. The application can respond to the detected movements and voice commands or voice volume of the person to define avatar traits and initiate pre-scripted audio-visual events in the virtual space to provide an entertaining experience. A performance in the virtual space can be captured and played back with automatic modifications, such as alterations to the avatar'"'"'s voice or appearance, or modifications made by another person.

Claims

1. A processor-implemented method for directing a performance in a motion capture system, comprising the processor-implemented steps of:

tracking a person in a field of view of the motion capture system, the tracking includes distinguishing the person in the field of view;

displaying a virtual space on a display, the virtual space includes an avatar which represents the person;

directing movement of the person; and

based on the tracking, detecting the movement of the person, and updating the virtual space based on the detecting the movement of the person.

2. The processor-implemented method of claim 1, wherein:

the directing movement of the person comprises providing a visual cue on the display, the visual cue identifies a location in the virtual space to which the avatar is requested to move, and the person achieves the requested movement of the avatar by moving in the physical space.

3. The processor-implemented method of claim 1, wherein:

the directing movement of the person comprises providing a visual cue on the display, the visual cue identifies a direction in the virtual space in which the avatar is requested to move, and the person achieves the requested movement of the avatar by moving in the physical space.

4. The processor-implemented method of claim 1, wherein:

the directing movement of the person comprises directing the person to execute a specific bodily movement by providing an animated coach on the display, the animated coach demonstrates the specific bodily movement.

5. The processor-implemented method of claim 1, further comprising:

altering an appearance trait of the avatar based on the detecting the movement of the person.

6. The processor-implemented method of claim 1, wherein:

the directing movement of the person comprises directing the person to execute a specific bodily movement, the method further comprising determining an extent degree to which the person executes the specific bodily movement, the updating is responsive to the extent to which the person executes the specific bodily movement.

7. The processor-implemented method of claim 1, wherein:

the directing movement of the person comprises directing the person to execute a specific bodily movement, the method further comprising determining a number of times the person executes the specific bodily movement, the updating is responsive to the number of times the person executes the specific bodily movement.

8. The processor-implemented method of claim 1, further comprising:

altering a vocal trait of the avatar based on the detecting the movement of the person.

9. The processor-implemented method of claim 1, wherein:

the updating comprise providing a pre-scripted audio-visual event in the virtual space.

10. The processor-implemented method of claim 1, further comprising:

detecting a voice input of the person, and updating the virtual space based on the detecting the voice input of the person.

11. A motion capture system, comprising:

a depth camera system having a field of view;

a display;

a processor in communication with the depth camera system and the display, the processor executes instructions to;

(a) track a person in the field of view to determine movement of the person in the field of view, (b) display a virtual space on the display, (c) direct movement of the person in one or more time periods, and (d) based on the tracking, detect the movement of the person at the one or more time periods, and, in response, one or more performances in the virtual space, each performance is initiated based on a respective detected movement in a respective time period.

12. The motion capture system of claim 11, wherein the processor executes the instructions to:

(e) record the one or more performances, (f) and modify and play back the one or more performances so that the virtual space is viewed on the display from a different camera angle, relative to when the one or more performances were initiated.

13. The motion capture system of claim 11, wherein the processor executes the instructions to:

(e) provide an avatar on the display which moves in the virtual word correspondingly to the movement of the person in the one or more time periods, (f) record the one or more performances, and (g) modify and play back the one or more performances so that the avatar has a modified vocal trait, relative to when the one or more performances were initiated.

14. The motion capture system of claim 11, wherein the processor executes the instructions to:

(e) provide an avatar on the display which moves in the virtual word correspondingly to the movement of the person in the one or more time periods, (f) record the one or more performances, and (g) modify and play back the one or more performances so that the avatar has a modified costume, relative to when the one or more performances were initiated.

15. Tangible computer readable storage having computer readable software embodied thereon for programming at least one processor to perform a method for providing a performance in a motion capture system, the method comprising:

tracking movement of at least one person in at least one field of view of at least one depth camera system to determine movement of the at least one person in the at least one field of view;

displaying a virtual space on at least one display, the virtual space includes an avatar which moves in the virtual space; and

based on the tracking, defining and exhibiting a first trait for the avatar in a first time period based on movement of the at least one person during the first time period, recording information which indicates that the first trait was defined for the avatar in the first time period, and subsequently defining and exhibiting a second trait for the avatar in a second time period based on movement of the at least one person during the second time period, the avatar also exhibits the first trait in the second time period based on the recorded information.

16. The tangible computer readable storage of claim 15, wherein:

at least one of the first and second traits is defined based on a voice volume of the at least one person.

17. The tangible computer readable storage of claim 15, wherein:

at least one of the first and second traits comprises a size of a body part of at least part of the avatar.

18. The tangible computer readable storage of claim 15, wherein:

the first trait is defined based on movement of a first person in the at least one field of view during the first time period, and the second trait is defined based on movement of a different, second person in the at least one field of view during the second time period.

19. The tangible computer readable storage of claim 18, wherein:

the first and second persons are in a same field of view.

20. The tangible computer readable storage of claim 18, wherein:

the first and second persons are in different fields of view.