Directed performance in motion capture system

• US 8,465,108 B2
• Filed: 09/05/2012
• Issued: 06/18/2013
• Est. Priority Date: 01/15/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Markovic, Relja, Latta, Stephen G, Geisner, Kevin A, Vuchetich, Christopher, Bennett, Darren A, Murphy, Brian S, Wright, Shawn C
• Primary Examiner(s): OSORIO, RICARDO
• Application Number:
  US13/603,912
• Publication Number:
  US 20120326976A1
• Time in Patent Office: 286 Days
• Field of Search: 345/156, 715/706, 715/704, 348/64
• US Class Current: 305/156
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/305 for providing a graphical or textual hint to the player
• CPC Class Codes: A63F 2300/61 Score computation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Techniques for enhancing the use of a motion capture system are provided. A motion capture system tracks movement and audio inputs from a person in a physical space, and provides the inputs to an application, which displays a virtual space on a display. Bodily movements can be used to define traits of an avatar in the virtual space. The person can be directed to perform the movements by a coaching avatar, or visual or audio cues in the virtual space. The application can respond to the detected movements and voice commands or voice volume of the person to define avatar traits and initiate pre-scripted audio-visual events in the virtual space to provide an entertaining experience. A performance in the virtual space can be captured and played back with automatic modifications, such as alterations to the avatar'"'"'s voice or appearance, or modifications made by another person.

Claims

1. A motion capture system, comprising:

a depth camera system, the depth camera system obtains images of a field of view;

a display; and

a processor in communication with the depth camera system and the display, the processor executes instructions to;

display a virtual space comprising an avatar on the display, provide directions to a person, the person performs movements in the field of view in a first time period in response to the directions, process the images to detect the movements of the person, update the virtual space so that the avatar provides a performance, the avatar exhibits a trait and moves correspondingly to the movements of the person in real time as the person performs the movements in the performance, and provide a play back of the performance in a second time period, the avatar exhibits a modification to the trait and moves correspondingly to the movements of the person in the play back of the performance.

2. The motion capture system of claim 1, wherein:

the trait comprises a costume of the avatar.

3. The motion capture system of claim 2, wherein:

the costume of the avatar is exchanged with another avatar in the virtual space.

4. The motion capture system of claim 1, wherein:

the trait comprises at least one of a hair color or a hair style of the avatar.

5. The motion capture system of claim 1, wherein:

the modification to the trait comprises a modulation of a voice.

6. The motion capture system of claim 1, wherein:

the trait comprises a facial expression.

7. The motion capture system of claim 1, wherein:

the trait comprises a manner of walking.

8. The motion capture system of claim 1, wherein:

the trait comprises a vocal trait.

9. The motion capture system of claim 8, wherein:

the vocal trait comprises at least one of tone, accent, cadence, rhythm, intonation, degree of articulation or loudness.

10. The motion capture system of claim 1, wherein:

the trait comprises a language, such that the avatar speaks in a different language in the play back of the performance in the second time period than in the performance in the first time period.

11. A processor-implemented method in a motion capture system, comprising the processor-implemented steps of:

receiving images of a person in a field of view of the motion capture system, the person performs movements in the field of view;

displaying a virtual space on a display, the virtual space comprises an avatar which represents the person;

based on the images, detecting the movements of the person;

in a first time period, creating a scene in a performance in the virtual space by translating the movements of the person to movements of an avatar in the virtual space as the person explores different features of the virtual space; and

in a second time period, playing back the performance by depicting the movements of the avatar with a modification of the scene.

12. The processor-implemented method of claim 11, wherein:

the modification of the scene comprise a different viewpoint of the virtual space in the second time period than in the first time period.

13. The processor-implemented method of claim 11, wherein:

the modification of the scene comprise a different lighting in the second time period than in the first time period.

14. The processor-implemented method of claim 11, wherein:

the modification of the scene comprise an enhanced soundtrack of the scene.

15. A computer readable storage device comprising computer readable software embodied thereon for programming a processor to perform a method in a motion capture system, the method comprising:

receiving images of a person in a field of view of the motion capture system;

displaying a virtual space on a display, the virtual space comprises an avatar which represents the person;

based on the images, detecting the person;

displaying the avatar in the virtual space;

providing a plurality of directed moments in a performance of the avatar, each directed moment of the plurality of directed moments requires the person to complete a directed movement and each directed movement must be successfully completed before a next directed moment of the plurality of directed moments, until a final directed moment of the plurality of directed moments is reached, the avatar moves correspondingly to the directed movements of the person to interact with objects in the virtual space;

providing a record of the performance of the avatar; and

once the person has successfully completed the final directed moment, automatically modifying the record of the performance to provide a modified record, and playing back the modified record.

16. The computer readable storage device of claim 15, wherein:

the directed movements are indicated by a plurality of visual cues in the virtual space.

17. The computer readable storage device of claim 16, wherein:

one visual cue of the plurality of visual cues is on a ground in the virtual space; and

the person performs one of the directed movements by causing the avatar to stand on the one of the visual cues.

18. The computer readable storage device of claim 15, wherein:

providing an optional moment in the performance of the avatar, the optional moment provides a visual cue in the virtual space for an action to perform; and

providing a visual reward in response to completion of the action by the person.

19. The computer readable storage device of claim 15, wherein:

providing a discoverable moment in the performance of the avatar, the discoverable moment comprises an action that is discoverable by the person but is not identified by a visual cue in the virtual space; and

providing a result in the virtual space in response to discovery of the action by the person.

20. The computer readable storage device of claim 15, wherein:

the modified record provides a surprise final moment of the performance.