Directed performance in motion capture system

• US 8,284,157 B2
• Filed: 01/15/2010
• Issued: 10/09/2012
• Est. Priority Date: 01/15/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Markovic, Relja, Latta, Stephen G, Geisner, Kevin A, Vuchetich, Christopher, Bennett, Darren A, Murphy, Brian S, Wright, Shawn C
• Primary Examiner(s): OSORIO, RICARDO
• Application Number:
  US12/688,804
• Publication Number:
  US 20110175801A1
• Time in Patent Office: 998 Days
• Field of Search: 345/156, 348/64, 715/706, 715/709
• US Class Current: 345/156
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/305 for providing a graphical or textual hint to the player
• CPC Class Codes: A63F 2300/61 Score computation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Techniques for enhancing the use of a motion capture system are provided. A motion capture system tracks movement and audio inputs from a person in a physical space, and provides the inputs to an application, which displays a virtual space on a display. Bodily movements can be used to define traits of an avatar in the virtual space. The person can be directed to perform the movements by a coaching avatar, or visual or audio cues in the virtual space. The application can respond to the detected movements and voice commands or voice volume of the person to define avatar traits and initiate pre-scripted audio-visual events in the virtual space to provide an entertaining experience. A performance in the virtual space can be captured and played back with automatic modifications, such as alterations to the avatar'"'"'s voice or appearance, or modifications made by another person.

Claims

1. A processor-implemented method for directing a performance in a motion capture system, comprising the processor-implemented steps of:

receiving images of a person in a field of view of the motion capture system;

based on the images, distinguishing the person in the field of view;

displaying a virtual space on a display, the virtual space includes an avatar which represents the person;

directing movement of the person; and

based on the images, detecting the movement of the person, and updating the virtual space based on the detecting the movement of the person, the updating comprise providing a pre-scripted audio-visual event in the virtual space.

2. The processor-implemented method of claim 1, wherein:

the directing movement of the person comprises providing a visual cue on the display, the visual cue identifies a location in the virtual space to which the avatar is requested to move in a requested movement, and the person achieves the requested movement of the avatar by moving in a physical space.

3. The processor-implemented method of claim 1, wherein:

the directing movement of the person comprises providing a visual cue on the display, the visual cue identifies a direction in the virtual space in which the avatar is requested to move in a requested movement, and the person achieves the requested movement of the avatar by moving in a physical space.

4. The processor-implemented method of claim 1, wherein:

the directing movement of the person comprises directing the person to execute a specific bodily movement by providing an animated coach on the display, the animated coach demonstrates the specific bodily movement.

5. The processor-implemented method of claim 1, further comprising:

altering an appearance trait of the avatar based on the detecting the movement of the person.

6. The processor-implemented method of claim 1, wherein:

the directing movement of the person comprises directing the person to execute a specific bodily movement, the method further comprising determining an extent which the person executes the specific bodily movement, the updating is responsive to the extent to which the person executes the specific bodily movement.

7. The processor-implemented method of claim 1, wherein:

the directing movement of the person comprises directing the person to execute a specific bodily movement, the method further comprising determining a number of times the person executes the specific bodily movement, the updating is responsive to the number of times the person executes the specific bodily movement.

8. The processor-implemented method of claim 1, further comprising:

altering a vocal trait of the avatar based on the detecting the movement of the person.

9. Tangible computer readable storage having computer readable software embodied thereon for programming at least one processor to perform a method for providing a performance in a motion capture system, the method comprising:

receiving images of at least one person in at least one field of view of at least one depth camera system;

based on the images, determining movement of the at least one person in the at least one field of view;

displaying a virtual space on at least one display, the virtual space includes an avatar which moves in the virtual space; and

based on the images, defining and exhibiting a first trait for the avatar in a first time period based on movement of the at least one person during the first time period, recording information which indicates that the first trait was defined for the avatar in the first time period, and subsequently defining and exhibiting a second trait for the avatar in a second time period based on movement of the at least one person during the second time period, the avatar also exhibits the first trait in the second time period based on the recorded information.

10. The tangible computer readable storage of claim 9, wherein:

at least one of the first and second traits is defined based on a voice volume of the at least one person.

11. The tangible computer readable storage of claim 9, wherein:

at least one of the first and second traits comprises a size of a body part of at least part of the avatar.

12. The tangible computer readable storage of claim 9, wherein:

the first trait is defined based on movement of a first person in the at least one field of view during the first time period, and the second trait is defined based on movement of a different, second person in the at least one field of view during the second time period.

13. The tangible computer readable storage of claim 12, wherein:

the first and second persons are in a same field of view.

14. The tangible computer readable storage of claim 12, wherein:

the first and second persons are in different fields of view.

15. A processor-implemented method for providing a performance in a motion capture system, comprising the processor-implemented steps of:

receiving images of at least one person in at least one field of view of at least one depth camera system;

based on the images, determining movement of the at least one person in the at least one field of view;

displaying a virtual space on at least one display, the virtual space includes an avatar which moves in the virtual space; and

based on the images, defining and exhibiting a first trait for the avatar in a first time period based on movement of the at least one person during the first time period, recording information which indicates that the first trait was defined for the avatar in the first time period, and subsequently defining and exhibiting a second trait for the avatar in a second time period based on movement of the at least one person during the second time period, the avatar also exhibits the first trait in the second time period based on the recorded information.

16. The processor-implemented method of claim 15, wherein:

at least one of the first and second traits is defined based on a voice volume of the at least one person.

17. The processor-implemented method of claim 15, wherein:

at least one of the first and second traits comprises a size of a body part of at least part of the avatar.

18. The processor-implemented method of claim 15, wherein:

the first trait is defined based on movement of a first person in the at least one field of view during the first time period, and the second trait is defined based on movement of a different, second person in the at least one field of view during the second time period.

19. The processor-implemented method of claim 18, wherein:

the first and second persons are in a same field of view.

20. The processor-implemented method of claim 18, wherein:

the first and second persons are in different fields of view.