Recognizing user intent in motion capture system

• US 8,334,842 B2
• Filed: 01/15/2010
• Issued: 12/18/2012
• Est. Priority Date: 01/15/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Markovic, Relja, Latta, Stephen G, Geisner, Kevin A, Bennett, Darren A, Steed, Jonathan T, Vance, Amos D
• Primary Examiner(s): OSORIO, RICARDO
• Application Number:
  US12/688,808
• Publication Number:
  US 20110175810A1
• Time in Patent Office: 1,068 Days
• Field of Search: 345/156, 348/64, 715/706, 715/709
• US Class Current: 345/158
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Techniques for facilitating interaction with an application in a motion capture system allow a person to easily begin interacting without manual setup. A depth camera system tracks a person in physical space and evaluates the person'"'"'s intent to engage with the application. Factors such as location, stance, movement and voice data can be evaluated. Absolute location in a field of view of the depth camera, and location relative to another person, can be evaluated. Stance can include facing a depth camera, indicating a willingness to interact. Movements can include moving toward or away from a central area in the physical space, walking through the field of view, and movements which occur while standing generally in one location, such as moving one'"'"'s arms around, gesturing, or shifting weight from one foot to another. Voice data can include volume as well as words which are detected by speech recognition.

Claims

1. A motion capture system, comprising:

a depth camera system having a field of view;

a display; and

at least one processor in communication with the depth camera system and the display, the at least one processor executes instructions to implement an application in the motion capture system, and provide a signal to the display to display images;

wherein;

the depth camera system and at least one processor, to track a first person and a second person in the field of view, distinguish the first person'"'"'s body and the second person'"'"'s body in the field of view, the second person engages with the application by controlling the second person'"'"'s body to control an avatar in a virtual space on the display, the second person is bound to the avatar, while the first person is not recognized as having an intent to engage with the application; and

the at least one processor, based on the tracking, allows the first person to engage with the application when the at least one processor determines that the first person has an intent to engage with the application, based on a location of the first person'"'"'s body relative to a location of the second person'"'"'s body in the field of view.

2. The motion capture system of claim 1, wherein:

based on the tracking, which determines an intent by the second person to disengage with the application, the at least one processor allows the first person to engage with the application in place of the second person, including binding the first person to the avatar in place of the second person, to allow the first person to control the avatar in the virtual space on the display by moving the first person'"'"'s body.

3. The motion capture system of claim 1, wherein:

based on the tracking, which determines an intent by the second person to remain engaged with the application, the at least one processor allows the first person to engage with the application by automatically generating, and binding the first person to, a separate avatar, to allow the first person to control the separate avatar in the virtual space on the display, while the second person is bound to, and controls, its avatar in the virtual space on the display.

4. The motion capture system of claim 1, wherein:

to allow the first person to engage with the application, the at least one processor divides the display into at least first and second regions, the first region provides the avatar of the first person and allows the first person to view the virtual space from a first point of view, and the second region provides the avatar of the second person and allows the second person to view the virtual space from a different, second point of view.

5. The motion capture system of claim 4, wherein:

the at least one processor swaps the avatar of the second person to the first region, and the avatar of the first person to the second region, when the tracking determines that the first person and second person have swapped positions in the field of view, so that the first region provides the avatar of the second person and allows the second person to view the virtual space from the second point of view, and the second region provides the avatar of the first person and allows the first person to view the virtual space from the first point of view.

6. The motion capture system of claim 4, wherein:

the at least one processor sets a relative position of the first and second regions on the display based on a relative position of the first person and the second person in the field of view, based on the tracking.

7. Tangible computer readable storage having computer readable software embodied thereon for programming at least one processor to perform a method in a motion capture system, the method comprising:

receiving images associated with a scene that includes a first person'"'"'s body in a field of view of the motion capture system;

based on the images;

distinguishing the first person'"'"'s body in the field of view, the first person interacts with an application by movement of the first person'"'"'s body to control an avatar in a virtual space on a display, identifying the movement of the first person'"'"'s body in the scene and distinguishing at least one additional person in the field of view, the at least one additional person does not control an avatar in the virtual space on the display; and

when a predefined criterion regarding a behavior of the at least one additional person is met, modifying at least one of a visual and audible output of the application.

8. The tangible computer readable storage of claim 7, wherein:

the predefined criterion comprises the at least one additional person moving in a way which indicates an increased level of interest in the application.

9. The tangible computer readable storage of claim 7, wherein:

the predefined criterion comprises the at least one additional person making a sound which indicates an increased level of interest in the application.

10. A processor-implemented method in a motion capture system, comprising the processor-implemented steps of:

receiving images associated with a scene that includes a first person'"'"'s body in a field of view of the motion capture system;

based on the images;

distinguishing the first person'"'"'s body in the field of view, the first person interacts with an application by movement of the first person'"'"'s body to control an avatar in a virtual space on a display, identifying the movement of the first person'"'"'s body in the scene and distinguishing at least one additional person in the field of view, the at least one additional person does not control an avatar in the virtual space on the display; and

when a predefined criterion regarding a behavior of the at least one additional person is met, modifying at least one of a visual and audible output of the application.

11. The processor-implemented method of claim 10, wherein:

the predefined criterion comprises the at least one additional person moving in a way which indicates an increased level of interest in the application.

12. The processor-implemented method of claim 10, wherein:

the predefined criterion comprises the at least one additional person making a sound which indicates an increased level of interest in the application.