Tracking Groups Of Users In Motion Capture System

• US 20110175809A1
• Filed: 01/15/2010
• Published: 07/21/2011
• Est. Priority Date: 01/15/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Haley, David C. JR., Geisner, Kevin A., Hill, David, Markovic, Relja, Latta, Stephen G., Murphy, Brian S., Bennett, Darren A., Wright, Shawn C.
• Granted Patent: US 8,933,884 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6072 of an input signal, e.g. pitch and rhythm extraction, voice recognition
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/8017 Driving on land or water; Flying
• CPC Class Codes: A63F 2300/8023 the game being played by multiple players at a common site, e.g. in an arena, theatre, shopping mall using a large public display
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

In a motion capture system, a unitary input is provided to an application based on detected movement and/or location of a group of people. Audio information from the group can also be used as an input. The application can provide real-time feedback to the person or group via a display and audio output. The group can control the movement of an avatar in a virtual space based on the movement of each person in the group, such as in a steering or balancing game. To avoid a discontinuous or confusing output by the application, missing data can be generated for a person who is occluded or partially out of the field of view. A wait time can be set for activating a new person and deactivating a currently-active person. The wait time can be adaptive based on a first detected position or a last detected position of the person.

Claims

1. A processor-implemented method for tracking multiple people in a motion capture system, comprising the processor-implemented steps of:

tracking at least one group which includes a first group, the first group comprises peoples'"'"' bodies in a field of view of the motion capture system, the tracking includes detecting the peoples'"'"' bodies in the field of view;

based on the tracking, determining movement of the first group as a first unitary entity based on movement of the people in the first group; and

providing inputs to an application which are based on the movement of the first group as the first unitary entity, the application displays a virtual space on a display, and updates the display based on the inputs.

2. The processor-implemented method of claim 1, wherein:

the determining movement of the first group as the first unitary entity comprises determining movement of a representative point location of the first group, the representative point location of the first group is based on locations of the peoples'"'"' bodies in the field of view.

3. The processor-implemented method of claim 2, wherein:

the determining movement of the first group as the first unitary entity comprises determining movement of a representative point location of the first group, the representative point location of the first group is based on sizes of the of the peoples'"'"' bodies in the field of view.

4. The processor-implemented method of claim 1, wherein:

the application provides at least one avatar on the display, and the first group controls the at least one avatar based on the movement of the first group by causing the avatar to move in a direction in the virtual space which is based on the movement of the first group in the field of view.

5. The processor-implemented method of claim 4, wherein:

the application comprises at least one of;

(a) a balancing game, and the first group controls a balance of the at least one avatar based on the direction in which the first group moves in the field of view, and (b)

a steering game in which the avatar comprises a vehicle, and the first group controls a steering direction of the vehicle based on the direction in which the first group moves in the field of view.

6. The processor-implemented method of claim 1, wherein:

based on the tracking, determining a number of people in the first group who perform a common movement, the inputs to the application are based on the number of people in the first group who perform the common movement, so that the first group controls the application based on the number of people in the first group who perform the common movement.

7. The processor-implemented method of claim 1, further comprising:

based on the tracking, for the people in the first group who perform the common movement, determining a degree to which they perform the common movement, the inputs to the application are based on the degree to which the common movement is performed, so that the first group controls the application based on the degree to which the common movement is performed.

8. The processor-implemented method of claim 1, further comprising:

tracking a voice volume of the first group, where the inputs to the application are based on the voice volume, so that the first group controls the application based on the voice volume.

9. The processor-implemented method of claim 1, further comprising:

receiving data via a network from another motion capture system which tracks at least one other person'"'"'s body in a field of view of the another motion capture system, the data indicates movement of the at least one other person'"'"'s body in the field of view of the another motion capture system, and the movement of the first group as the first unitary entity is determined based on the movement of the people in the first group, including the movement of the at least one other person.

10. The processor-implemented method of claim 1, wherein the at least one group includes a second group, the method further comprising:

tracking the second group, the second group comprises peoples'"'"' bodies in the field of view of the motion capture system;

based on the tracking of the second group, determining movement of the second group as a second unitary entity based on movement of the people in the second group; and

determining a multi-group movement which is based on the movement of the first group and the movement of the second group;

wherein the inputs provided to the application are based on the multi-group movement.

11. A motion capture system, comprising:

a depth camera system having a field of view;

a display;

at least one processor associated with the depth camera system and the display, the at least one processor executes instructions to;

(a) track a person in the field of view to determine movement of the person in the field of view and to provide data which represents one portion of the person which is visible to the depth camera system, where another portion of the person is not visible to the depth camera system, (b) map the data to a corresponding portion of a skeletal model, (c) generate data which represents the another portion of the person which is not visible to the depth camera system, (d) map the generated data to a remaining portion of the skeletal model to provide a mapped skeletal model, (e) provide an avatar which represents the person based on the mapped skeletal model, so that the avatar represents both the visible and not visible portions of the person, and (f) provide a signal to the display to display images, where the images include the avatar, and the avatar is controlled responsive to the movement.

12. The motion capture system of claim 11, wherein:

the at least one processor updates the avatar as the visible and not visible portions of the person vary in different frames of image data which are provided by the depth camera system.

13. The motion capture system of claim 11, wherein:

the not visible portion of the person is out of the field of view.

14. The motion capture system of claim 11, wherein:

the not visible portion of the person is occluded in the field of view.

15. Tangible computer readable storage having computer readable software embodied thereon for programming at least one processor to perform a method in a motion capture system, the method comprising:

based on frames of image data of a field of view of a depth camera system, tracking at least one person'"'"'s body in the field of view, the tracking provides results for use by an application in providing an avatar in a virtual space on a display, such that the avatar represents the least one person'"'"'s body and is controlled by movement of the at least one person'"'"'s body;

determining when the results from the tracking are consistent with at least one predetermined scenario which poses a risk of causing the application to provide an erroneous output on the display; and

based on the determining, reducing the risk of causing the application to provide the erroneous output on the display.

16. The tangible computer readable storage of claim 15, wherein:

the at least one predetermined scenario comprises the at least one person'"'"'s body moving a distance which is more than a realistic distance in the field of view in a specific time interval; and

based on the results from the tracking being consistent with the at least one predetermined scenario, the reducing the risk comprises imposing a limit on movement of the avatar on the display.

17. The tangible computer readable storage of claim 15, wherein the at least one predetermined scenario comprises the tracking failing to detect the at least one person'"'"'s body in the field of view, and the application erroneously concluding that the at least one person no longer intends to interact with the application;

and

based on the results from the tracking being consistent with the at least one predetermined scenario, the reducing the risk comprises imposing a wait time before the application concludes that the at least one person no longer intends to interact with the application, where the wait time is a function of a distance of a last detected location of the at least one person'"'"'s body in the field of view from an edge of the field of view, and the wait time is longer when the distance is greater.

18. The tangible computer readable storage of claim 17, wherein:

the application deactivates the avatar in the virtual space to represent that the at least one person is no longer interacting with the application, upon completion of the minimum wait time.

19. The tangible computer readable storage of claim 15, wherein the at least one predetermined scenario comprises at least one of:

(a) the tracking erroneously detecting at least one new person'"'"'s body in the field of view, and (b) the application erroneously concluding that the at least one new person intends to interact with the application; and

based on the results from the tracking being consistent with the at least one predetermined scenario, the reducing the risk comprises imposing a minimum wait time before the application concludes that the at least one new person is interacting with the application, where the minimum wait time is a function of a distance of a first detected location of the at least one new person'"'"'s body in the field of view from an edge of the field of view, and the minimum wait time is longer when the distance is greater.

20. The tangible computer readable storage of claim 19, wherein:

the application displays a new avatar in the virtual space on the display to represent the at least one new person upon completion of the minimum wait time.