Tracking groups of users in motion capture system

• US 7,961,174 B1
• Filed: 07/30/2010
• Issued: 06/14/2011
• Est. Priority Date: 01/15/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hill, David, Bennett, Darren A, Haley, David C, Geisner, Kevin A, Latta, Stephen G, Murphy, Brian S, Wright, Shawn C, Markovic, Relja
• Primary Examiner(s): Hjerpe; Richard
• Assistant Examiner(s): HARRIS, DOROTHY H
• Application Number:
  US12/847,133
• Time in Patent Office: 319 Days
• Field of Search: 345156-158
• US Class Current: 345/158
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6072 of an input signal, e.g. pitch and rhythm extraction, voice recognition
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/8017 Driving on land or water; Flying
• CPC Class Codes: A63F 2300/8023 the game being played by multiple players at a common site, e.g. in an arena, theatre, shopping mall using a large public display
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

In a motion capture system, a unitary input is provided to an application based on detected movement and/or location of a group of people. Audio information from the group can also be used as an input. The application can provide real-time feedback to the person or group via a display and audio output. The group can control the movement of an avatar in a virtual space based on the movement of each person in the group, such as in a steering or balancing game. To avoid a discontinuous or confusing output by the application, missing data can be generated for a person who is occluded or partially out of the field of view. A wait time can be set for activating a new person and deactivating a currently-active person. The wait time can be adaptive based on a first detected position or a last detected position of the person.

Claims

1. A processor-implemented method for tracking multiple people in a motion capture system, comprising the processor-implemented steps of:

using the motion capture system, tracking at least one group which includes a first group, the first group comprises peoples'"'"' bodies in a field of view of the motion capture system, the tracking includes detecting the peoples'"'"' bodies, including sizes of the peoples'"'"' bodies, in the field of view;

based on the tracking, detecting representative locations of the peoples'"'"' bodies in the field of view, detecting a point location of the first group as a first unitary entity based on the representative locations of the peoples'"'"' bodies in the field of view, and determining movement of the point location of the first group as the first unitary entity based on movement of the representative locations of the peoples'"'"' bodies in the field of view; and

providing inputs to an application which are based on the movement of the point location of the first group, the application displays a virtual space on a display, and updates the display based on the inputs.

2. The processor-implemented method of claim 1, wherein:

the detecting the sizes of the peoples'"'"' bodies includes detecting a width of each of the peoples'"'"' bodies.

3. The processor-implemented method of claim 1, further comprising:

based on the detecting the peoples'"'"' bodies in the field of view, determining a number of people in the first group who perform a common movement, the inputs to the application are based on the number of people in the first group who perform the common movement, so that the first group controls the application based on the number of people in the first group who perform the common movement.

4. The processor-implemented method of claim 3, further comprising:

based on the detecting the peoples'"'"' bodies in the field of view, for the people in the first group who perform the common movement, determining an extent to which the people in the first group perform the common movement, the inputs to the application are based on the extent to which the common movement is performed.

5. The processor-implemented method of claim 4, wherein:

for each person in the first group who performs the common movement, the extent to which the common movement is performed is based on an extent of movement in an angular range.

6. The processor-implemented method of claim 4, wherein:

the extent to which the common movement is performed is based on a number of times the common movement is performed by each person in the first group who performs the common movement.

7. The processor-implemented method of claim 1, further comprising:

tracking a voice volume of the first group, where the inputs to the application are based on the voice volume, so that the first group controls the application based on the voice volume.

8. The processor-implemented method of claim 1, wherein the first group comprises at least one other person'"'"'s body in a field of view of another motion capture system, the method further comprising:

receiving data via a network from the another motion capture system, the data indicates movement of a representative location of the at least one other person'"'"'s body in the field of view of the another motion capture system, wherein the movement of the point location of the first group is determined based on the movement of the representative location of the at least one other person'"'"'s body in the field of view of the another motion capture system.

9. The processor-implemented method of claim 1, wherein the at least one group includes a second group, the method further comprising:

tracking the second group, the second group comprises peoples'"'"' bodies in the field of view of the motion capture system;

based on the tracking of the second group, determining movement of a point location of the second group as a second unitary entity based on movement of representative locations of the peoples'"'"' bodies in the second group; and

determining a multi-group movement which is based on the movement of the point location of the first group and the movement of the point location of the second group, at a common time point;

wherein the inputs provided to the application are based on the multi-group movement.

10. The processor-implemented method of claim 1, wherein:

virtual space provided by the application includes at least one avatar on a display, and the first group controls the at least one avatar based on the movement of the first group by causing the avatar to move in a direction in the virtual space which is based on the movement of the first group.

11. The processor-implemented method of claim 10, wherein:

the application comprises a balancing game in which the first group controls a balance of the at least one avatar based on the movement of the first group.

12. The processor-implemented method of claim 10, wherein:

the application comprises a steering game in which the avatar comprises a vehicle, and the first group controls a steering direction of the vehicle based on the movement of the first group.