Visual target tracking

• US 8,588,465 B2
• Filed: 12/07/2009
• Issued: 11/19/2013
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Geiss, Ryan M.
• Primary Examiner(s): Bali, Vikkram
• Assistant Examiner(s): CHEN, XUEMEI G
• Application Number:
  US12/632,599
• Publication Number:
  US 20100197395A1
• Time in Patent Office: 1,443 Days
• Field of Search: 345/473, 345/474, 345/475, 348/169, 382/103, 701/2
• US Class Current: 382/103
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/6615 using models with different levels of detail [LOD]
• CPC Class Codes: A63F 2300/8029 Fighting without shooting
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models

Abstract

A method of tracking a target includes classifying a pixel having a pixel address with one or more pixel cases. The pixel is classified based on one or more observed or synthesized values. An example of an observed value for a pixel address includes an observed depth value obtained from a depth camera. Examples of synthesized values for a pixel address include a synthesized depth value calculated by rasterizing a model of the target; one or more body-part indices estimating a body part corresponding to that pixel address; and one or more player indices estimating a target corresponding to that pixel address. One or more force vectors are calculated for the pixel based on the pixel case, and the force vector is mapped to one or more force-receiving locations of the model representing the target to adjust the model representing the target into an adjusted pose.

Claims

1. A method of tracking a human target, the method comprising:

receiving an observed depth image of the human target from a source, the observed depth image including a plurality of observed pixels;

obtaining a synthesized depth image of a model representing the human target, the synthesized depth image including a plurality of synthesized pixels;

classifying a synthesized pixel of the plurality of synthesized pixels with a pixel case;

computing a force vector for the synthesized pixel based on the pixel case of the synthesized pixel; and

mapping the force vector to one or more force-receiving locations of the model representing the human target to adjust the model representing the human target into an adjusted pose.

2. The method of claim 1, where the synthesized depth image is obtained, at least in part, by rasterizing the model representing the human target.

3. The method of claim 1, where the synthesized pixel is classified with the pixel case based on at least a body-part index of the synthesized pixel.

4. The method of claim 3, where the synthesized pixel is classified with the pixel case, at least in part, based on a difference between an original body-part index of the synthesized pixel and a snap body-part index of the synthesized pixel or a grow body-part index of the synthesized pixel.

5. The method of claim 1, where the synthesized pixel is classified with the pixel case based on at least a player index of the synthesized pixel.

6. The method of claim 5, where the synthesized pixel is classified with the pixel case, at least in part, based on a difference between an original player index of the synthesized pixel and a snap player index of the synthesized pixel or a grow player index of the synthesized pixel.

7. The method of claim 1, where the synthesized pixel is classified with the pixel case based on at least a depth-value difference between the synthesized pixel and a corresponding observed pixel.

8. The method of claim 1, where the synthesized pixel has a pixel address, and where the synthesized pixel at the pixel address is classified with the pixel case based on one or more of an original player index, a snap player index, a grow player index, an original body-part index, a snap body-part index, a grow body-part index, an original synthesized depth value, a snap depth value, and an observed depth value for that pixel address.

9. The method of claim 1, where the pixel case is selected from a set of defined pixel cases including a refine-z pixel case, a magnetism pixel case, a push pixel case, a pull pixel case, a self-occluding push pixel case, and a self-occluding pull pixel case.

10. The method of claim 1, where the force vector mapped to the force-receiving location is one of a plurality of force vectors mapped to that force-receiving location, each such force vector corresponding to a synthesized pixel and classified pixel case, and each such force vector contributing to adjustment of the model representing the human target into an adjusted pose.

11. A method of tracking a target, the method comprising:

representing the target with a model including a plurality of portions, each portion associated with a part index corresponding to a part of the target;

classifying a portion of the model with a pixel case based on one or both of the part index associated with the portion of the model and a difference between a modeled position of the portion of the model and an observed position of a corresponding portion of the target; and

adjusting the model in accordance with the pixel case of the portion of the model.

12. The method of claim 11, where the target is a human target and the part index is a body-part index corresponding to a part of the human target.

13. The method of claim 11, where each portion of the model includes one or more pixels, each pixel having a pixel address, an original body-part index, a snap body-part index, and a grow body-part index;

and where the portion of the model is classified with the pixel case based on, for each of the one or more pixels, the original body-part index, the snap body-part index, the grow body-part index, and the difference between the modeled position of the portion of the model at that pixel address and the observed position of the corresponding portion of the target at that pixel address.

14. The method of claim 11, where the pixel case is selected from a set of defined pixel cases including a refine-z pixel case, a magnetism pixel case, a push pixel case, a pull pixel case, a null pixel case, a self-occluding push pixel case, and a self-occluding pull pixel case.

15. The method of claim 11, where the plurality of portions of the model correspond to a plurality of synthesized pixels of a synthesized depth image, and classifying the portion of the model with the pixel case includes classifying a synthesized pixel corresponding to that portion of the model.

16. A method of tracking a human target, the method comprising:

receiving an observed depth image of the human target from a source, the observed depth image including a plurality of observed pixels;

obtaining a synthesized depth image of a model representing the human target, the synthesized depth image including a plurality of synthesized pixels, each synthesized pixel including a body-part index corresponding to a body part of the human target;

classifying a first synthesized pixel having a first body-part index corresponding to a first part of the human target with a first pixel case;

classifying a second synthesized pixel having a second body-part index corresponding to a second part of the human target with a second pixel case;

computing a first force vector for the first synthesized pixel in accordance with the first pixel case;

computing a second force vector for the second synthesized pixel in accordance with the second pixel case;

mapping the first force vector to a first force-receiving location of the model, the first force-receiving location corresponding to the first part of the human target; and

mapping the second force vector to a second force-receiving location of the model, the second force-receiving location corresponding to the second part of the human target.

17. The method of claim 16, where the synthesized depth image is obtained by rasterizing the model representing the human target.

18. The method of claim 16, where the first pixel case is a magnetism pixel case and the second pixel case includes one or more of a refine-z pixel case, a push pixel case, a pull pixel case, a self-occluding push pixel case, and a self-occluding pull pixel case.

19. The method of claim 18, where the second synthesized pixel is classified with a pixel case based on a difference between the second synthesized pixel and a corresponding observed pixel.

20. The method of claim 18, where the second synthesized pixel is classified with a pixel case based on a difference between an original body-part index of the second synthesized pixel and a snap body-part index of the second synthesized pixel or a grow body-part index of the second synthesized pixel.