Visual target tracking

• US 8,565,476 B2
• Filed: 12/07/2009
• Issued: 10/22/2013
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Geiss, Ryan M.
• Primary Examiner(s): Bali, Vikkram
• Assistant Examiner(s): CHEN, XUEMEI G
• Application Number:
  US12/632,672
• Publication Number:
  US 20100197400A1
• Time in Patent Office: 1,415 Days
• Field of Search: 345/420, 345/473, 345/474, 345/475, 348/169, 382/103
• US Class Current: 382/103
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/6615 using models with different levels of detail [LOD]
• CPC Class Codes: A63F 2300/8029 Fighting without shooting
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A target tracking method includes representing a human target with a machine-readable model configured for adjustment into a plurality of different poses. The machine-readable model includes a plurality of joints, including one or more magnetism joints, and each joint has a three-dimensional world space position. The method further includes receiving an observed depth image of the human target from a source. The observed depth image includes a plurality of observed pixels. A magnetism body part is assigned to one or more of the plurality of observed pixels, and a magnetism joint position is estimated based on world space positions of the one or more observed pixels assigned the magnetism body part. A joint of the machine-readable model is then shifted toward the magnetism joint position.

Claims

1. A method of tracking a human target, the method comprising:

representing the human target with a machine-readable model configured for adjustment into a plurality of different poses, the machine-readable model including a plurality of joints, including one or more magnetism joints, each joint having a three-dimensional world space position;

receiving an observed depth image of the human target from a source, the observed depth image including a plurality of observed pixels;

assigning a magnetism body part to one or more of the plurality of observed pixels;

estimating a magnetism joint position based on world space positions of the one or more observed pixels assigned the magnetism body part; and

shifting a joint of the machine-readable model toward the magnetism joint position.

2. The method of claim 1, further comprising assigning a non-magnetism body part to another of the plurality of observed pixels and shifting the machine-readable model toward the another of the plurality of observed pixels.

3. The method of claim 1, where assigning a magnetism body part to an observed pixel includes finding a closest bone of the machine-readable model to that observed pixel, and assigning that observed pixel a body part associated with that bone.

4. The method of claim 1, where the magnetism body part corresponds to an arm of the machine-readable model.

5. The method of claim 1, where the magnetism body part corresponds to a hand of the machine-readable model.

6. The method of claim 1, where the magnetism body part corresponds to a leg of the machine-readable model.

7. A method of tracking a human target, the method comprising:

representing the human target with a machine-readable model configured for adjustment into a plurality of different poses, the machine-readable model including a plurality of joints, including one or more magnetism joints, each joint having a three-dimensional world space position;

receiving an observed depth image of the human target from a source, the observed depth image including a plurality of observed pixels;

assigning a magnetism body part to one or more of the plurality of observed pixels;

calculating a centroid for observed pixels assigned the magnetism body part;

estimating a magnetism joint position based on the centroid; and

applying a magnetism force vector to a joint of the machine-readable model to shift the joint toward the magnetism joint position.

8. The method of claim 7, further comprising assigning a non-magnetism body part to another of the plurality of observed pixels and applying a non-magnetism force vector to the machine-readable model to shift the machine-readable model toward the another of the plurality of observed pixels.

9. The method of claim 7, where the centroid is one of a plurality of centroids calculated for observed pixels assigned the magnetism body part.

10. The method of claim 7, where calculating a centroid includes calculating an orthodox centroid where all contributing observed pixels are weighted equally.

11. The method of claim 7, where calculating a centroid includes calculating a biased centroid where some contributing observed pixels are weighted differently than other contributing observed pixels.

12. The method of claim 7, where estimating the magnetism joint position includes estimating the magnetism joint position to be a position of the centroid.

13. The method of claim 7, where estimating the magnetism joint position includes estimating the magnetism joint position to be a position found by projecting from a known location through the centroid by a known bone length.

14. The method of claim 7, where assigning a magnetism body part to an observed pixel includes finding a closest bone of the machine-readable model to that observed pixel, and assigning that observed pixel a body part associated with that bone.

15. The method of claim 7, where the magnetism body part corresponds to an arm of the machine-readable model.

16. The method of claim 7, where the magnetism body part corresponds to a hand of the machine-readable model.

17. The method of claim 7, where the magnetism body part corresponds to a leg of the machine-readable model.

18. The method of claim 7, where the magnetism body part corresponds to a foot of the machine-readable model.

19. The method of claim 7, where the magnetism body part corresponds to a head of the machine-readable model.

20. A method of tracking a human target, the method comprising:

representing the human target with a machine-readable model configured for adjustment into a plurality of different poses, the machine-readable model including a plurality of joints, including one or more limb joints, each joint having a three-dimensional world space position;

receiving an observed depth image of the human target from a source, the observed depth image including a plurality of observed pixels, each observed pixel having a world space depth;

assigning a limb body part to one or more of the plurality of observed pixels;

determining to which of one or more bones of the limb body part each observed pixel is closest;

calculating a centroid for observed pixels assigned to each of the one or more bones of the limb body part;

estimating one or more magnetism joint positions for the limb body part, based on the one or more centroids associated with the limb body part;

applying one or more magnetism force vectors to each of one or more limb joints of the machine-readable model to shift the one or more limb joints toward the one or more magnetism joint positions;

assigning a non-limb body part to another of the plurality of observed pixels; and

applying one or more non-magnetism force vectors to a non-limb body part of the machine-readable model to shift the non-limb body part toward the another of the plurality of observed pixels.