Visual target tracking

• US 8,577,084 B2
• Filed: 12/07/2009
• Issued: 11/05/2013
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Geiss, Ryan M.
• Primary Examiner(s): Bali, Vikkram
• Assistant Examiner(s): CHEN, XUEMEI G
• Application Number:
  US12/632,587
• Publication Number:
  US 20100195869A1
• Time in Patent Office: 1,429 Days
• Field of Search: 701/2, 345/473, 345/474, 345/475, 348/169, 382/103
• US Class Current: 382/103
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A visual target tracking method includes representing a human target with a machine-readable model configured for adjustment into a plurality of different poses and receiving an observed depth image of the human target from a source. The observed depth image is compared to the model. A refine-z force vector is then applied to one or more force-receiving locations of the model to move a portion of the model towards a corresponding portion of the observed depth image if that portion of the model is Z-shifted from that corresponding portion of the observed depth image.

Claims

1. A method of tracking a human target, the method comprising:

representing the human target with a machine-readable model configured for adjustment into a plurality of different poses;

rasterizing the machine-readable model of the human target into a synthesized depth image, the synthesized depth image including a synthesized pixel of interest having a synthesized depth value;

receiving an observed depth image of the human target from a source, the observed depth image including an observed pixel having an observed depth value and a same pixel address as the synthesized pixel of interest;

classifying the synthesized pixel of interest with a refine-z pixel case if the synthesized depth value does not match the observed depth value and an absolute difference between the synthesized depth value and the observed depth value is less than a predetermined threshold;

computing a refine-z force vector for the synthesized pixel of interest, a magnitude of the refine-z force vector being based on the absolute difference between the synthesized depth value and the observed depth value such that the refine-z force vector increases in proportion to the absolute difference between the synthesized depth value and the observed depth value; and

mapping the refine-z force vector to one or more force-receiving locations of the machine-readable model representing the human target to move the machine-readable model towards the human target in the observed depth image.

2. The method of claim 1, where the refine-z force vector is applied along a Z axis perpendicular to an image plane.

3. A method of tracking a human target, the method comprising:

representing the human target with a machine-readable model configured for adjustment into a plurality of different poses;

receiving an observed depth image of the human target from a source;

comparing the observed depth image to a depth image representing the model; and

applying a refine-z force vector to a force-receiving location of the model to move a portion of the model towards a corresponding portion of the observed depth image if that portion of the model is Z-shifted from that corresponding portion of the observed depth image.

4. A method of tracking a human target, the method comprising:

representing the human target with a machine-readable model configured for adjustment into a plurality of different poses;

representing the machine-readable model of the human target in a synthesized depth image including a synthesized pixel of interest;

receiving an observed depth image of the human target from a source, the observed depth image including an observed pixel corresponding to the synthesized pixel of interest;

comparing a synthesized depth value of the synthesized pixel of interest to an observed depth value of the observed pixel; and

applying a force vector to one or more force-receiving locations of the machine-readable model to move the machine-readable model towards the observed depth image if the synthesized depth value does not match the observed depth value and an absolute difference between the synthesized depth value and the observed depth value is less than a predetermined threshold.

5. The method of claim 4, where representing the machine-readable model of the human target in the synthesized depth image includes rasterizing the machine-readable model of the human target as part of the synthesized depth image.

6. The method of claim 4, where the force vector is applied along a Z axis perpendicular to an image plane.

7. The method of claim 4, where the synthesized pixel of interest includes a body-part index corresponding to a body part of the machine-readable model, and where the force vector is applied along a vector normal to a face of the body part to which the body-part index corresponds.

8. The method of claim 4, where the force vector is applied along a vector normal to a surface comprising corresponding nearby observed pixels.

9. The method of claim 4, where a magnitude of the force vector is based on the absolute difference between the synthesized depth value and the observed depth value.

10. The method of claim 9, where the magnitude of the force vector increases in proportion to the absolute difference between the synthesized depth value and the observed depth value.

11. The method of claim 10, where the synthesized pixel of interest includes a body-part index corresponding to a body part of the machine-readable model, where the force vector is applied along a vector normal to a face of the body part to which the body-part index corresponds, and where the magnitude of the force vector is a scalar projection of an unscaled version of the force vector on the vector normal to the face.

12. The method of claim 10, where the force vector is applied along a vector normal to a surface comprising corresponding nearby observed pixels, and where the magnitude of the force vector is a scalar projection of an unscaled version of the force vector on the vector normal to the surface.

13. The method of claim 4, where the force-receiving location is a nearest qualifying force-receiving location to the synthesized pixel of interest.

14. The method of claim 13, where the nearest qualifying force-receiving location is a nearest joint of the machine-readable model.

15. The method of claim 13, where the nearest qualifying force-receiving location is a nearest centroid of the machine-readable model.

16. The method of claim 13, where the nearest qualifying force-receiving location is determined by comparing a distance between the synthesized pixel of interest and each qualifying force-receiving location.

17. The method of claim 13, where the nearest qualifying force-receiving location is determined by comparing a distance between the synthesized pixel of interest and each qualifying force-receiving location on a body part that is associated with a body-part index of the synthesized pixel of interest.

18. The method of claim 13, where the force vector is one of a plurality of force vectors applied to a weighted blend of nearest qualifying force-receiving locations.

19. The method of claim 13, where application of the force vector is biased towards relatively more mobile qualifying force-receiving locations.

20. The method of claim 13, where application of the force vector is biased to a less-than-nearest qualifying force-receiving location that is more mobile than the nearest qualifying force-receiving location.