VISUAL TARGET TRACKING

• US 20100197393A1
• Filed: 12/07/2009
• Published: 08/05/2010
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Geiss, Ryan M.
• Granted Patent: US 8,565,477 B2
• Time in Patent Office: Days
• US Class Current: 463/30
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/6615 using models with different levels of detail [LOD]
• CPC Class Codes: A63F 2300/8029 Fighting without shooting
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

A target tracking method includes representing a human target with a machine-readable model configured for adjustment into a plurality of different poses and receiving an observed depth image of the human target from a source. One or more push force vectors are applied to one or more force-receiving locations of the model to push the model in an XY plane towards a silhouette of the human target in the observed depth image when portions of the model are shifted away from the silhouette of the human target in the observed depth image. One or more pull force vectors are applied to one or more force-receiving locations of the model to pull the model in an XY plane towards the silhouette of the human target in the observed depth image when portions of the observed depth image are shifted away from the silhouette of the model.

Claims

1. A method of tracking a human target, the method comprising:

representing the human target with a machine-readable model configured for adjustment into a plurality of different poses;

rasterizing the machine-readable model of the human target as part of a synthesized depth image including a synthesized pixel of interest;

receiving an observed depth image of the human target from a source, the observed depth image including an observed pixel corresponding to the synthesized pixel of interest; and

if an observed depth value of the observed pixel is less than a synthesized depth value of the synthesized pixel of interest by more than a pull threshold amount, applying a pull force vector to a force-receiving location of the model to pull the model toward the synthesized pixel of interest;

or

if the observed depth value is greater than the synthesized depth value by more than a push threshold amount, applying a push force vector to a force-receiving location of the model to push the model away from the synthesized pixel of interest.

2. The method of claim 1, where a magnitude of the pull force vector is proportional to a pull-offset distance between the synthesized pixel of interest and a nearest qualifying pixel on a silhouette of the model.

3. The method of claim 2, where the magnitude of the pull force vector (D2), in screen-space, is:

D2=2*(D1−

0.5 pixels), where D1 is the pull-offset distance, in pixels.

4. The method of claim 2, where a direction of the pull force vector is parallel to a vector extending from the nearest qualifying pixel on the silhouette of the model to the synthesized pixel of interest.

5. The method of claim 2, where the nearest qualifying pixel on the silhouette of the model is found using a one dimensional search along a gradient of a blurred height map.

6. The method of claim 5, where the nearest qualifying pixel on the silhouette of the model is found by testing silhouette pixels near a silhouette pixel found using the one dimensional search.

7. The method of claim 1, where a magnitude of the push force vector is proportional to a push-offset distance between the synthesized pixel of interest and a nearest qualifying pixel on a silhouette of the human target in the observed depth image.

8. The method of claim 7, where the magnitude of the push force vector (D2), in screen space, is:

D2=2*(D1−

0.5 pixels), where D1 is the push-offset distance, in pixels.

9. The method of claim 7, where a direction of the push force vector is parallel to a vector extending from the synthesized pixel of interest to the nearest qualifying pixel on the silhouette of the human target in the observed depth image.

10. The method of claim 7, where the nearest qualifying pixel on the silhouette of the human target in the observed depth image is found using a one dimensional search along a gradient of a blurred height map.

11. The method of claim 10, where the nearest qualifying pixel on the silhouette of the human target is found by testing silhouette pixels near a silhouette pixel found using the one dimensional search.

12. The method of claim 1, where the push force vector or the pull force vector is a three-dimensional vector including a Z-component.

13. A method of tracking a human target, the method comprising:

representing the human target with a machine-readable model configured for adjustment into a plurality of different poses;

receiving an observed depth image of the human target from a source;

for portions of the model shifted away from a silhouette of the human target in the observed depth image, applying one or more push force vectors to one or more force-receiving locations of the model to push the model in an XY plane towards the silhouette of the human target in the observed depth image; and

for portions of the observed depth image shifted away from a silhouette of the model, applying one or more pull force vectors to one or more force-receiving locations of the model to pull the model in an XY plane towards the silhouette of the human target in the observed depth image.

14. The method of claim 13, where a magnitude of each push force vector is proportional to a push-offset distance with which a portion of the model is shifted away from the silhouette of the human target in the observed depth image.

15. The method of claim 14, where the push-offset distance is found using a one dimensional search along a gradient of a blurred height map.

16. The method of claim 13, where a magnitude of each pull force vector is proportional to a pull-offset distance with which a portion of the observed depth image is shifted away from the silhouette of the model.

17. The method of claim 16, where the pull-offset distance is found using a one dimensional search along a gradient of a blurred height map.

18. The method of claim 13, where the push force vectors and the pull force vectors are three-dimensional vectors including Z-components.

19. A method of tracking a human target, the method comprising:

representing the human target with a machine-readable model configured for adjustment into a plurality of different poses;

rasterizing the machine-readable model of the human target as part of a synthesized depth image, the synthesized depth image including a synthesized pixel of interest having a synthesized depth value;

receiving an observed depth image of the human target from a source, the observed depth image including an observed pixel corresponding to the synthesized pixel of interest and having an observed depth value; and

if the synthesized depth value is less than the observed depth value by more than a push threshold amount, then;

classifying the synthesized pixel of interest with a push pixel case;

finding a push-offset distance between the synthesized pixel of interest and a silhouette of the human target in the observed depth image;

computing a push force vector for the synthesized pixel of interest, a magnitude of the push force vector being based on the push-offset distance; and

mapping the push force vector to a force-receiving location of the machine-readable model representing the human target to push the machine-readable model in an XY plane towards the silhouette of the human target in the observed depth image; and

if the synthesized depth value is greater than the observed depth value by more than a pull threshold amount, then;

classifying the synthesized pixel of interest with a pull pixel case;

finding a pull-offset distance between the synthesized pixel of interest and a silhouette of the model in the synthesized depth image;

computing a pull force vector for the synthesized pixel of interest, a magnitude of the pull force vector being based on the pull-offset distance; and

mapping the pull force vector to a force-receiving location of the machine-readable model representing the human target to pull the machine-readable model in an XY plane towards the silhouette of the human target in the observed depth image.

20. The method of claim 19, where the push force vector or the pull force vector is a three-dimensional vector including a Z-component.