Visual target tracking

• US 8,577,085 B2
• Filed: 12/07/2009
• Issued: 11/05/2013
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Geiss, Ryan M.
• Primary Examiner(s): Bali, Vikkram
• Assistant Examiner(s): CHEN, XUEMEI G
• Application Number:
  US12/632,652
• Publication Number:
  US 20100197392A1
• Time in Patent Office: 1,429 Days
• Field of Search: 345/420, 345/473, 345/474, 345/475, 348/169, 382/103
• US Class Current: 382/103
• CPC Class Codes: A63F 2300/1006 having additional degrees of freedom
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models

Abstract

A target tracking method includes modeling the target in a first frame with a first frame iteration of a machine-readable model and receiving an observed depth image of a second frame of a scene including the target. The first frame iteration of the machine-readable model is then adjusted into a second frame iteration of the machine-readable model based on the observed depth image of the second frame.

Claims

1. A method of tracking a target in a plurality of successive frames, the method comprising:

receiving a previous frame iteration of a machine-readable model representing a pose of the target in the previous frame;

receiving an observed depth image of a scene including the target, the observed depth image imaging the scene in a current frame; and

adjusting the previous frame iteration of the machine-readable model into a current frame iteration of the machine-readable model based on a comparison of the observed depth image of the current frame and a depth image representing the previous frame iteration of the machine-readable model.

2. The method of claim 1, where finding the current frame iteration of the machine-readable model includes moving one or more portions of the previous frame iteration of the machine-readable model based on the observed depth image.

3. The method of claim 2, where a portion of the previous frame iteration of the machine-readable model is moved so that a depth value at a pixel of the machine-readable model more closely matches a corresponding depth value of a pixel of the observed depth image.

4. The method of claim 1, where the machine-readable model includes a skeletal model including a plurality of skeletal points, each skeletal point defining a three-dimensional world space position for that skeletal point.

5. The method of claim 4, where finding the current frame iteration of the machine-readable model includes calculating a world space direction and a world space distance to move one or more of the plurality of skeletal points from the previous frame to the current frame.

6. The method of claim 5, where the world space direction and the world space distance are calculated for a skeletal point so that a depth value at a pixel of the machine-readable model proximate to the skeletal point more closely matches a corresponding depth value of a pixel of the observed depth image.

7. The method of claim 5, where finding the current frame iteration of the machine-readable model includes adjusting the previous frame iteration of the machine-readable model into an adjusted model based on momentum derived from two or more previous frame iterations of the machine-readable model and rasterizing the adjusted model into a synthesized depth image, the synthesized depth image including a synthesized depth value at each pixel rasterized from the adjusted model.

8. The method of claim 7, where rasterizing the adjusted model into the synthesized depth image includes labeling a first pixel of the synthesized depth image with an original body-part index indicating a forward-most part of the adjusted model at the first pixel.

9. The method of claim 8, where rasterizing the adjusted model into the synthesized depth image includes labeling the first pixel of the synthesized depth image with a snap body-part index indicating a point on a surface of the machine-readable model that is closest to an observed depth value at the first pixel.

10. The method of claim 9, where rasterizing the adjusted model into the synthesized depth image includes labeling one or more neighboring pixels neighboring the first pixel with a grow body-part index equal to the snap body-part index of the first pixel if the neighboring pixels do not have a snap body-part index corresponding to a body part and depth values of the neighboring pixels are within a tolerance of a depth value of the first pixel.

11. The method of claim 1, where the current frame iteration is found without looking up the current frame iteration in a database.

12. A method of tracking a target in a plurality of successive frames, the method comprising:

modeling the target in a first frame with a first frame iteration of a machine-readable model;

receiving an observed depth image of a second frame of a scene including the target; and

adjusting the first frame iteration of the machine-readable model into a second frame iteration of the machine-readable model based on a comparison of the observed depth image of the second frame and a depth image representing the first frame iteration of the machine-readable model.

13. The method of claim 12, where adjusting the first frame iteration of the machine-readable model into the second frame iteration of the machine-readable model includes moving one or more portions of the first frame iteration of the machine-readable model based on the observed depth image.

14. The method of claim 13, where a portion of the first frame iteration of the machine-readable model is moved so that a depth value at a pixel of the second frame iteration of the machine-readable model more closely matches a corresponding depth value of a pixel of the observed depth image than a depth value at a pixel of the first frame iteration of the machine-readable model.

15. The method of claim 12, where the machine-readable model includes a skeletal model including a plurality of skeletal points, each skeletal point defining a three-dimensional world space position for that skeletal point.

16. The method of claim 15, where adjusting the first frame iteration of the machine-readable model into the second frame iteration of the machine-readable model includes calculating a world space direction and a world space distance to move one or more of the plurality of skeletal points from the first frame to the second frame.

17. A computing system for tracking a target in a plurality of successive frames, the computing system comprising:

an input to receive depth information from a source configured to capture the depth information;

a logic subsystem operatively connected to the input; and

a data-holding subsystem holding instructions executable by the logic subsystem to;

model the target in a first frame with a first frame iteration of a machine-readable model;

receive from the source via the input an observed depth image of a second frame of a scene including the target; and

adjust the first frame iteration of the machine-readable model into a second frame iteration of the machine-readable model based on a comparison of the observed depth image of the second frame and a depth image representing the first frame iteration of the machine-readable model.

18. The computing system of claim 17, where the data-holding subsystem holds instructions executable by the logic subsystem so that adjusting the first frame iteration of the machine-readable model into the second frame iteration of the machine-readable model includes moving one or more portions of the first frame iteration of the machine-readable model based on the observed depth image.

19. The computing system of claim 17, where the data-holding subsystem holds instructions executable by the logic subsystem so that a portion of the first frame iteration of the machine-readable model is moved so that a depth value at a pixel of the second frame iteration of the machine-readable model more closely matches a corresponding depth value of a pixel of the observed depth image than a depth value at a pixel of the first frame iteration of the machine-readable model.

20. The computing system of claim 17, where the source includes a depth camera.