Systems and Methods for Tracking a Model

• US 20130100129A1
• Filed: 12/13/2012
• Published: 04/25/2013
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mathe, Zsolt
• Granted Patent: US 8,660,310 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device and a model of a user in the depth image may be generated. The background of a received depth image may be removed to isolate a human target in the received depth image. A model may then be adjusted to fit within the isolated human target in the received depth image. To adjust the model, a joint or a bone may be magnetized to the closest pixel of the isolated human target. The joint or the bone may then be refined such that the joint or the bone may be further adjusted to a pixel equidistant between two edges the body part of the isolated human target where the joint or bone may have been magnetized.

Claims

1. A method, comprising:

receiving a depth image of a scene;

isolating a target in the depth image;

determining that a joint of a computer model that corresponds to the target is outside an edge of the isolated target;

determining a new location for the joint of the computer model based on a pixel within the isolated target; and

updating the computer model based on the new location of the joint.

2. The method of claim 1, wherein the computer model comprises a skeletal model having joints and bones.

3. The method of claim 1, wherein determining the new location for the joint of the computer model based on the pixel within the isolated target comprises:

examining a set of pixels adjacent to the pixel within the isolated target to determine whether at least one of the pixels in the set of pixels has a valid depth value;

calculating distances between the pixel within the isolated target and each pixel in the set of pixels having a valid depth value; and

selecting a shortest distance from the calculated distances as the closest pixel.

4. The method of claim 1, wherein determining the new location for the joint of the computer model based on the pixel within the isolated target comprises:

determining the new location of the joint as having an X value, a Y value, and a Z value associated with the pixel.

5. The method of claim 1, wherein determining the new location for the joint of the computer model based on the pixel within the isolated target comprises:

selecting the pixel within the isolated target based on the pixel being equidistant from a first edge of the isolated target associated with a pixel closest to a prior location for the joint, and a second edge of the isolated target associated with a pixel opposite the pixel closest to the prior location for the joint.

6. The method of claim 5, wherein determining the new location for the joint of the computer model based on the pixel within the isolated target comprises:

determining the new location of the joint as having an X value, a Y value, and a Z value associated with the pixel that is equidistant from the first edge and the second edge.

7. The method of claim 1, wherein isolating the target in the depth image comprises:

removing data associated with a background of the depth image.

8. A computer-readable storage medium having stored thereon computer executable instructions for tracking a model, that when executed upon a computer, cause the computer to perform operations comprising:

receiving a depth image of a scene;

isolating a target in the depth image;

determining that a joint of a computer model that corresponds to the target is outside an edge of the isolated target;

determining a new location for the joint of the computer model based on a pixel within the isolated target; and

updating the computer model based on the new location of the joint.

9. The computer-readable storage medium of claim 8, wherein the computer model comprises a skeletal model having joints and bones.

10. The computer-readable storage medium of claim 8, wherein determining the new location for the joint of the computer model based on the pixel within the isolated target comprises:

examining a set of pixels adjacent to the pixel within the isolated target to determine whether at least one of the pixels in the set of pixels has a valid depth value;

calculating distances between the pixel within the isolated target and each pixel in the set of pixels having a valid depth value; and

selecting a shortest distance from the calculated distances as the closest pixel.

11. The computer-readable storage medium of claim 8, wherein determining the new location for the joint of the computer model based on the pixel within the isolated target comprises:

determining the new location of the joint as having an X value, a Y value, and a Z value associated with the pixel.

12. The computer-readable storage medium of claim 8, wherein determining the new location for the joint of the computer model based on the pixel within the isolated target comprises:

selecting the pixel within the isolated target based on the pixel being equidistant from a first edge of the isolated target associated with a pixel closest to a prior location for the joint, and a second edge of the isolated target associated with a pixel opposite the pixel closest to the prior location for the joint.

13. The computer-readable storage medium of claim 12, wherein determining the new location for the joint of the computer model based on the pixel within the isolated target comprises:

determining the new location of the joint as having an X value, a Y value, and a Z value associated with the pixel that is equidistant from the first edge and the second edge.

14. The computer-readable storage medium of claim 8, wherein isolating the target in the depth image comprises:

removing data associated with a background of the depth image.

15. A system for tracking a model, comprising:

a memory bearing instructions that, upon execution by a processor, cause the system to at least;

receive a depth image of a scene;

isolate a target in the depth image;

determine that a joint of a computer model that corresponds to the target is outside an edge of the isolated target;

determine a new location for the joint of the computer model based on a pixel within the isolated target; and

update the computer model based on the new location of the joint.

16. The system of claim 15, wherein the computer model comprises a skeletal model having joints and bones.

17. The system of claim 15, wherein the instructions that, upon execution by the processor, cause the system to at least determine the new location for the joint of the computer model based on the pixel within the isolated target further cause the system to at least:

examine a set of pixels adjacent to the pixel within the isolated target to determine whether at least one of the pixels in the set of pixels has a valid depth value;

calculate distances between the pixel within the isolated target and each pixel in the set of pixels having a valid depth value; and

select a shortest distance from the calculated distances as the closest pixel.

18. The system of claim 15, wherein the instructions that, upon execution by the processor, cause the system to at least determine the new location for the joint of the computer model based on the pixel within the isolated target further cause the system to at least:

determine the new location of the joint as having an X value, a Y value, and a Z value associated with the pixel.

19. The system of claim 15, wherein the instructions that, upon execution by the processor, cause the system to at least determine the new location for the joint of the computer model based on the pixel within the isolated target further cause the system to at least:

select the pixel within the isolated target based on the pixel being equidistant from a first edge of the isolated target associated with a pixel closest to a prior location for the joint, and a second edge of the isolated target associated with a pixel opposite the pixel closest to the prior location for the joint.

20. The system of claim 19, wherein the instructions that, upon execution by the processor, cause the system to at least determine the new location for the joint of the computer model based on the pixel within the isolated target further cause the system to at least:

determine the new location of the joint as having an X value, a Y value, and a Z value associated with the pixel that is equidistant from the first edge and the second edge.