Systems and methods for tracking a model

• US 8,320,619 B2
• Filed: 06/15/2009
• Issued: 11/27/2012
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mathe, Zsolt
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US12/484,592
• Publication Number:
  US 20100303290A1
• Time in Patent Office: 1,261 Days
• Field of Search: 382/100, 382/103, 382/106, 382/107, 382/123, 382/155, 382/162, 382/168, 382/180, 382/181, 382189-199, 382/219, 382/232, 382/254, 382/274, 382/276, 382/291, 382/312, 382/305, 375/240.08, 375/240.12, 482/57
• US Class Current: 382/103
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device and a model of a user in the depth image may be generated. The background of a received depth image may be removed to isolate a human target in the received depth image. A model may then be adjusted to fit with in the isolated human target in the received depth image. To adjust the model, a joint or a bone may be magnetized to the closest pixel of the isolated human target. The joint or the bone may then be refined such that the joint or the bone may be further adjusted to a pixels equidistant between two edges the body part of the isolated human target where the joint or bone may have been magnetized.

Claims

1. A method of creating a mask of an object in a scene, the method comprising:

receiving a depth image of a scene;

identifying an object in the depth image by filling the object;

comparing the object to a pattern;

removing a background surrounding the object in response to determining that the object matches the pattern to produce a mask of the object; and

calculating a plurality of measurements of the object based on the mask of the object.

2. The method of claim 1, wherein identifying the object in the depth image by filling the object comprises:

separating objects in the depth image based on a plurality of depth values of a plurality of pixels in the image,

assigning a first pixel with a first depth value a first color based on the first depth value;

assigning a second pixel with a second depth value a second color based on the second depth value;

generating a colorized image of the scene using a plurality of pixels with assigned colors, the first and second colors of the first and second pixels visually depicting the depth distances of objects in the scene; and

identifying the object based on the assigned colors of the pixels.

3. The method of claim 1, wherein calculating the plurality of measurements of the object comprises:

measuring a first width of the object at a first position and measuring a second width of the object at a second position.

4. The method of claim 1, wherein identifying the object in the depth image by filling the object comprises:

changing a color of the object such that the color of the object is distinguishable from a color of the background.

5. The method of claim 1, wherein comparing the object to the pattern comprises:

comparing the object to the pattern, the pattern being associated with a body model of a human in a position.

6. The method of claim 1, wherein comparing the object to the pattern comprises:

comparing the object to a plurality of patterns, each pattern of the plurality of patterns being associated with a body model of a human in a position or pose.

7. The method of claim 1, wherein removing the background surrounding the object comprises:

removing a value for a pixel of the background from the depth image.

8. The method of claim 1, wherein removing the background surrounding the object comprises:

filling the background in a manner different from filling the object.

9. The method of claim 1, wherein calculating a plurality of measurements of the target comprises:

measuring a plurality of width, length, or location measurements of the target.

10. The method of claim 1, further comprising:

calculating a skeletal model of the object based on the plurality of measurements of the object.

11. The method of claim 1, wherein the object is a human, a first measurement of the plurality of measurements corresponds to the top of the human'"'"'s head, and further comprising:

scanning the mask of the object downward from the top of the human'"'"'s head to determine the location of the neck or shoulders of the human.

12. A system for creating a mask of an object in a scene, comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, when executed on the processor, cause the system to at least;

receive a depth image of a scene;

identify an object in the depth image by filling the object;

compare the object to a pattern;

remove a background surrounding the object in response to determining that the object matches the pattern to produce a mask of the object; and

calculate a plurality of measurements of the object based on the mask of the object.

13. The system of claim 12, wherein the processor-executable instructions that, when executed on the processor, cause the system to at least identify the object in the depth image by filling the object further cause the system to at least:

separate objects in the depth image based on a plurality of depth values of a plurality of pixels in the image,

assign a first pixel with a first depth value a first color based on the first depth value;

assign a second pixel with a second depth value a second color based on the second depth value;

generate a colorized image of the scene using a plurality of pixels with assigned colors, the first and second colors of the first and second pixels visually depicting the depth distances of objects in the scene; and

identify the object based on the assigned colors of the pixels.

14. The system of claim 12, wherein the processor-executable instructions that, when executed on the processor, cause the system to at least calculate the plurality of measurements of the object further cause the system to at least:

measure a first width of the object at a first position and measuring a second width of the object at a second position.

15. The system of claim 12, wherein the processor-executable instructions that, when executed on the processor, cause the system to at least identify the object in the depth image by filling the object further cause the system to at least:

change a color of the object such that the color of the object is distinguishable from a color of the background.

16. The system of claim 12, wherein the processor-executable instructions that, when executed on the processor, cause the system to at least compare the object to the pattern further cause the system to at least:

compare the object to the pattern, the pattern being associated with a body model of a human in a position.

17. The system of claim 12, wherein the processor-executable instructions that, when executed on the processor, cause the system to at least compare the object to the pattern further cause the system to at least:

compare the object to a plurality of patterns, each pattern of the plurality of patterns being associated with a body model of a human in a position or pose.

18. The system of claim 12, wherein the processor-executable instructions that, when executed on the processor, cause the system to at least remove the background surrounding the object further cause the system to at least:

remove a value for a pixel of the background from the depth image.

19. A computer-readable storage medium for creating a colorized image of a scene, bearing computer-executable instructions that, when executed on a computer, cause the computer to perform operations comprising:

receiving a depth image of a scene;

separating objects in the depth image based on a plurality of depth values of a plurality of pixels in the image,

assigning a first pixel with a first depth value a first color based on the first depth value;

assigning a second pixel with a second depth value a second color based on the second depth value; and

generating a colorized image of the scene using a plurality of pixels with assigned colors, the first and second colors of the first and second pixels visually depicting the depth distances of objects in the scene.

20. The computer-readable storage medium of claim 19, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform comprising:

identifying an object in the depth image based on the assigned colors of the pixels;

comparing the object to a pattern;

removing a background surrounding the object in response to determining that the object matches the pattern to produce a mask of the object; and

calculating a plurality of measurements of the object based on the mask of the object.