Systems and methods for tracking a model

• US 8,483,436 B2
• Filed: 11/04/2011
• Issued: 07/09/2013
• Est. Priority Date: 10/07/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lee, Johnny Chung, Leyvand, Tommer, Stachniak, Simon Piotr, Peeper, Craig
• Primary Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US13/289,823
• Publication Number:
  US 20120057753A1
• Time in Patent Office: 613 Days
• Field of Search: 382/100, 382/103, 382/106, 382/107, 382/128, 382/129, 382/130, 382/131, 382/132, 128/922, 378 4- 27, 345/418, 345/419, 345/632, 345/619, 345/630, 345/629, 345/467, 345/468, 348169-172
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04N 13/20 Image signal generators

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A model may be adjusted based on a location or position of one or more extremities estimated or determined for a human target in the grid of voxels. The model may also be adjusted based on a default location or position of the model in a default pose such as a T-pose, a DaVinci pose, and/or a natural pose.

Claims

1. A method for tracking a user comprising:

receiving a depth image that was captured by a depth camera;

identifying an estimated location or position of a part of the user in the depth image;

adjusting a model of the user based on the estimated location or position of the part of the user; and

in response to a failure to identify a location or position of the part of the user in a second depth image, adjusting the model to move the location or position of the part of the user in the model to a default position of the part of the user.

2. The method of claim 1, further comprising:

mapping the model to an on screen avatar.

3. The method of claim 1, further comprising:

providing the model to a gesture library.

4. The method of claim 1, wherein the model comprises a skeletal model having joints and bones.

5. The method of claim 4, wherein adjusting the model comprises:

adjusting a joint of the skeletal model to the estimated location or position.

6. The method of claim 1, further comprising:

determining that the estimated location or position is valid.

7. The method of claim 1, further comprising:

adjusting the model based on a geometric constraint.

8. The method of claim 1, wherein the default pose comprises:

a T-Pose, a Di Vinci pose, or a natural pose.

9. The method of claim 1, further comprising:

magnetizing the model to one or more of the pixels in the depth image.

10. The method of claim 1, further comprising:

determining that a pose associated with the model is valid.

11. The method of claim 1, further comprising:

refining the model based on the received depth image.

12. The method of claim 1, further comprising:

downsampling the pixels in the depth image before identifying the estimated location or position of the extremity of the user.

13. A computer readable storage medium for tracking a user, the computer readable storage medium having stored thereon computer executable instructions that, when executed on a computer, cause the computer to perform operations comprising:

receiving a depth image;

identifying an estimated location or position of a part of the user in the depth image;

adjusting a model of the user based on the estimated location or position of the part of the user; and

in response to a failure to identify a location or position of the part of the user in a second depth image, adjusting the model to move the location or position of the part of the user in the model to a default position of the part of the user.

14. The computer readable storage medium of claim 13, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform operations comprising:

mapping the model to an on screen avatar.

15. The computer readable storage medium of claim 13, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform operations comprising:

providing the model to a gesture library.

16. The computer readable storage medium of claim 13, wherein the model comprises:

a skeletal model having joints and bones.

17. The computer readable storage medium of claim 13, wherein the instructions further comprise:

determining whether the estimated location or position is valid.

18. A system for tracking a user comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, when executed on the processor, cause the system to at least;

receive a depth image;

identify an estimated location or position of a part of the user in the depth image;

adjust a model of the user based on the estimated location or position of the part of the user; and

in response to a failure to identify a location or position of the part of the user in a second depth image, adjust the model to move the location or position of the part of the user in the model to a default position of the part of the user.

19. The system of claim 18, wherein the memory further bears processor-executable instructions that, when executed on the processor, cause the system to at least:

map the model to an on screen avatar.

20. The system of claim 18, wherein the memory further bears processor-executable instructions that, when executed on the processor, cause the system to at least:

provide the model to a gesture library.