SYSTEMS AND METHODS FOR TRACKING A MODEL

• US 20120057753A1
• Filed: 11/04/2011
• Published: 03/08/2012
• Est. Priority Date: 10/07/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lee, Johnny Chung, Leyvand, Tommer, Stachniak, Simon Piotr, Peeper, Craig
• Granted Patent: US 8,483,436 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04N 13/20 Image signal generators

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A model may be adjusted based on a location or position of one or more extremities estimated or determined for a human target in the grid of voxels. The model may also be adjusted based on a default location or position of the model in a default pose such as a T-pose, a DaVinci pose, and/or a natural pose.

Claims

1. A method for tracking a user comprising:

receiving information having a plurality of depth elements;

generating a data structure based in part on the depth elements;

using the data structure to make an estimation of a location or position of an extremity of the user; and

adjusting a model based on the estimation.

2. The method of claim 1, further comprising:

mapping the model to an on screen avatar.

3. The method of claim 1, further comprising:

providing the model to a gesture library.

4. The method of claim 1, wherein the model comprises a skeletal model having joints and bones.

5. The method of claim 4, wherein adjusting the model comprises:

adjusting a joint of the skeletal model to the estimated location or position.

6. The method of claim 1, further comprising:

determining that the estimated location or position is valid.

7. The method of claim 1, further comprising:

adjusting the model based on a geometric constraint.

8. The method of claim 1, further comprising relaxing a body part of the model using a default pose.

9. The method of claim 8, wherein the default pose comprises:

a T-Pose, a Di Vinci pose, or a natural pose.

10. The method of claim 1, further comprising:

magnetizing the model to at least part of the data structure based in part on the depth elements.

11. The method of claim 1, further comprising:

determining that a pose associated with the model is valid.

12. The method of claim 1, further comprising:

refining the model based on the received information having a plurality of depth elements.

13. The method of claim 1, further comprising:

downsampling the pixels in the received information having a plurality of depth elements to generate the data structure based in part on the depth elements.

14. A computer readable storage medium for tracking a user, the computer readable storage medium having stored thereon computer executable instructions comprising:

receiving information having a plurality of depth elements;

generating a data structure based in part on the depth elements;

using the data structure to make an estimation of a location or position of an extremity of the user; and

adjusting a model based on the estimation.

15. The computer readable storage medium of claim 14, wherein the instructions further comprise:

mapping the model to an on screen avatar.

16. The computer readable storage medium of claim 14, wherein the instructions further comprise:

providing the model to a gesture library.

17. The computer readable storage medium of claim 14, wherein the model comprises:

a skeletal model having joints and bones.

18. The computer readable storage medium of claim 17, wherein adjusting the model comprises:

adjusting a joint of the skeletal model to the estimated location or position.

19. The computer readable storage medium of claim 14, wherein the instructions further comprise:

determining whether the estimated location or position is valid.

20. A system for tracking a user comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, when executed on the processor, cause the system to at least;

receive information having a plurality of depth elements;

generate a data structure based in part on the depth elements;

use the data structure to make an estimation of a location or position of an extremity of the user; and

adjust a model based on the estimation.