Systems and methods for tracking a model

• US 8,897,495 B2
• Filed: 05/08/2013
• Issued: 11/25/2014
• Est. Priority Date: 10/07/2009
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lee, Johnny Chung, Leyvand, Tommer, Stachniak, Simon Piotr, Peeper, Craig C.
• Primary Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US13/889,901
• Publication Number:
  US 20130243257A1
• Time in Patent Office: 566 Days
• Field of Search: 382/100, 382/103, 382/106, 382/107, 345/418, 345/419, 345/632, 345/619, 345/630, 345/629, 345/467, 345/468, 348/169, 348/170, 348/171, 348/172
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04N 13/20 Image signal generators

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A model may be adjusted based on a location or position of one or more extremities estimated or determined for a human target in the grid of voxels. The model may also be adjusted based on a default location or position of the model in a default pose such as a T-pose, a DaVinci pose, and/or a natural pose.

Claims

1. A method for tracking a user, comprising:

receiving a depth image that was captured by a depth camera;

identifying an estimated location or position of a part of the user in the depth image;

adjusting a model of the user based on the estimated location or position of the part of the user; and

in response to a failure to identify a location or position of the part of the user in a second depth image, associating the part of the user with a portion of the second depth image based on a location or position of a default position of the part of the user.

2. The method of claim 1, further comprising:

mapping the model to an on-screen avatar.

3. The method of claim 1, further comprising:

providing the model to a gesture library.

4. The method of claim 1, wherein the model comprises a skeletal model having joints and bones.

5. The method of claim 4, wherein associating the part of the user with the portion of the second depth image:

adjusting a joint of the skeletal model to the portion of the second depth image.

6. The method of claim 1, further comprising:

determining that the estimated location or position is valid.

7. The method of claim 1, further comprising:

adjusting the model based on a geometric constraint.

8. The method of claim 1, wherein the default pose comprises:

a T-Pose, a Di Vinci pose, or a natural pose.

9. The method of claim 1, further comprising:

magnetizing the model to one or more pixels in the depth image.

10. The method of claim 1, further comprising:

determining that a pose associated with the model is valid.

11. The method of claim 1, further comprising:

refining the model based on the received depth image.

12. The method of claim 1, further comprising:

downsampling pixels in the depth image before identifying the estimated location or position of the part of the user.

13. A computer readable storage device for tracking a user, the computer readable storage medium having stored thereon computer executable instructions that, when executed on a computer, cause the computer to perform operations comprising:

receiving a depth image;

identifying an estimated location or position of a part of the user in the depth image;

adjusting a model of the user based on the estimated location or position of the part of the user; and

in response to a failure to identify a location or position of the part of the user in a second depth image, associating the part of the user with a portion of the second depth image based on a location or position of a default position of the part of the user.

14. The computer readable storage device of claim 13, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform operations comprising:

mapping the model to an on-screen avatar.

15. The computer readable storage device of claim 13, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform operations comprising:

providing the model to a gesture library.

16. The computer readable storage device of claim 13, wherein the model comprises:

a skeletal model having joints and bones.

17. The computer readable storage device of claim 13, wherein the instructions further comprise:

determining whether the estimated location or position is valid.

18. A system for tracking a user, comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, when executed on the processor, cause the system to at least;

receive a depth image;

identify an estimated location or position of a part of the user in the depth image;

adjust a model of the user based on the estimated location or position of the part of the user; and

in response to a failure to identify a location or position of the part of the user in a second depth image, associate the part of the user with a portion of the second depth image based on a location or position of a default position of the part of the user.

19. The system of claim 18, wherein the memory further bears processor-executable instructions that, when executed on the processor, cause the system to at least:

map the model to an on-screen avatar.

20. The system of claim 18, wherein the memory further bears processor-executable instructions that, when executed on the processor, cause the system to at least:

provide the model to a gesture library.