Systems and methods for tracking a model

• US 8,891,827 B2
• Filed: 11/15/2012
• Issued: 11/18/2014
• Est. Priority Date: 10/07/2009
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lee, Johnny Chung, Leyvand, Tommer, Stachniak, Szymon Piotr, Peeper, Craig
• Primary Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US13/678,288
• Publication Number:
  US 20130070058A1
• Time in Patent Office: 733 Days
• Field of Search: 382/100, 382/103, 382/106, 382/107, 382/128, 382/129, 382/130, 382/131, 382/132, 128/922, 378 4- 27, 345/418, 345/419, 345/632, 345/619, 345/630, 345/629, 345/467, 345/468, 348/169, 348/170, 348/171, 348/172
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04N 13/20 Image signal generators

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A model may be adjusted based on a location or position of one or more extremities estimated or determined for a human target in the grid of voxels. The model may also be adjusted based on a default location or position of the model in a default pose such as a T-pose, a DaVinci pose, and/or a natural pose.

Claims

1. A method for tracking a user, comprising:

receiving a depth image, the depth image being captured by a depth camera;

identifying an estimated location or position of an extremity of the user in the depth image;

adjusting a model based on the estimated location or position of the extremity; and

based at least on determining that a location or position of the extremity has not been estimated from a second depth image, adjusting the model to move the location or position of the extremity to a default position of the extremity.

2. The method of claim 1, further comprising:

mapping the model to an on screen avatar.

3. The method of claim 1, further comprising:

providing the model to a gesture library.

4. The method of claim 1, wherein the model comprises a skeletal model having joints and bones.

5. The method of claim 4, wherein adjusting the model comprises:

adjusting a joint of the skeletal model to the estimated location or position.

6. The method of claim 1, further comprising:

determining that the estimated location or position is valid.

7. The method of claim 1, further comprising:

adjusting the model based on a geometric constraint.

8. The method of claim 1, wherein the default pose comprises at least one of the following:

a T-Pose, a Di Vinci pose, and a natural pose.

9. The method of claim 1, further comprising:

magnetizing the model to one or more pixels in the depth image.

10. The method of claim 1, further comprising:

determining that a pose associated with the model is valid.

11. The method of claim 1, further comprising:

refining the model based on the received depth image.

12. The method of claim 1, further comprising:

downsampling at least two pixels in the depth image before identifying the estimated location or position of the extremity of the user.

13. The method of claim 1, further comprising:

based at least on determining that a location or position of a second extremity has been estimated from the second depth image, adjusting the model to move the location or position of the second extremity corresponding to the estimated location or position of the second extremity.

14. A computer readable storage device for tracking a user, the computer readable storage medium having stored thereon computer executable instructions that, when executed on a computer, cause the computer to perform operations comprising:

receiving a depth image;

identifying an estimated location or position of an extremity of the user in the depth image;

adjusting a model based on the estimated location or position of the extremity; and

based at least on determining that a location or position of the extremity has not been estimated from a second depth image, adjusting the model to move the location or position of the extremity to a default position of the extremity.

15. The computer readable storage device of claim 14, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform operations comprising:

mapping the model to an on screen avatar.

16. The computer readable storage device of claim 14, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform operations comprising:

providing the model to a gesture library.

17. The computer readable storage device of claim 14, wherein the model comprises a skeletal model having joints and bones.

18. The computer readable storage device of claim 17, wherein adjusting the model comprises:

adjusting a joint of the skeletal model to the estimated location or position.

19. The computer readable storage device of claim 14, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform operations comprising:

determining whether the estimated location or position is valid.

20. A system for tracking a user comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, bearing processor-executable instructions that, when executed on the processor, cause the system to at least;

receive a depth image;

identify an estimated location or position an extremity of the user in the depth image;

adjust a model based on the estimated location or position of the extremity; and

based at least on determining that a location or position of the extremity has not been estimated from a second depth image, adjust the model to move the location or position of the extremity to a default position of the extremity.