Systems and methods for tracking a model

• US 8,325,984 B2
• Filed: 06/09/2011
• Issued: 12/04/2012
• Est. Priority Date: 10/07/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lee, Johnny Chung, Leyvand, Tommy, Stachniak, Simon Piotr, Peeper, Craig
• Primary Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US13/156,457
• Publication Number:
  US 20110234589A1
• Time in Patent Office: 544 Days
• Field of Search: 382/100, 382/103, 382/106, 382/107, 345/418, 345/419, 345/632, 345/619, 345/630, 345/629, 345/467, 345/468, 348/169, 348/170, 348/171, 348/172
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04N 13/20 Image signal generators

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A model may be adjusted based on a location or position of one or more extremities estimated or determined for a human target in the grid of voxels. The model may also be adjusted based on a default location or position of the model in a default pose such as a T-pose, a DaVinci pose, and/or a natural pose.

Claims

1. A method for tracking a user, comprising:

generating a grid of voxels based on a depth image, the depth image being captured by a depth camera;

identifying an estimated location or position of an extremity of the user in the grid of voxels;

adjusting a model based on the estimated location or position of the extremity; and

in response to determining that a location or position of the extremity has not been estimated from a second grid of voxels based on a second depth image, adjusting the model to move the location or position of the extremity to a default position of the extremity.

2. The method of claim 1, further comprising:

mapping the model to an on screen avatar.

3. The method of claim 1, further comprising:

providing the model to a gesture library.

4. The method of claim 1, wherein the model comprises a skeletal model having joints and bones.

5. The method of claim 4, wherein adjusting the model comprises:

adjusting a joint of the skeletal model to the estimated location or position.

6. The method of claim 1, further comprising:

determining that the estimated location or position is valid.

7. The method of claim 1, further comprising:

adjusting the model based on a geometric constraint.

8. The method of claim 1, wherein the default pose comprises at least one of the following:

a T-Pose, a Di Vinci pose, and a natural pose.

9. The method of claim 1, further comprising:

magnetizing the model to one or more of the voxels in the grid of voxels.

10. The method of claim 1, further comprising:

determining that a pose associated with the model is valid.

11. The method of claim 1, further comprising:

refining the model based on the received depth image.

12. The method of claim 1, further comprising:

downsampling the pixels in the depth image to generate the grid of voxels.

13. The method of claim 1, further comprising:

in response to determining that a location or position of a second extremity has been estimated from the second grid of voxels, adjusting the model to move the location or position of the second extremity corresponding to the estimated location or position of the second extremity.

14. A computer readable storage medium for tracking a user, the computer readable storage medium having stored thereon computer executable instructions that, when executed on a computer, cause the computer to perform operations comprising:

generating a grid of voxels based on a depth image;

identifying an estimated location or position of an extremity of the user in the grid of voxels; and

adjusting a model based on the estimated location or position of the extremity; and

in response to determining that a location or position of the extremity has not been estimated from a second grid of voxels based on a second depth image, adjusting the model to move the location or position of the extremity to a default position of the extremity.

15. The computer readable storage medium of claim 14, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform operations comprising:

mapping the model to an on screen avatar.

16. The computer readable storage medium of claim 14, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform operations comprising:

providing the model to a gesture library.

17. The computer readable storage medium of claim 14, wherein the model comprises a skeletal model having joints and bones.

18. The computer readable storage medium of claim 17, wherein adjusting the model comprises:

adjusting a joint of the skeletal model to the estimated location or position.

19. The computer readable storage medium of claim 14, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform operations comprising:

determining whether the estimated location or position is valid.

20. A system for tracking a user comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, bearing processor-executable instructions that, when executed on the processor, cause the system to at least;

generate a grid of voxels based on a depth image;

identify an estimated location or position an extremity of the user in the grid of voxels;

adjust a model based on the estimated location or position of the extremity; and

in response to determining that a location or position of the extremity has not been estimated from a second grid of voxels based on a second depth image, adjust the model to move the location or position of the extremity to a default position of the extremity.