Systems And Methods For Tracking A Model

• US 20110081045A1
• Filed: 11/18/2009
• Published: 04/07/2011
• Est. Priority Date: 10/07/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Stachniak, Simon Piotr, Leyvand, Tommer, Lee, Johnny Chung, Peeper, Craig
• Granted Patent: US 7,961,910 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04N 13/20 Image signal generators

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A model may be adjusted based on a location or position of one or more extremities estimated or determined for a human target in the grid of voxels. The model may also be adjusted based on a default location or position of the model in a default pose such as a T-pose, a DaVinci pose, and/or a natural pose.

Claims

1. A method for tracking a user in a scene, the method comprising:

receiving a depth image;

generating a grid of voxels based on the depth image;

determining whether a location or position has been estimated for an extremity of a human target included the grid of voxels; and

adjusting a body part of a model associated with the extremity to the location or position when, based on the determination, the location or position has been estimated for the extremity.

2. The method of claim 1, wherein the model comprises a skeletal model having joints and bones.

3. The method of claim 2, wherein adjusting the body part of the model associated with the extremity to the location or position comprises adjusting a joint of the skeletal model associated with the extremity to the location or position.

4. The method of claim 1, further comprising determining whether the location or position has been estimated for the extremity of the human target included the grid of voxels further comprises determining whether the location or position is valid, and wherein the body part of the model associated with the extremity is adjusted to the location or position when, based on the determinations, the location or position has been estimated for the extremity and the location or position estimated for the extremity is valid.

5. The method of claim 1, further comprising adjusting the model based on a geometric constraint.

6. The method of claim 1, further comprising relaxing the body part of the model associated with the extremity to a default location or position associated with a default pose when, based on the determination, the location or position has not been determined for the estimated extremity.

7. The method of claim 6, wherein the default pose comprises at least one of the following:

a T-Pose, a Di Vinci pose, and a natural pose.

8. The method of claim 6, further comprising adjusting the body part of the model to a location or position of a voxel of the human target closest to the default location or position associated with the default pose.

9. The method of claim 1, further comprising determining whether a pose associated with the model and the one or more adjusted body parts is valid.

10. The method of claim 1, further comprising refining a position of the body part of the model based on one or more pixels associated with the extremity of the human target in the received depth image.

11. A computer-readable storage medium having stored thereon computer executable instructions for tracking a user in a scene, the computer executable instructions comprising instructions for:

receiving a depth image;

generating a grid of voxels based on the depth image;

determining whether a location or position has been estimated for an extremity of a human target included the grid of voxels;

relaxing a body part of a model associated with the extremity to a default location or position associated with a default pose when, based on the determination, the location or position has not been determined for the estimated extremity; and

magnetizing the body part of the model to a location or position of a voxel of the human target closest to the default location or position associated with the default pose.

12. The computer-readable storage medium of claim 11, wherein the default pose comprises at least one of the following:

a T-Pose, a Di Vinci pose, and a natural pose.

13. The computer-readable storage medium of claim 12, further comprising instructions for refining a position of the body part of the model based on one or more pixels associated with the extremity of the human target in the received depth image.

14. A computer-readable storage medium of claim 11, further comprising instructions for adjusting the model based on a geometric constraint.

15. The computer-readable storage medium of claim 14, wherein the geometric constraint comprises at least one of the following:

an angle of the body part for a typical human, a length of the body part for the typical human, a width of the body part for the typical human, an angle determined for the extremity, a length determined for the extremity, and a width determined for the extremity.

16. The computer-readable storage medium of claim 11, further comprising instructions for adjusting a body part of a model associated with the extremity to the location or position estimated for the extremity when, based on the determination, the location or position has been estimated for the extremity.

17. A system for tracking a user in a scene, the system comprising:

a capture device, wherein the capture device comprises a camera component that receives a depth image of a scene; and

a computing device in operative communication with the capture device, wherein the computing device comprises a processor that generates a grid of voxels based on the depth image;

determines whether a location or position has been estimated for an extremity of a human target included the grid of voxels;

adjusts a body part of a model associated with the extremity to the location or position when, based on the determination, the location or position has been estimated for the extremity; and

adjusts the body part of the model to a closest voxel associated with the human target when, based on the determination, the location or position has not been estimated for the extremity.

18. The system of claim 17, wherein the processor further adjusts the body part of the model to a closest voxel associated with the human target when, based on the determination, the location or position has not been estimated for the extremity by relaxing the body part of the model associated with the extremity to a default location or position associated with a default pose;

and adjusting the body part of the model to a location or position of a voxel of the human target closest to the default location or position associated with the default pose.

19. The system of claim 17, wherein the processor further refines a position of the body part of the model based on one or more pixels associated with the extremity of the human target in the received depth image.

20. The system of claim 17, wherein the processor further adjusts the model based on a geometric constraint.