Human tracking system

• US 9,821,226 B2
• Filed: 01/22/2015
• Issued: 11/21/2017
• Est. Priority Date: 10/07/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Leyvand, Tommer, Lee, Johnny, Peeper, Craig, Stachniak, Szymon, Liu, Shao
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US14/602,333
• Publication Number:
  US 20150131862A1
• Time in Patent Office: 1,034 Days
• Field of Search: None
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 2300/1075 using a touch screen
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/34 Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
• CPC Class Codes: G06V 10/426 Graphical representations
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A background included in the grid of voxels may also be removed to isolate one or more voxels associated with a foreground object such as a human target. A location or position of one or more extremities of the isolated human target may be determined and a model may be adjusted based on the location or position of the one or more extremities.

Claims

1. A method for estimating a dimension of a joint of a user in a scene, comprising:

receiving, by a computing device comprising a processor and memory, a depth image, the depth image being indicative of the user in the scene;

determining, by the computing device, a plurality of proposed dimensions for the joint of the user in the depth image;

determining, by the computing device, the dimension of the joint based on the plurality of proposed dimensions;

updating, by the computing device, a computer-maintained model that corresponds to the user to have the dimension of the joint, wherein the dimension of the joint comprises a length and a width; and

generating, by the computing device, one or more motions or movements by mapping the updated computer-maintained model.

2. The method of claim 1, further comprising calculating a mean and a standard deviation for each of the proposed dimensions.

3. The method of claim 1, further comprising rejecting one or more of the plurality of proposed dimensions that are outside a defined percentage deviation.

4. The method of claim 1, further comprising determining the dimension of the joint based on an estimator that has a highest ratio between a standard deviation thereof and a number of the plurality of proposed dimensions.

5. The method of claim 1, wherein the dimension of the joint comprises at least one of length or width.

6. The method of claim 1, further comprising determining whether the dimension of the joint is correct based on one or more heuristics.

7. A system for estimating a dimension of a joint of a user in a scene, comprising:

one or more memories having stored therein computer-executable instructions; and

one or more processors operable to execute the computer-executable instructions, wherein the computer-executable instructions comprise instructions for performing operations comprising;

receiving a depth image, the depth image being indicative of the user in the scene;

determining a plurality of proposed dimensions for the joint of the user in the depth image;

determining the dimension of the joint based on the plurality of proposed dimensions; and

updating a computer-maintained model that corresponds to the user to have the dimension of the joint, wherein the dimension of the joint comprises a length and a width; and

generating data indicative of one or more motions or movements by mapping the updated computer-maintained model.

8. The system of claim 7, wherein the operations further comprise rejecting one or more of the plurality of proposed dimensions that are outside a defined percentage deviation.

9. The system of claim 7, wherein the operations further comprise determining the dimension of the joint based on an estimator that has a highest ratio between a standard deviation thereof and a number of the plurality of proposed dimensions.

10. The system of claim 7, wherein the dimension of the joint comprises at least one of length or width.

11. The system of claim 7, wherein the operations further comprise determining whether the dimension of the joint is correct based on one or more heuristics.

12. The system of claim 7, wherein the operations further comprise calculating a mean for each of the proposed dimensions.

13. The system of claim 7, wherein the operations further comprise calculating a standard deviation for each of the proposed dimensions.

14. A computer-readable storage device not consisting of only transitory signals, the storage device having stored thereon computer-executable instructions for estimating a dimension of a joint of a user in a scene, wherein the computer-executable instructions comprise instructions for performing operations comprising:

receiving a depth image, the depth image being indicative of the user in the scene;

determining a plurality of proposed dimensions for the joint of the user in the depth image;

determining the dimension of the joint based on the plurality of proposed dimensions; and

updating a computer-maintained model that corresponds to the user to have the dimension of the joint, wherein the dimension of the joint comprises a length and a width; and

generating data indicative of one or more motions or movements by mapping the updated computer-maintained model.

15. The computer-readable storage device of claim 14, wherein the operations further comprise rejecting one or more of the plurality of proposed dimensions that are outside a defined percentage deviation.

16. The computer-readable storage device of claim 14, wherein the operations further comprise determining the dimension of the joint based on an estimator that has a highest ratio between a standard deviation thereof and a number of the plurality of proposed dimensions.

17. The computer-readable storage device of claim 14, wherein the dimension of the joint comprises at least one of length or width.

18. The computer-readable storage device of claim 14, wherein the operations further comprise determining whether the dimension of the joint is correct based on one or more heuristics.

19. The computer-readable storage device of claim 14, wherein the operations further comprise calculating a mean for each of the proposed dimensions.

20. The computer-readable storage device of claim 14, wherein the operations further comprise calculating a standard deviation for each of the proposed dimensions.