Human body pose estimation

• US 8,503,720 B2
• Filed: 05/20/2009
• Issued: 08/06/2013
• Est. Priority Date: 05/01/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Shotton, Jamie Daniel Joseph, Fitzgibbon, Andrew William
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US12/454,628
• Publication Number:
  US 20100278384A1
• Time in Patent Office: 1,539 Days
• Field of Search: 382/103, 382/203
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Techniques for human body pose estimation are disclosed herein. Depth map images from a depth camera may be processed to calculate a probability that each pixel of the depth map is associated with one or more segments or body parts of a body. Body parts may then be constructed of the pixels and processed to define joints or nodes of those body parts. The nodes or joints may be provided to a system which may construct a model of the body from the various nodes or joints.

Claims

1. A method for determining a position of a body using a set of one or more segments, each segment being associated with a portion of the body, the method comprising:

identifying a set of one or more pixels in a depth map that includes at least part of the body, the depth map being captured with a depth camera;

for each pixel in the set of pixels, assigning to the pixel a first probability that the pixel is within each segment in the set of segments; and

using the assigned first probabilities to determine a centroid pixel within each of the segments.

2. The method of claim 1, wherein assigning to the pixel the first probability that the pixel is within each segment comprises:

determining at least one of depth or second probability at the pixel;

selecting at least one other pixel each at some angle and distance from the pixel;

determining at least one of depth or second probability at each of the at least one other pixels; and

assigning the first probability based at least in part on a relationship between the depth or second probability at the pixel and the depth or second probability at each of the at least one other pixels.

3. The method of claim 1, wherein assigning to the pixel the first probability that the pixel is within each segment comprises:

determining a background depth;

determining depth at the pixel; and

determining that the pixel is probably not within any of the segments when the depth at the pixel is similar to the background depth.

4. The method of claim 1, further comprising:

determining a location of one or more joints based on the centroid pixels.

5. The method of claim 2, wherein the distance and the angle of each other pixel from the pixel is determined based on a decision tree.

6. The method of claim 2, wherein the distance and the angle of the of the at least one other pixel from the first pixel is selected invariant with respect to depth.

7. The method of claim 5, wherein the first probability that the pixel is within each segment is determined using two or more decision trees, each tree being generated based on a different volume of body pose data.

8. A system for determining a position of a body using a set of one or more segments, each segment being associated with a portion of the body, the system comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, when executed on the processor, cause the system to at least;

identify a set of one or more pixels in the depth map that includes at least part of the body, the depth map being captured with a depth camera;

for each pixel in the set of pixels, assign to the pixel a first probability that the pixel is within each segment in the set of segments; and

use the assigned first probabilities to determine a centroid pixel within each of the segments.

9. The system of claim 8, wherein the instructions that, when executed on the processor, cause the system to at least assign to each pixel the first probability that each segment covers the pixel further cause the system to at least:

determine at least one of a depth or a second probability at the pixel;

select at least one other pixel each at some angle and distance from the pixel;

determine at least one of the depth or the second probability at each of the at least one other pixels; and

assign the first probability based at least in part on a relationship between the depth or second probability at the pixel and the depth or second probability at each of the at least one other pixels.

10. The system of claim 8, wherein the instructions that, when executed on the processor, cause the system to at least assign to each pixel the second probability that each segment covers the pixel further cause the system to at least:

determine a background depth;

determine depth at the pixel; and

determine that the pixel is probably not within any of the segments when the depth at the pixel is similar to the background depth.

11. The system of claim 9, wherein the memory further bears instructions that, when executed on the processor, cause the system to at least:

determine the distance and the angle of each other pixel from the pixel using a decision tree.

12. The system of claim 8, wherein the memory further bears instructions that, when executed on the processor, cause the system to at least:

determine a location of one or more joints based on the centroid pixels.

13. The system of claim 9, wherein the distance and the angle of the of the at least one other pixel from the first pixel is selected invariant with respect to depth.

14. A computer readable storage device having recorded thereon computer executable instructions for determining a position of a body using a set of one or more segments, each segment being associated with a portion of the body, the instructions comprising:

identifying a set of one or more pixels in a depth map that includes at least part of the body;

for each pixel in the set of pixels, assigning to the pixel a first probability that the pixel is within each segment in the set of segments; and

using the assigned first probabilities to determine a centroid pixel within each of the segments.

15. The computer readable storage device of claim 14, wherein assigning to the pixel the first probability that the pixel is within each segment comprises:

determining at least one of a depth or a second probability at the pixel;

selecting at least one other pixel each at some angle and distance from the pixel;

determining depth or second probability at each of the at least one other pixels; and

assigning the first probability based at least in part on a relationship between the depth or second probability at the pixel and the depth at each of the at least one other pixels.

16. The computer readable storage device of claim 14, wherein assigning to the pixel the first probability that the pixel is within each segment comprises:

determining a background depth;

determining depth at the pixel; and

determining that the pixel is probably not within any of the segments when the depth at the pixel is similar to the background depth.

17. The computer readable storage device of claim 15, wherein the distance and the angle of each other pixel from the pixel is determined based on a decision tree.

18. The computer readable storage device of claim 17, wherein the decision tree is generated based on a volume of body pose data.

19. The computer readable storage device of claim 18, wherein the first probability that the pixel is within each segment is determined using two or more decision trees, each tree being generated based on a different volume of body pose data.