Human body pose estimation

• US 20100278384A1
• Filed: 05/20/2009
• Published: 11/04/2010
• Est. Priority Date: 05/01/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Shotton, Jamie Daniel Joseph, Fitzgibbon, Andrew William
• Granted Patent: US 8,503,720 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Techniques for human body pose estimation are disclosed herein. Depth map images from a depth camera may be processed to calculate a probability that each pixel of the depth map is associated with one or more segments or body parts of a body. Body parts may then be constructed of the pixels and processed to define joints or nodes of those body parts. The nodes or joints may be provided to a system which may construct a model of the body from the various nodes or joints.

Claims

1. A method for determining a position of a body using a set of one or more segments, each segment being associated with a portion of the body, the method comprising:

identifying a set of one or more pixels in a depth map that includes at least part of the body; and

for each pixel in the set of pixels, assigning to the pixel a first probability that the pixel is within each segment in the set of segments.

2. The method of claim 1 wherein assigning to the pixel the first probability that the pixel is within each segment comprises:

determining at least one of depth or second probability at the pixel;

selecting at least one other pixel each at some angle and distance from the pixel;

determining at least one of depth or second probability at each of the at least one other pixels; and

assigning the first probability based at least in part on a relationship between the depth or second probability at the pixel and the depth or second probability at each of the at least one other pixels.

3. The method of claim 1 wherein assigning to the pixel the first probability that the pixel is within each segment comprises:

determining a background depth;

determining depth at the pixel; and

determining that the pixel is probably not within any of the segments when the depth at the pixel is similar to the background depth.

4. The method of claim 1, further comprising using the assigned first probabilities to determine a centroid pixel within each of the segments.

5. The method of claim 4 further comprising determining a location of one or more joints based on the centroid pixels.

6. The method of claim 2 wherein the distance and the angle of each other pixel from the pixel is determined based on a decision tree.

7. The method of claim 2 wherein the distance and the angle of the of the at least one other pixel from the first pixel is selected invariant with respect to depth.

8. The method of claim 6 wherein the first probability that the pixel is within each segment is determined using two or more decision trees, each tree being generated based on a different volume of body pose data.

9. A system for determining a position of a body using a set of one or more segments, each segment being associated with a portion of the body, the system comprising:

a depth camera for providing a depth map;

a processor that executes instructions comprising;

identifying a set of one or more pixels in the depth map that includes at least part of the body; and

for each pixel in the set of pixels, assigning to the pixel a first probability that the pixel is within each segment in the set of segments.

10. The system of claim 9 wherein the instructions for assigning to each pixel the first probability that each segment covers the pixel comprise:

determining at least one of a depth or a second probability at the pixel;

selecting at least one other pixel each at some angle and distance from the pixel;

determining at least one of the depth or the second probability at each of the at least one other pixels; and

assigning the first probability based at least in part on a relationship between the depth or second probability at the pixel and the depth or second probability at each of the at least one other pixels.

11. The system of claim 9 wherein the instructions for assigning to each pixel the second probability that each segment covers the pixel comprise:

determining a background depth;

determining depth at the pixel; and

determining that the pixel is probably not within any of the segments when the depth at the pixel is similar to the background depth.

12. The system of claim 9 wherein the instructions further comprise using the assigned second probabilities to determine a centroid pixel within each of the segments.

13. The system of claim 10 wherein the instructions further comprise determining the distance and the angle of each other pixel from the pixel using a decision tree.

14. A computer readable medium having recorded thereon computer executable instructions for determining a position of a body using a set of one or more segments, each segment being associated with a portion of the body, the instructions comprising:

identifying a set of one or more pixels in a depth map that includes at least part of the body; and

for each pixel in the set of pixels, assigning to the pixel a first probability that the pixel is within each segment in the set of segments.

15. The computer readable medium of claim 14 wherein assigning to the pixel the first probability that the pixel is within each segment comprises:

determining at least one of a depth or a second probability at the pixel;

selecting at least one other pixel each at some angle and distance from the pixel;

determining depth or second probability at each of the at least one other pixels; and

assigning the first probability based at least in part on a relationship between the depth or second probability at the pixel and the depth at each of the at least one other pixels.

16. The computer readable medium of claim 14 wherein assigning to the pixel the first probability that the pixel is within each segment comprises:

determining a background depth;

determining depth at the pixel; and

determining that the pixel is probably not within any of the segments when the depth at the pixel is similar to the background depth.

17. The computer readable medium of claim 14 wherein the instructions further comprise using the assigned first probabilities to determine a centroid pixel within each of the segments.

18. The computer readable medium of claim 15 wherein the distance and the angle of each other pixel from the pixel is determined based on a decision tree.

19. The computer readable medium of claim 18 wherein the decision tree is generated based on a volume of body pose data.

20. The computer readable medium of claim 19 wherein the first probability that the pixel is within each segment is determined using two or more decision trees, each tree being generated based on a different volume of body pose data.