Systems And Methods For Estimating An Occluded Body Part

• US 20100303302A1
• Filed: 06/26/2009
• Published: 12/02/2010
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Tsunoda, Kudo, Kipman, Alex A., Perez, Kathryn Stone, Geiss, Ryan Michael, Finocchio, Mark J.
• Granted Patent: US 9,182,814 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A depth image of a scene may be received, observed, or captured by a device. The depth image may include a human target that may have, for example, a portion thereof non-visible or occluded. For example, a user may be turned such that a body part may not be visible to the device, may have one or more body parts partially outside a field of view of the device, may have a body part or a portion of a body part behind another body part or object, or the like such that the human target associated with the user may also have a portion body part or a body part non-visible or occluded in the depth image. A position or location of the non-visible or occluded portion or body part of the human target associated with the user may then be estimated.

Claims

1. A device, the device comprising:

a camera component, wherein the camera component receives a depth image of a scene; and

a processor, wherein the processor executes computer executable instructions, and wherein the computer executable instructions comprise instructions for;

receiving the depth image from the camera component;

determining whether a portion of a human target in the depth image is non-visible; and

estimating the non-visible portion of the human target.

2. The device of claim 1, wherein the instructions for estimating the non-visible portion of the human target further comprises instructions for dragging the non-visible portion of the human target behind a visible portion of the human target, wherein the non-visible portion drifts behind the visible portion of the human target.

3. The device of claim 2, wherein the instructions for estimating the non-visible portion of the human target further comprises instructions for generating at least one of the following:

an estimated X value, an estimated Y value, and an estimated Z value for the non-visible portion.

4. The device of claim 3, wherein the estimated X value, the estimated Y value, and the estimated Z value are generated based on the visible portion of the human target.

5. The device of claim 4, wherein the estimated X value, the estimated Y values, and the estimated Z value of the non-visible portion is generated based on at least one of the following:

an X value of the visible portion, a Y value of the visible portion, a Z value of the visible portion, a force vector computed for the visible portion, momentum of the visible portion, a velocity of the visible portion, a rotation of the visible portion, and a trajectory of the visible portion.

6. The device of claim 3, wherein the estimated X value, the estimated Y value, and the estimated Z value are generated based on historical data associated with the non-visible portion, wherein the historical data comprises data associated with a previously received depth image where the non-visible portion is visible.

7. The device of claim 3, further comprising instructions for:

adjusting a model of the human target based on the estimated X value, the estimated Y value, and the estimated Z value; and

processing the model of the human target with the estimated X value, the estimated Y value, and the estimated Z value.

8. The device of claim 1, wherein the instructions for estimating a further comprise instructions for generating a confidence value associated with an accuracy of the estimated non-visible portion.

9. The device of claim 1, further comprising instructions for adjusting the estimated, non-visible portion when a pixel associated the non-visible portion becomes visible in a subsequent depth image.

10. A computer-readable storage medium having stored thereon computer executable instructions, the computer executable instructions comprising instructions for:

receiving a depth image, wherein the depth image includes a human target associated with a user;

determining whether a body part of the human target in the depth image is occluded; and

estimating a location or a position of the occluded body part.

11. The computer-readable storage medium of claim 10, wherein the instructions for estimating the location or the position of the occluded body part further comprises instructions for dragging the occluded body part behind a visible body part of the human target, wherein the occluded body part drifts behind the visible body part of the human target.

12. The computer-readable storage medium of claim 11, wherein the instructions for estimating the location or the position of the occluded body part further comprises instructions for generating at least one of the following:

an estimated X value, an estimated Y value, and an estimated Z value for the occluded body part.

13. The computer-readable storage medium of claim 12, wherein the estimated X value, the estimated Y value, and the estimated Z value are generated based on the visible body part of the human target.

14. The computer-readable storage medium of claim 13, wherein the estimated X value, the estimated Y values, and the estimated Z value of the occluded body part is generated based on at least one of the following:

an X value of the visible body part, a Y value of the visible body part, a Z value of the visible body part, a force vector computed for the visible body part, momentum of the visible body part, a velocity of the visible body part, a rotation of the visible body part, and a trajectory of the visible body part.

15. The computer-readable storage medium of claim 12, wherein the estimated X value, the estimated Y value, and the estimated Z value are generated based on historical data associated with the non-visible portion, wherein the historical data comprises data associated with a previously received depth image where the non-visible portion is visible.

16. The computer-readable storage medium of claim 10, further comprising instructions for adjusting the estimated, occluded body part when a pixel associated the occluded body part becomes visible in a subsequent depth image.

17. A system, the system comprising:

a capture device, wherein the capture device comprises a camera component that receives a depth image of a scene, wherein the depth image comprises a human target associated with a user; and

a computing device in operative communication with the capture device, wherein the computing device comprises a processor that receives the depth image from the capture device, determines whether a body part of the human target is visible; and

generates an estimated X value, an estimated Y value, and an estimated Z value for the non-visible body part.

19. The system of claim 17, wherein the estimated X value, the estimated Y value, and the estimated Z value are generated based on the visible body part of the human target.

20. The system of claim 19, wherein the estimated X value, the estimated Y values, and the estimated Z value of the non-visible body part is generated based on at least one of the following:

an X value of the visible body part, a Y value of the visible body part, a Z value of the visible body part, a force vector computed for the visible body part, momentum of the visible body part, a velocity of the visible body part, a rotation of the visible body part, and a trajectory of the visible body part.

18. The system of 17, wherein processor further drags the non-visible body part behind a visible body part of the human target, wherein the non-visible body part drifts behind the visible body part of the human target.