BODY SCAN

• US 20110032336A1
• Filed: 10/28/2010
• Published: 02/10/2011
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mathe, Zsolt, Geiss, Ryan Michael, Marais, Charles Claudius
• Granted Patent: US 8,467,574 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A depth image of a scene may be received, observed, or captured by a device. The depth image may then be analyzed to determine whether the depth image includes a human target. For example, the depth image may include one or more targets including a human target and non-human targets. Each of the targets may be flood filled and compared to a pattern to determine whether the target may be a human target. If one or more of the targets in the depth image includes a human target, the human target may be scanned. A skeletal model of the human target may then be generated based on the scan.

Claims

1. A device for capturing depth information of a scene, the device comprising:

a camera component, wherein the camera component receives a depth image of the scene; and

a processor, wherein the processor executes computer executable instructions, and wherein the computer executable instructions comprise instructions for;

receiving the depth image of the scene from the camera component;

determining whether the depth image includes a human target;

scanning the human target to locate at least a first joint, if the depth image includes a human target;

identifying a reference point on the human target from the scan;

repositioning the at least first joint if a distance between the first joint and the reference point on the human target is greater than a value until a position of the at least first joint is within a tolerance defined for the at least first joint; and

generating a model of the human target based at least on the adjusted location of the at least first joint.

2. The device of claim 1, wherein the instructions for determining whether the depth image includes the human target comprises instructions for:

determining one or more edges associated with each target in the scene;

flood filling each target based on the determined one or more edges; and

comparing each flood filled target with a pattern to determine whether each flood filled target matches the pattern.

3. The device of claim 2, wherein the pattern includes a machine representation of a predetermined body model associated with a human.

4. The device of claim 2, wherein the instructions for comparing each flood filled target with a pattern to determine whether each flood filled target matches the pattern further comprises instructions for calculating a confidence value that indicates an accuracy to which each of the flood filled targets in the depth image corresponds to the pattern.

5. The device of claim 4, wherein the confidence value of each target is used to determine whether the depth image includes a human target.

6. The device of claim 1, wherein the instructions for scanning the human target further comprise instructions for:

removing an area of the depth image not matching the human target;

creating a bitmask of the human target; and

scanning the bitmask to determine various body parts of the human target.

7. The device of claim 1, further comprising instructions for adjusting a joint in the model of the human target.

8. The device of claim 1, wherein the instructions for adjusting the location of the at least first joint comprise instructions for:

comparing a dY value with a dX value associated with the joint, wherein the dY value and the dX value correspond to distances between the joint and a respective reference point of the human target;

repositioning the joint in a first direction on a Y axis by the dX value if, based on the comparison, the dY value may be greater than the dX value to generate a first new shoulder joint; and

repositioning the joint in a second direction on a X axis by the dY value if, based on the comparison, the dY value may be less the dX value to generate a second new shoulder joint.

9. The device of claim 1, further comprising instructions for providing the model to a computing system, wherein the computing system tracks one or more movements of the model, and wherein the computing system is controlled based on the tracked one or more movements.

10. The device of claim 1, wherein the model comprises a skeletal model.

11. A computer-readable storage medium having stored thereon computer executable instructions for scanning a human in a captured scene, the computer executable instructions comprising instructions for:

receiving the depth image of the scene, wherein the depth image includes a target;

comparing the target with a pattern to determine whether the target matches the pattern;

scanning the human target to locate at least a first joint if the target matches the pattern;

identifying a reference point on the human target from the scan;

repositioning the at least first joint if a distance between the first joint and the reference point on the human target is greater than a value until a position of the at least first joint is within a tolerance defined for the at least first joint; and

generating a model of the human target based at least on the adjusted location of the at least first joint.

12. The computer-readable medium of claim 11, wherein the pattern includes a machine representation of a predetermined body model associated with a human.

13. The computer-readable medium of claim 11, wherein the instructions for comparing the target with the pattern to determine whether each target matches the pattern further comprise instructions for

calculating a confidence value that indicates the accuracy to which the target in the depth image corresponds to the pattern.

14. The computer-readable medium of claim 13, wherein the confidence value of target is used to determine whether the depth image includes a human target.

15. The computer-readable medium of claim 11, wherein the instructions for scanning the human target further comprise instructions for:

removing an area of the depth image not matching the human target;

creating a bitmask of the human target; and

scanning the bitmask to determine various body parts of the human target.

16. The computer-readable medium of claim 11, further comprising instructions for providing the model to a computing system, wherein the computing system tracks one or more movements of the model, and wherein the computing system is controlled based on the tracked one or more movements.

17. The computer-readable medium of claim 11, wherein the model comprises a skeletal model.

18. A system for rendering a model of a user;

the system comprising;

a capture device, wherein the capture device comprises a camera component that receives a depth image of a scene;

a first processor that determines a human target associated with the user in the depth image,

scans the human target to locate at least a first joint,

identifying a reference point on the human target from the scan;

repositions the at least first joint if a distance between the first joint and the reference point on the human target is greater than a value until a position of the at least first joint is within a tolerance defined for the at least first joint; and

generates the model of the human target based at least on the adjusted location of the at least first joint; and

a computing device in operative communication with the capture device, wherein the computing device comprises a second processor that receives the generated model of the human target and tracks one or more movements of the model.

19. The system of claim 18, wherein the computing device further comprises a gestures library stored thereon, and wherein the second processor compares the one or more tracked movements with the gestures library to determine whether to perform a control to perform based on the one or more movements.

20. The system of claim 18, wherein the first processor determines the human target associated with the user in the depth image by flood filling each target in the scene and comparing each flood filled target with a pattern of a body model of a human to determine whether each flood filled target matches the pattern of the body model of the human.

21. The system of claim 20, wherein a confidence value is calculated, based on the comparison, to determine whether each target is the human target, and wherein the confidence value provides an indication of an accuracy to which each of the flood filled targets in the depth image corresponds to the pattern of the body model of the human.

22. The system of claim 18, wherein the first processor scans the human target by

removing an area of the depth image not matching the human target,

creating a bitmask of the human target, and

scanning the bitmask to determine various body parts of the human target,

wherein the confidence value provides an indication of an accuracy to which each of the flood filled targets in the depth image corresponds to the pattern of the body model of the human.

23. The system of claim 18, wherein the model comprises a skeletal model.