Body scan

• US 8,897,493 B2
• Filed: 01/04/2013
• Issued: 11/25/2014
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mathe, Zsolt, Marais, Charles Claudius, Geiss, Ryan Michael
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US13/694,781
• Publication Number:
  US 20130129169A1
• Time in Patent Office: 690 Days
• Field of Search: 382103-118, 382/154, 382/181, 382/190, 382284-294, 348/207.99, 348/211.1, 348/46, 348/77, 345156-158, 345173-175, 345/473
• US Class Current: 382/103
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A depth image of a scene may be received, observed, or captured by a device. The depth image may then be analyzed to determine whether the depth image includes a human target. For example, the depth image may include one or more targets including a human target and non-human targets. Each of the targets may be flood filled and compared to a pattern to determine whether the target may be a human target. If one or more of the targets in the depth image includes a human target, the human target may be scanned. A skeletal model of the human target may then be generated based on the scan.

Claims

1. A method for scanning a target in a captured scene:

receiving an image of the scene, the image including the target;

comparing the target with a pattern to determine that the target matches the pattern;

repositioning at least a first joint if a distance between the at least first joint and a reference point is greater than a value until a position of the at least first joint is within a tolerance defined for the at least first joint; and

generating a model of the target based at least on the repositioned at least first joint.

2. The method of claim 1, wherein the pattern includes a machine representation of a predetermined body model associated with a human.

3. The method of claim 1, wherein comparing the target with the pattern to determine that the target matches the pattern further comprises calculating a confidence value that indicates an accuracy to which the target corresponds to the pattern.

4. The method of claim 1, further comprising:

scanning the target to locate the at least first joint; and

identifying the reference point from the scan.

5. The method of claim 4, wherein scanning the target further comprises:

removing an area of the image not matching the target;

creating a bitmask of the target; and

scanning the bitmask to determine various body parts of the target.

6. The method of claim 1, further comprising providing the model to a computing system, wherein the computing system tracks one or more movements of the model, and wherein the computing system is controlled based on the tracked one or more movements.

7. A device for capturing depth information of a scene, the device comprising:

a processor, wherein the processor is operable to execute computer executable instructions, and wherein the computer executable instructions comprise instructions for;

determining that a depth image includes a target;

repositioning at least a first joint if a distance between the first joint and a reference point is greater than a value until a position of the at least first joint is within a tolerance defined for the at least first joint; and

generating a model of the target based at least on the repositioned at least first joint.

8. The device of claim 7, wherein the instructions for determining that the depth image includes the target comprise instructions for:

determining one or more edges associated with each target in the scene;

flood filling each target based on the determined one or more edges; and

comparing each flood filled target with a pattern to determine whether each flood filled target matches the pattern.

9. The device of claim 8, wherein the pattern includes a machine representation of a predetermined body model associated with a human.

10. The device of claim 8, wherein the instructions for comparing each flood filled target with a pattern to determine whether each flood filled target matches the pattern further comprise instructions for calculating a confidence value that indicates an accuracy to which each of the flood filled target corresponds to the pattern.

11. The device of claim 7, wherein the computer executable instructions further comprise instructions for:

scanning the target to locate the at least first joint; and

identifying the reference point from the scan.

12. The device of claim 11, wherein the instructions for scanning the target further comprise instructions for:

removing an area of the depth image not matching the target;

creating a bitmask of the target; and

scanning the bitmask to determine various body parts of the target.

13. The device of claim 7, wherein the instructions for repositioning the at least first joint comprise instructions for:

comparing a dY value with a dX value associated with the at least first joint, wherein the dY value and the dX value correspond to distances between the at least first joint and a respective reference point of the target;

repositioning the at least first joint in a first direction on a Y axis by the dX value if, based on the comparison, the dY value may be greater than the dX value to generate a first new shoulder joint; and

repositioning the at least first joint in a second direction on a X axis by the dY value if, based on the comparison, the dY value may be less the dX value to generate a second new shoulder joint.

14. The device of claim 7, further comprising instructions for providing the model to a computing system, wherein the computing system tracks one or more movements of the model, and wherein the computing system is controlled based on the tracked one or more movements.

15. A computer-readable storage device having stored thereon computer executable instructions for scanning a target in a captured scene, the computer executable instructions comprising instructions for:

receiving an image of the scene, the image including the target;

comparing the target with a pattern to determine that the target matches the pattern;

repositioning at least a first joint if a distance between the at least first joint and a reference point is greater than a value until a position of the at least first joint is within a tolerance defined for the at least first joint; and

generating a model of the target based at least on the repositioned at least first joint.

16. The computer-readable storage device of claim 15, wherein the pattern includes a machine representation of a predetermined body model associated with a human.

17. The computer-readable storage device of claim 15, wherein the instructions for comparing the target with the pattern to determine that the target matches the pattern further comprise instructions for calculating a confidence value that indicates an accuracy to which the target corresponds to the pattern.

18. The computer-readable storage device of claim 15, wherein the computer executable instructions further comprise instructions for:

scanning the target to locate the at least first joint; and

identifying the reference point from the scan.

19. The computer-readable storage device of claim 18, wherein the instructions for scanning the target further comprise instructions for:

removing an area of the image not matching the target;

creating a bitmask of the target; and

scanning the bitmask to determine various body parts of the target.

20. The computer-readable storage device of claim 15, wherein the computer executable instructions further comprise instructions for providing the model to a computing system, wherein the computing system tracks one or more movements of the model, and wherein the computing system is controlled based on the tracked one or more movements.