Body scan

• US 9,607,213 B2
• Filed: 03/16/2015
• Issued: 03/28/2017
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Mathe, Zsolt, Marais, Charles Claudius, Geiss, Ryan Michael
• Primary Examiner(s): Bayard, Djenane
• Application Number:
  US14/658,503
• Publication Number:
  US 20150262001A1
• Time in Patent Office: 743 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A depth image of a scene may be received, observed, or captured by a device. The depth image may then be analyzed to determine whether the depth image includes a human target. For example, the depth image may include one or more targets including a human target and non-human targets. Each of the targets may be flood filled and compared to a pattern to determine whether the target may be a human target. If one or more of the targets in the depth image includes a human target, the human target may be scanned. A skeletal model of the human target may then be generated based on the scan.

Claims

1. A method performed using a system, comprising:

receiving an image, wherein the image includes an image of one or more body parts corresponding to a human target;

comparing the image of the one or more body parts with a pattern of at least part of a human to determine that the one or more body parts corresponds to the pattern, the pattern being separate from the image;

generating a model of the one or more body parts based at least on determining that the image of the one or more body parts corresponds to the pattern;

tracking one or more changes of the model; and

controlling a program executing on the system using at least the tracked one or more changes to the model.

2. A method as in claim 1, wherein the system includes a game console.

3. A method as in claim 1, wherein the one or more body parts in the image are flood filled.

4. A method as in claim 1, further comprising using the image of the one or more body parts to render an on screen avatar associated with the model.

5. A method as in claim 1, wherein the human target is associated with an object and the model corresponds to the one or more body parts and the object.

6. A method as in claim 1, wherein the system stores gesture filters, further comprising comparing the one or more changes of the model to the gesture filters and using at least the results of the comparison to control the program.

7. A method as in claim 1, further comprising downsampling the received image to a lower resolution after receiving the image.

8. A method as in claim 1, wherein tracking one or more changes of the model includes tracking motion of the one or more body parts and adjusting the model based at least on the tracked motion of the one or more body parts.

9. A method as in claim 8, further comprising applying one or more constraints to movements of the body model.

10. A system, comprising:

a processor operable to execute instructions;

a memory to store the instructions, the instructions comprising;

instructions that, in response to execution by the processor, result in an image including an image of one or more body parts corresponding to a human target being compared to a pattern of at least part of a human to determine that the image of the one or more body parts corresponds to the pattern, wherein the pattern is separate from the image;

instructions that, in response to execution by the processor, generate a model of the one or more body parts based at least on a determination that the image of the one or more body parts corresponds to the pattern;

instructions that, in response to execution by the processor, track one or more changes to the model; and

instructions that, in response to execution by the processor, control a program that is to be executed by the system through the use of at least the tracked one or more changes to the model.

11. A system as in claim 10, further comprising a game console.

12. A system as in claim 10, wherein the one or more body parts in the image are flood filled.

13. A system as in claim 10, wherein the instructions further comprise instructions that control the processor to use the image of the one or more body parts to render an on screen avatar associated with the model.

14. A system as in claim 10, wherein the human target is associated with an object and the model corresponds to the one or more body parts and the object.

15. A system as in claim 10, further comprising gesture filters and instructions that control the processor to compare the one or more changes of the model to the gesture filters and to use at least the results of the comparison to control the program.

16. A system as in claim 10, further comprising instructions that control the processor to downsample the received image to a lower resolution after the image has been received.

17. A system as in claim 10, wherein the instructions that track one or more changes of the model include instructions that track motion of the one or more body parts and adjust the model based at least on the tracked motion of the one or more body parts.

18. A system as in claim 17, further comprising instructions that apply one or more constraints to movements of the body model.

19. A computer-readable storage device having stored thereon computer executable instructions for scanning a human target in a captured scene, said instructions comprising:

instructions to receive an image including an image of one or more body parts corresponding to the human target;

instructions to compare the image of the one or more body parts with a pattern of at least part of a human to determine that the image of the one or more body parts corresponds to the pattern, the pattern being separate from the image;

instructions to generate a model of the one or more body parts based at least on determining that the image of the one or more body parts corresponds to the pattern;

instructions to track one or more changes of the model; and

instructions to control a program executing on the system using at least the tracked one or more changes to the model.

20. A device as in claim 19, wherein the one or more body parts in the image are flood filled.

21. A device as in claim 19, further comprising instructions to use the one or more body parts to render an on screen avatar associated with the model.

22. A device as in claim 19, wherein the human target is associated with an object and the model corresponds to the one or more body parts and the object.

23. A device as in claim 19, further comprising instructions to compare the one or more changes of the model to gesture filters and to use at least the results of the comparison to control the program.

24. A device as in claim 19, further comprising instructions to downsample the received image to a lower resolution after the image has been received.

25. A device as in claim 19, wherein the instructions to track one or more changes of the model include instructions that track motion of the one or more body parts and adjust the model based at least on the tracked motion of the one or more body parts.

26. A device as in claim 25, further comprising instructions that apply one or more constraints to movements of the body model.