Environment and/or target segmentation

• US 8,896,721 B2
• Filed: 01/11/2013
• Issued: 11/25/2014
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mathe, Zsolt, Marais, Charles Claudius
• Primary Examiner(s): NGUYEN, LUONG TRUNG
• Application Number:
  US13/739,708
• Publication Number:
  US 20130129227A1
• Time in Patent Office: 683 Days
• Field of Search: None
• US Class Current: 348/222.1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 10/26 Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
• CPC Class Codes: G06V 20/80 Recognising image objects characterised by unique random patterns
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

A depth image of a scene may be observed or captured by a capture device. The depth image may include a human target and an environment. One or more pixels of the depth image may be analyzed to determine whether the pixels in the depth image are associated with the environment of the depth image. The one or more pixels associated with the environment may then be discarded to isolate the human target and the depth image with the isolated human target may be processed.

Claims

1. A system, comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, in response to execution by the processor, cause the system to at least;

receive a depth image of a scene;

determine a centroid of a target in the scene;

define a bounding box for the target based on the centroid of the target and a measurement of the target; and

identify a pixel of the depth image that is outside of the bounding box as being associated with an environment of the scene.

2. The system of claim 1, wherein the memory further bears instructions that, in response to execution by the processor, cause the system to at least:

define the bounding box for the target from depth history data.

3. The system of claim 2, wherein the memory further bears instructions that, in response to execution by the processor, cause the system to at least:

track a maximum depth value for the pixel of the depth image; and

store the maximum depth value in historic depth value data.

4. The system of claim 3, wherein the memory further bears instructions that, in response to execution by the processor, cause the system to at least:

identify a pixel associated with a floor in the scene; and

store a depth value of the pixel associated with the floor in the historic depth value data.

5. The system of claim 1, wherein the instructions that, in response to execution by the processor, cause the system to at least define the bounding box for the target further cause the system to at least:

flood-fill a pixel within the bounding box for the target.

6. The system of claim 5, wherein the instructions that, in response to execution by the processor, cause the system to at least define the bounding box for the target further cause the system to at least:

define the bounding box for the target using a predetermined edge tolerance value.

7. The system of claim 6, wherein the instructions that, in response to execution by the processor, cause the system to at least define the bounding box for the target using the predetermined edge tolerance value further cause the system to at least:

adjust the predetermined edge tolerance value based on a measurement of the target, or a position of the pixel.

8. The system of claim 1, wherein the instructions that, in response to execution by the processor, cause the system to at least identify a pixel of the depth image that is outside of the bounding box as being associated with an environment of the scene further cause the system to at least:

flood-fill the pixel of the depth image that is outside of the bounding box.

9. The system of claim 1, wherein the memory further bears instructions that, in response to execution by the processor, cause the system to at least:

identify a pixel associated with an infrared shadow in the depth image; and

replace a depth value of the pixel associated with the infrared shadow with a calculated depth value.

10. The system of claim 1, wherein the memory further bears instructions that, in response to execution by the processor, cause the system to at least:

remove the pixel of the depth image that is outside of the bounding box in response to determining that the pixel is outside of the bounding box.

11. A method, comprising:

receiving a depth image of a scene;

determining a centroid of a target in the scene;

defining a bounding box for the target based on the centroid of the target and a measurement of the target; and

identifying a pixel of the depth image that is outside of the bounding box as being associated with an environment of the scene.

12. The method of claim 11, further comprising:

defining the bounding box for the target from depth history data.

13. The method of claim 11, further comprising:

tracking a maximum depth value for the pixel of the depth image; and

storing the maximum depth value in historic depth value data.

14. The method of claim 13, further comprising:

identifying a pixel associated with a floor in the scene; and

storing a depth value of the pixel associated with the floor in the historic depth value data.

15. A computer-readable storage device, bearing computer-readable instructions that, when executed on a computer, cause the computer to perform comprising:

receiving a depth image of a scene;

determining a centroid of a target in the scene;

defining a bounding box for the target based on the centroid of the target and a measurement of the target; and

identifying a pixel of the depth image that is outside of the bounding box as being associated with an environment of the scene.

16. The computer-readable storage device of claim 15, wherein defining the bounding box for the target comprises:

flood-filling a pixel within the bounding box for the target.

17. The computer-readable storage device of claim 16, wherein defining the bounding box for the target comprises:

defining the bounding box for the target using a predetermined edge tolerance value.

18. The computer-readable storage device of claim 17, wherein defining the bounding box for the target using the predetermined edge tolerance value comprises:

adjusting the predetermined edge tolerance value based on a measurement of the target, or a position of the first pixel.

19. The computer-readable storage device of claim 15, wherein identifying a pixel of the depth image that is outside of the bounding box as being associated with an environment of the scene comprises:

flood-filling the pixel of the depth image that is outside of the bounding box.

20. The computer-readable storage device of claim 15, further bearing computer-readable instructions that, when executed on the computer, cause the computer to perform comprising:

identify a pixel associated with an infrared shadow in the depth image; and

replacing a depth value of the pixel associated with the infrared shadow with a calculated depth value.