Systems And Methods For Removing A Background Of An Image

• US 20110081044A1
• Filed: 10/07/2009
• Published: 04/07/2011
• Est. Priority Date: 10/07/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Stachniak, Simon, Lee, Johnny, Peeper, Craig, Leyvand, Tommer
• Granted Patent: US 8,867,820 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A background included in the grid of voxels may then be discarded to isolate one or more voxels associated with a foreground object such as a human target and the isolated voxels associated with the foreground object may be processed.

Claims

1. A method for processing depth information of a scene, the method comprising:

receiving a depth image;

generating a grid of voxels based on the depth image;

determining whether one or more voxels in the grid are associated with a background;

discarding the one or more voxels associated with the background to isolate voxels associated with a foreground object in the depth image; and

processing the grid with the isolated foreground object.

2. The method of claim 1, further comprising analyzing each of the voxels in the grid of voxels to determine one or more objects in the depth image.

3. The method of claim 2, wherein analyzing each of the voxels in the grid of voxels to determine one or more objects in the depth image comprises:

comparing a value associated with a particular voxel in the grid with corresponding values of voxels adjacent to the particular voxel in the grid; and

identifying the particular voxel and one or more of the adjacent voxels as being part of an object if, based on the comparison, a difference between the value and the corresponding values is less than a threshold.

4. The method of claim 2, further comprising determining whether to merge an object in the one or more objects with a nearby object in the one or more objects.

5. The method of claim 4, wherein determining whether to merge the object in the one or more objects with the nearby object in the one or more objects comprises:

comparing X-values, Y-values, or depth values associated with voxels in the object with X-values, Y-values, or depth values associated with voxels of the nearby object; and

merging the object with the nearby object if, based on the comparison, the X-values, Y-values, or the depth values associated with the voxels in the object overlap the X-values Y-values, or depth values associated with the voxels of the nearby object.

6. The method of claim 2, wherein determining whether the one or more voxels in the grid are associated with the background comprises determining whether an object of the one or more objects in the grid is moving.

7. The method of claim 6, wherein the one or more objects in the grid that are moving are not associated with the background.

8. The method of claim 6, wherein determining whether the object in the grid is moving comprises:

comparing a value associated with each voxel in the objects with a corresponding value of a reference voxel included in a reference plate;

identifying each voxel in the object as moving when a difference between the value and the corresponding value exceeds a motion threshold.

9. The method of claim 8, wherein determining whether the object in the grid is moving further comprises:

calculating a foreground score based on a percentage of moving voxels associated with the object; and

identifying the object as moving when the foreground score exceeds a motion threshold.

10. The method of claim 1, wherein determining whether the one or more voxels in the grid are associated with the background comprises comparing a minimum depth value of the voxels in the grid with a reference plate comprising a depth history of the voxels in the grid.

11. A computer-readable storage medium having stored thereon executable instructions for processing depth information of a scene, the computer executable instructions comprising instructions for:

receiving a depth image of the scene, wherein the depth image comprises one or more objects;

determining whether the one or more objects in the depth image are moving; and

isolating the one or more objects that are moving; and

processing the one or more isolated objects.

12. The computer-readable storage medium of claim 11, further comprising determining whether to merge an object in the one or more objects with a nearby object in the one or more objects.

13. The computer-readable storage medium of claim 12, wherein determining whether to merge the object in the one or more objects with the nearby object in the one or more objects comprises:

comparing X-values, Y-values, or depth values associated with voxels in the object with X-values, Y-values, or depth values associated with voxels of the nearby object; and

merging the object with the nearby object if, based on the comparison, the X-values, Y-values, or the depth values associated with the voxels in the object overlap the X-values, Y-values, or depth values associated with the voxels of the nearby object.

14. The computer-readable storage medium of claim 11, wherein determining whether the object in the grid is moving comprises:

comparing a value associated with each voxel in the objects with a corresponding value of a reference voxel included in a reference plate; and

identifying each voxel in the object as moving when a difference between the value and the corresponding value exceeds a motion threshold.

15. The computer-readable storage medium of claim 14, wherein determining whether the object in the grid is moving further comprises:

calculating a foreground score based on a percentage of moving voxels associated with the object; and

identifying the object as moving when the foreground screen exceeds a motion threshold.

16. A system for processing depth information of a scene, the system comprising:

a capture device, wherein the capture device comprises a camera component that receives a depth image of a scene; and

a computing device in operative communication with the capture device, wherein the computing device comprises a processor that generates a downsampled depth image based on one or more pixels in the depth image received from the capture device;

determines whether objects in the downsampled depth image are moving;

discards the non-moving objects; and

processes the downsampled depth image with the isolated moving objects.

17. The system of claim 16, wherein the processor further determines whether to merge one or more of the objects.

18. The system of claim 17, wherein the processor determines whether to merge one or more of the objects by:

comparing X-values or depth values associated with voxels in one of the object with X-values or depth values associated with voxels of a nearby object; and

merging the object with the nearby object if, based on the comparison, the X-values or the depth values associated with the voxels in the object overlap the X-values or depth values associated with the voxels of the nearby object.

19. The system of claim 16, wherein the processor determines whether the objects are moving by:

comparing a value associated with each voxel in the objects with a corresponding value of a reference voxel included in a reference plate; and

identifying each voxel in the objects as moving when a difference between the value and the corresponding value exceeds a motion threshold.

20. The system of claim 19, where the processor further determines whether the objects are moving by:

calculating a foreground score based on a percentage of moving voxels associated with the objects; and

identifying the objects as moving when the foreground screen exceeds a motion threshold.

21. The method of claim 20, wherein the processor discards the non-moving objects by removing the objects that are not identified as moving.