SYSTEMS AND METHODS FOR REMOVING A BACKGROUND OF AN IMAGE

• US 20140112547A1
• Filed: 12/30/2013
• Published: 04/24/2014
• Est. Priority Date: 10/07/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Peeper, Craig, Lee, Johnny, Leyvand, Tommer, Stachniak, Szymon
• Granted Patent: US 9,679,390 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images

Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A background included in the grid of voxels may then be discarded to isolate one or more voxels associated with a foreground object such as a human target and the isolated voxels associated with the foreground object may be processed.

Claims

1. A computer-readable storage medium having stored thereon executable instructions for processing depth information of a scene, the computer executable instructions comprising instructions for:

receiving a depth image of the scene, wherein the depth image comprises one or more objects;

determining that at least one object of the one or more objects in the depth image is moving;

isolating the at least one object that is moving; and

processing the at least one isolated object.

2. The computer-readable storage medium of claim 1, the computer executable instructions further comprising instructions for determining whether to merge an object in the one or more objects with a nearby object in the one or more objects.

3. The computer-readable storage medium of claim 2, wherein determining whether to merge the object in the one or more objects with the nearby object in the one or more objects comprises:

comparing X-values, Y-values, or depth values associated with voxels in the object with X-values, Y-values, or depth values associated with voxels of the nearby object; and

merging the object with the nearby object if, based on the comparison, the X-values, Y-values, or the depth values associated with the voxels in the object overlap the X-values, Y-values, or depth values associated with the voxels of the nearby object.

4. The computer-readable storage medium of claim 1, wherein determining that at least one object of the one or more objects in the depth image is moving comprises:

comparing a value associated with each voxel in the one or more objects with a corresponding value of a reference voxel included in a reference plate; and

identifying each voxel in the one or more objects as moving when a difference between the value and the corresponding value exceeds a motion threshold.

5. The computer-readable storage medium of claim 4, wherein determining that at least one object of the one or more objects in the depth image is moving further comprises:

calculating a foreground score based on a percentage of moving voxels associated with the at least one object; and

identifying the at least one object as moving when the foreground score exceeds a motion threshold.

6. The computer-readable storage medium of claim 1, the computer executable instructions further comprising instructions for determining that at least one object of the one or more objects is non-moving.

7. The computer-readable storage medium of claim 6, the computer executable instructions further comprising instructions for discarding the at least one non-moving object of the one or more objects.

8. A system for processing depth information of a scene, the system comprising:

a capture device, wherein the capture device comprises a camera component that receives a depth image of the scene; and

a computing device in operative communication with the capture device, wherein the computing device comprises a processor that generates a downsampled depth image comprising one or more objects based on one or more pixels in the depth image received from the capture device, determines that at least one object of the one or more objects in the downsampled depth image is moving, isolates the at least one object that is moving and processes the at least one isolated object.

9. The system of claim 8, wherein the processor further determines whether to merge an object in the one or more objects with a nearby object in the one or more objects.

10. The system of claim 9, wherein the processor determines whether merge an object in the one or more objects with a nearby object in the one or more objects:

comparing X-values or depth values associated with voxels in one of the object with X-values or depth values associated with voxels of a nearby object; and

merging the object with the nearby object if, based on the comparison, the X-values or the depth values associated with the voxels in the object overlap the X-values or depth values associated with the voxels of the nearby object.

11. The system of claim 8, wherein the processor determines that at least one object of the one or more objects in the downsampled depth image is moving by:

comparing a value associated with each voxel in the one or more objects with a corresponding value of a reference voxel included in a reference plate; and

identifying each voxel in the one or more objects as moving when a difference between the value and the corresponding value exceeds a motion threshold.

12. The system of claim 11, wherein the processor further determines that at least one object of the one or more objects in the downsampled depth image is moving by:

calculating a foreground score based on a percentage of moving voxels associated with the at least one object; and

identifying the at least one object as moving when the foreground score exceeds a motion threshold.

13. The system of claim 8, wherein the processor discards at least one non-moving object of the one or more objects.

14. A method for processing depth information of a scene, the method comprising:

receiving a depth image of the scene, wherein the depth image comprises one or more objects;

determining that at least one object of the one or more objects in the depth image is moving;

isolating the at least one object that is moving; and

processing the at least one isolated object.

15. The method of claim 14, further comprising determining whether to merge an object in the one or more objects with a nearby object in the one or more objects.

16. The method of claim 15, wherein determining whether to merge the object in the one or more objects with the nearby object in the one or more objects comprises:

comparing X-values, Y-values, or depth values associated with voxels in the object with X-values, Y-values, or depth values associated with voxels of the nearby object; and

merging the object with the nearby object if, based on the comparison, the X-values, Y-values, or the depth values associated with the voxels in the object overlap the X-values, Y-values, or depth values associated with the voxels of the nearby object.

17. The method of claim 14, wherein determining that at least one object of the one or more objects in the depth image is moving comprises:

comparing a value associated with each voxel in the one or more objects with a corresponding value of a reference voxel included in a reference plate; and

identifying each voxel in the one or more objects as moving when a difference between the value and the corresponding value exceeds a motion threshold.

18. The method of claim 17, wherein determining that at least one object of the one or more objects in the depth image is moving further comprises:

calculating a foreground score based on a percentage of moving voxels associated with the at least one object; and

identifying the at least one object as moving when the foreground score exceeds a motion threshold.

19. The method of claim 14, further comprising instructions for determining that at least one object of the one or more objects is non-moving.

20. The method of claim 19, further comprising instructions for discarding at least one non-moving object of the one or more objects.