Method and system to segment depth images and to detect shapes in three-dimensionally acquired data

• US 20060239558A1
• Filed: 02/06/2006
• Published: 10/26/2006
• Est. Priority Date: 02/08/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Rafii, Abbes, Gokturk, Salih Burak
• Granted Patent: US 8,009,871 B2
• Time in Patent Office: Days
• US Class Current: 382/181
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

A method and system analyzes data acquired by image systems to more rapidly identify objects of interest in the data. In one embodiment, z-depth data are segmented such that neighboring image pixels having similar z-depths are given a common label. Blobs, or groups of pixels with a same label, may be defined to correspond to different objects. Blobs preferably are modeled as primitives to more rapidly identify objects in the acquired image. In some embodiments, a modified connected component analysis is carried out where image pixels are pre-grouped into regions of different depth values preferably using a depth value histogram. The histogram is divided into regions and image cluster centers are determined. A depth group value image containing blobs is obtained, with each pixel being assigned to one of the depth groups.

Claims

1. A method to recognize at least one object in image data acquired by an imaging system whose output is representable as an array of pixels that acquire at least one of intensity acquired data and z-depth acquired data, the method comprising the following steps:

(a) examining at least some of said image data;

(b) grouping together to form segments those of said pixels whose acquired data falls within a predetermined threshold value;

(c) assigning on a per-segment basis a label to pixels comprising each of said segments;

(d) defining at least one pre-defined blob;

(e) examining said segments to determine presence in said image data of a pre-defined blob; and

(f) outputting a signal appropriate to a recognized pre-defined blob.

2. The method of claim 1, wherein:

step (b) includes grouping together pixels based upon their intensity acquired data.

3. The method of claim 1, wherein:

step (b) includes grouping together pixels based upon their z-depth acquired data.

4. The method of claim 1, wherein step (b) includes grouping together pixels based upon their z-depth acquired data, where said grouping together is based upon pre-determined z-depth values.

5. The method of claim 1, wherein step (b) includes grouping together pixels based upon their z-depth acquired data, wherein grouped-together said pixels are within a number k of neighboring pixels, where k is selected from a group consisting of k=4 and k=8.

6. The method of claim 1, wherein step (b) includes grouping together pixels based upon their z=depth acquired data, wherein pixels representing an overlapping portion of objects are assigned a single label.

7. The method of claim 1, wherein step (b) includes grouping together pixels based upon their z=depth acquired data, wherein pixels representing an overlapping portion of objects are assigned values intermediate values associated with said objects.

8. The method of claim 1, wherein step (b) includes grouping together pixels based upon their z=depth acquired data, wherein pixels are pre-grouped based upon a histogram record of pixel z-depth values.

9. The method of claim 8, further including dividing said histogram into at least two sub-regions and determining image cluster centers such that each relevant said pixel is assigned to a z-depth-determined group.

10. The method of claim 9, wherein at least two of said sub-regions overlap.

11. The method of claim 1, wherein said imaging system is selected from a group consisting of (i) an intensity-based system, (ii) a time-of-flight system, and (iii) a phase-shift z-depth measuring system.

12. An image-analyzing system, to recognize at least one object in image data acquired by an imaging system whose output is representable as an array of pixels that acquire at least one of intensity acquired data and z-depth acquired data, the image-analyzing system comprising:

means for examining at least some of said image data;

means for grouping together those of said pixels whose acquired data falls within a predetermined threshold value, to form segments;

means for assigning on a per-segment basis a label to pixels comprising each of said segments;

memory storing at least one pre-defined blob;

means for comparing said segments with contents of said memory to determine presence in said image data of a pre-defined blob; and

means for outputting a signal appropriate to a recognized pre-defined blob.

13. The image-analyzing system of claim 12, wherein said means for grouping carries out grouping based upon at least one pixel characteristic selected from a group consisting of intensity data acquired by said pixels, z-depth data acquired by said pixels, and z-depth acquired data using pre-determined z-depth values.

14. The image-analyzing system of claim 13, wherein said means for grouping forms grouped-together pixels that are within a number k of neighboring pixels, where k is selected from a group consisting of k=4 and k=8.

15. The image-analyzing system of claim 13, wherein said means for grouping groups-together pixels based upon their z=depth acquired data, wherein pixels representing an overlapping portion of objects are assigned a single label.

16. The image-analyzing system of claim 13, wherein said means for grouping groups-together pixels based upon their z=depth acquired data, wherein pixels representing an overlapping portion of objects are assigned values intermediate values associated with said objects.

17. The image-analyzing system of claim 13, wherein said means for grouping groups together pixels based upon their z=depth acquired data, wherein pixels are pre-grouped based upon a histogram record of pixel z-depth values.

18. The image-analyzing system of claim 17, wherein said histogram is divided into at least two sub-regions, and image cluster centers are determined such that each relevant said pixel is assigned to a z-depth-determined group.

19. The image-analyzing system of claim 12, wherein at least two of said means for examining, said means for grouping, and said means for assigning are carried out in software executable by a microprocessor in said image-analyzing system.

20. The image-analyzing system of claim 12, wherein said imaging system is selected from a group consisting of (i) an intensity-based system, (ii) a time-of-flight system, and (iii) a phase-shift z-depth measuring system.