METHOD AND SYSTEM TO SEGMENT DEPTH IMAGES AND TO DETECT SHAPES IN THREE-DIMENSIONALLY ACQUIRED DATA

• US 20130057654A1
• Filed: 10/31/2012
• Published: 03/07/2013
• Est. Priority Date: 02/08/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rafii, Abbas, Gokturk, Salih Burak
• Granted Patent: US 9,311,715 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

A method and system analyzes data acquired by image systems to more rapidly identify objects of interest in the data. In one embodiment, z-depth data are segmented such that neighboring image pixels having similar z-depths are given a common label. Blobs, or groups of pixels with a same label, may be defined to correspond to different objects. Blobs preferably are modeled as primitives to more rapidly identify objects in the acquired image. In some embodiments, a modified connected component analysis is carried out where image pixels are pre-grouped into regions of different depth values preferably using a depth value histogram. The histogram is divided into regions and image cluster centers are determined. A depth group value image containing blobs is obtained, with each pixel being assigned to one of the depth groups.

Claims

1. One or more computer-storage media having computer-executable instructions embodied thereon that when executed by a computing device perform a method of three-dimensional (“

3D”

) image analysis, the method comprising;

receiving 3D image data comprising a plurality of points having 3D coordinate information;

pre-grouping points into sub-regions based upon z-depth values;

grouping at least some points within a sub-region to form a segment corresponding to a real-world object of interest;

calculating an image cluster center for the segment; and

modeling the segment as a geometric primitive shape.

2. The media of claim 1, wherein the real-world object of interest is a human hand.

3. The media of claim 1, wherein the real-world object of interest is a person'"'"'s body part.

4. The media of claim 1, wherein the real-world object of interest is a person.

5. The media of claim 1, wherein the image cluster center has an assigned depth value that is an average of depth values assigned to points forming the segment.

6. The media of claim 1, wherein the method further comprises using a time-of-flight 3D camera to capture the 3D image data.

7. One or more computer-storage media having computer-executable instructions embodied thereon that when executed by a computing device perform a method of three-dimensional (“

3D”

) image analysis, the method comprising;

receiving 3D image data comprising a plurality of points having 3D coordinate information;

pre-grouping points into sub-regions;

grouping at least some points within a sub-region to form a segment corresponding to a real-world object of interest;

calculating an image cluster center for the segment; and

modeling the segment as a plurality of geometric primitive shapes.

8. The media of claim 7, wherein the sub-regions correspond to an object described by the 3D image data.

10. The media of claim 8, wherein the image cluster center has a depth value equal to an average of depth values of points forming the segment.

11. The media of claim 7, wherein the real-world object of interest is a person.

12. The media of claim 7, where in the real-world object of interest corresponds to part of the person.

13. The media of claim 7, wherein grouping together points into the segment is based upon each point'"'"'s z-depth value.

14. The media of claim 7, wherein the sub-regions correspond to a predefined object that is first recognized from the 3D image data.

15. A method of object analysis using three-dimensional image data, the method comprising:

receiving 3D image data comprising a plurality of points having 3D coordinate information;

pre-grouping points into sub-regions based upon a histogram record of z-depth values;

grouping at least some points within a sub-region to form a segment corresponding to a real-world object;

calculating an image cluster center for the segment; and

modeling the segment as a plurality of geometric primitive shapes.

16. The method of claim 15, wherein the geometric primitive shapes correspond to a 3D object.

17. The method of claim 15, wherein grouping together at least some points is done based upon their z-depth values and the points are within a number k of neighboring pixels

18. The method of claim 17, wherein k is selected from a group consisting of k=4 and k=8.

19. The method of claim 15, wherein the image center has a depth value equal to an average of depth values of points forming the segment.

20. The method of claim 15, wherein the 3D image data is received from one or more of an intensity-based system, a time-of-flight system, and a phase-shift z-depth measuring system.