Interpolating oriented disks in 3D space for constructing high fidelity geometric proxies from point clouds

• US 9,256,980 B2
• Filed: 09/13/2012
• Issued: 02/09/2016
• Est. Priority Date: 05/31/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kirk, Adam
• Primary Examiner(s): Cain, II, Leon T
• Application Number:
  US13/614,852
• Publication Number:
  US 20130321418A1
• Time in Patent Office: 1,244 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2210/56 Particle system, point based geometry or rendering
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04S 2400/15 Aspects of sound capture and related signal processing for recording or reproduction

Abstract

An “Oriented Disk Interpolator” provides various techniques for interpolating between points in a point cloud using RGB images (or images in other color spaces) to produce a smooth implicit surface representation that can then be digitally sampled for ray-tracing or meshing to create a high fidelity geometric proxy from the point cloud. More specifically, the Oriented Disk Interpolator uses image color-based consistency to build an implicit surface from oriented points and images of the scene by interpolating disks in 3D space relative to a point cloud of a scene or objects within the scene. The resulting implicit surface is then available for a number of uses, including, but not limited to, constructing a high fidelity geometric proxy.

Claims

1. A computer-implemented process for constructing implicit surfaces from point clouds, comprising using a computing device to perform process actions for:

receiving an oriented point cloud, each point in the cloud having a normal direction;

for each point in the cloud, identifying a set of closest neighboring points within a radius around each point;

defining a set of oriented disks, such that for each point in the cloud, a corresponding one of the oriented disks is delimited by the position, normal and radius of the corresponding point;

for each node in a volumetric grid, identifying a corresponding set of neighboring oriented disks as any oriented disk that is closer to the node than the radius of that oriented disk;

for each node having a set of neighboring oriented disks, computing a signed distance as a weighted average of a signed distance to each neighboring oriented disk; and

delimiting an implicit surface corresponding to the oriented point cloud by using the signed distances of each node to construct a signed distance function on the volumetric grid.

2. The computer-implemented process of claim 1 wherein:

a number, n, of closest neighboring points for each point in the oriented point cloud is an adjustable parameter; and

wherein the radius around each point, and thus the radius of each corresponding oriented disk, varies to encompass n neighboring points.

3. The computer-implemented process of claim 1 wherein the radius around each point is computed as a weighted average of the distances to the set of neighboring points.

4. The computer-implemented process of claim 1 further comprising receiving a plurality of color images representing 3D image data of a scene and wherein the oriented point cloud is derived based on the 3D image data.

5. The computer-implemented process of claim 4 further comprising determining oriented disk colors by projecting each oriented disk into each of the color images in which the oriented disk is visible relative to a geometry of corresponding sensor calibration parameters.

6. The computer-implemented process of claim 5 wherein a color of each grid node is determined by projecting each grid node onto each neighboring oriented disk, and computing a bilateral function of the differences of the oriented disk colors and a distance to the oriented disk from the corresponding node.

7. The computer-implemented process of claim 1 further comprising ray-tracing the implicit surface to create a 3D surface visualization of the implicit surface.

8. The computer-implemented process of claim 1 further comprising meshing the implicit surface to create a 3D geometric proxy.

9. The computer-implemented process of claim 1 wherein a density of the nodes in the volumetric grid is an adjustable parameter.

10. A computer-readable memory device having computer executable instructions stored therein for generating implicit surfaces from point clouds, said instructions causing a computing device to execute a method comprising:

constructing a point cloud from 3D image data of a scene and determining a normal for each point to create an oriented point cloud;

for each point in the cloud, identifying a set of the closest n neighboring points around each point and determining a radius around each point as a function of the number and distances of those neighboring points;

defining a set of oriented disks, such that for each point in the cloud, a corresponding one of the oriented disks is delimited by the position, normal and radius of the corresponding point;

for each node in a volumetric grid, identifying a corresponding set of neighboring oriented disks as any oriented disk that is closer to the node than the radius of that oriented disk;

for each node having a set of neighboring oriented disks, computing a signed distance as a function of the distances from the node to each neighboring oriented disk; and

delimiting an implicit surface corresponding to the oriented point cloud by using the signed distances of each node to construct a signed distance function on the volumetric grid.

11. The computer-readable memory device of claim 10 wherein the number, n, of the closest neighboring points is an adjustable parameter.

12. The computer-readable memory device of claim 10 wherein the radius around each point is computed as a weighted average of the distances to the set of neighboring points.

13. The computer-readable memory device of claim 10 further comprising determining oriented disk colors from the 3D image data.

14. The computer-readable memory device of claim 13 wherein a color of each grid node is determined based on the colors of each neighboring oriented disk.

15. The computer-readable memory device of claim 10 further comprising ray-tracing the implicit surface to create a 3D surface visualization of the 3D image data.

16. The computer-readable memory device of claim 10 further comprising meshing the implicit surface to create a 3D geometric proxy of the 3D image data.

17. A computer-implemented process for creating an implicit surface representation of a 3D scene, comprising using a computing device to perform process actions for:

constructing a point cloud from 3D image data of a scene and determining a normal for each point to create an oriented point cloud;

for each point in the cloud, identifying a set of the closest n neighboring points around each point and determining a radius around each point as a function of the number and distances of those neighboring points;

defining a set of oriented disks, such that for each point in the cloud, a corresponding one of the oriented disks is delimited by the position, normal and radius of the corresponding point;

determining oriented disk colors from the 3D image data;

for each node in a volumetric grid, identifying a corresponding set of neighboring oriented disks as any oriented disk that is closer to the node than the radius of that oriented disk;

for each node having a set of neighboring oriented disks, computing a signed distance as a function of the distances from the node to each neighboring oriented disk;

determining a color of each grid node based on the colors of each neighboring oriented disk; and

delimiting an implicit surface corresponding to the oriented point cloud by using the signed distances and colors of each node to construct a signed distance function on the volumetric grid.

18. The computer-implemented process of claim 17 wherein the number, n, of the closest neighboring points is an adjustable parameter.

19. The computer-implemented process of claim 17 wherein the radius around each point is computed as a weighted average of the distances to the set of neighboring points.

20. The computer-implemented process of claim 17 further comprising using the signed distance function to construct a 3D geometric proxy of the 3D image data.