SELECTIVE SURFACE MESH REGENERATION FOR 3-DIMENSIONAL RENDERINGS

• US 20160364907A1
• Filed: 06/10/2015
• Published: 12/15/2016
• Est. Priority Date: 06/10/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Schoenberg, Michael John
• Granted Patent: US 10,366,534 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0127 comprising devices increasing the depth of field
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06T 17/205 Re-meshing

Abstract

A 3D voxel grid of an environment is generated, with a signed value assigned to each voxel representative of that voxel'"'"'s distance from a nearest surface. For each subset of the 3D voxel grid comprising a surface, an initial surface mesh is generated. As new depth information is received, an updated surface is generated only for those subsets of the 3D voxel grid that exhibit greater than threshold change in signed values over time.

Claims

1. A method, comprising:

computer-generating a 3D voxel grid of an environment;

for each voxel in the 3D voxel grid, computer-assigning a signed value representative of that voxel'"'"'s distance from a nearest surface;

for each subset of the 3D voxel grid comprising a surface, computer-generating an initial surface mesh; and

selectively computer-generating an updated surface mesh on only those subsets of the 3D voxel grid that exhibit greater than threshold change in signed values over time.

2. The method of claim 1, wherein the 3D voxel grid is computer-generated from depth information received via a depth camera, the depth information comprising:

one or more depth images of the environment; and

information indicating a 6DOF pose of the depth camera relative to the environment at time of depth image capture.

3. The method of claim 2, wherein the signed value for a voxel is a signed distance field value proportionate to a distance between the voxel and the nearest surface.

4. The method of claim 3, wherein the nearest surface includes a surface point nearest to the voxel when compared to all other imaged surface points.

5. The method of claim 3, wherein the nearest surface includes a surface point located on a ray extending from the depth camera through the voxel.

6. The method of claim 3, wherein the signed distance field value is normalized and then truncated if an absolute value of the normalized signed distance field value is greater than a threshold.

7. The method of claim 1, wherein computer-generating an initial surface mesh comprises extracting an isosurface from the 3D voxel grid based on the signed values for each voxel within the 3D voxel grid.

8. The method of claim 7, wherein extracting an isosurface from the 3D voxel grid comprises executing a Marching Cubes algorithm on the 3D voxel grid.

9. The method of claim 1, further comprising:

for each voxel in the 3D voxel grid, computer-assigning an updated signed value based on the new depth information;

for each subset of the 3D voxel grid, recognizing a difference between the updated signed values and previously assigned signed values for all voxels within the subset; and

generating an updated surface mesh on only those subsets of the 3D voxel grid where the recognized difference is greater than a threshold.

10. The method of claim 9, wherein the recognized difference between each new signed value and each previous signed value is summed over time, and wherein an updated surface mesh is generated for subsets of the 3D voxel grid responsive to a sum of recognized differences exceeding the threshold.

11. The method of claim 9, where recognizing a difference between the updated signed values and previously assigned signed values for all voxels within the subset comprises:

calculating a sum of absolute differences between the updated signed values and previously assigned signed values at each voxel within the subset.

12. A 3D modeling computer, comprising:

a depth camera interface configured to receive depth information from a depth camera;

a logic machine; and

a storage machine holding instructions executable by a logic machine to;

receive the depth information for an environment;

generate a 3D voxel grid of the environment;

for each voxel in the 3D voxel grid, assign a signed value representative of that voxel'"'"'s distance from a nearest surface;

for each subset of the 3D voxel grid comprising a surface, generate an initial surface mesh based on the signed values; and

selectively generate an updated surface mesh on only those subsets of the 3D voxel grid that exhibit greater than threshold change in signed values over time.

13. The 3D modeling computer of claim 12, wherein receiving depth information for an environment comprises:

receiving one or more depth images of the environment from the depth camera; and

receiving 6DOF pose information from a sensor subsystem indicating a location and orientation of the depth camera at time of depth image capture.

14. The 3D modeling computer of claim 13, wherein the signed value for a voxel is a signed distance field value proportionate to a distance between the voxel and a nearest corresponding point in the one or more depth images.

15. The 3D modeling computer of claim 14, wherein the signed distance field value is normalized and then truncated if an absolute value of the normalized signed distance field value is greater than a threshold.

16. The 3D modeling computer of claim 12, wherein the storage machine further holds instructions executable by the logic machine to:

receive new depth information for the environment;

for each voxel in the 3D voxel grid, assign an updated signed value based on the new depth information;

for each subset of the 3D voxel grid, recognize a difference between the updated signed values and previously assigned signed values for all voxels within the subset; and

generate an updated surface mesh on only those subsets of the 3D voxel grid where the recognized difference is greater than a threshold.

17. The 3D modeling computer of claim 16, wherein the new depth information includes two or more subsequent depth images, wherein the recognized difference between each new signed value and each previous signed value is summed over time, and wherein an updated surface mesh is generated for subsets of the 3D voxel grid responsive to a sum of recognized differences exceeding the threshold.

18. The 3D modeling computer of claim 16, wherein the storage machine further holds instructions executable by the logic machine to:

recognize a difference between the updated signed values and previously assigned signed values by calculating a sum of absolute differences between the updated signed values and previously assigned signed values at each voxel within the subset.

19. A method, comprising:

receiving depth information for an environment via a depth camera;

computer-generating a 3D voxel grid of the environment;

for each voxel in the 3D voxel grid, computer-assigning a signed value representative of that voxel'"'"'s distance from a nearest surface;

for each subset of the 3D voxel grid comprising a surface, computer-generating with a marching cubes algorithm an initial surface mesh; and

selectively computer-generating with the marching cubes algorithm an updated surface mesh on only those subsets of the 3D voxel grid that exhibit greater than threshold change in signed values over time.

20. The method of claim 19, where greater than threshold change in signed values over time includes a sum of recognized changes exceeding the threshold.