Combining depth-maps from different acquisition methods

• US 20150024336A1
• Filed: 07/18/2014
• Published: 01/22/2015
• Est. Priority Date: 07/18/2013
• Status: Active Grant

• Current Assignee: A.Tron3D Gmbh
• Original Assignee: A.Tron3D Gmbh
• Inventors: ANGERER, Helmut, NOWAK, Christoph, JESENKO, Jurgen, KOINIG, Horst, BLASSNIG, Andreas, KELZ, Engelbert, ZINNER, Christian, THIERJUNG, Stefan
• Granted Patent: US 9,363,501 B2
• Time in Patent Office: Days
• US Class Current: 433/29
• CPC Class Codes: A61B 5/0088 for oral or dental tissue
• CPC Class Codes: A61C 19/04 Measuring instruments specially adapted for dentistry A61C5/44 takes precedence; testing vitality of teeth A61B5/0534; radiation diagnosis specially adapted for dentistry A61B6/51
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01B 2210/52 Combining or merging partially overlapping images to an overall image
• CPC Class Codes: G06T 15/06 Ray-tracing
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30036 Dental; Teeth
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/56 provided with illuminating means

Abstract

A method and system capturing three-dimensional information of a scene on a structure includes operating a light pattern projector to project a known light pattern onto the scene; using at least two cameras, taking images of the scene, the images being two-dimensional images taken chronologically synchronous; using first and second images, and the known position and orientation of the cameras with respect to each other, extracting a first depth map of the scene; using the first image, the known light pattern, and the known position and orientation of the first camera and the light pattern projector with respect to each other, extracting a second depth map of the scene; and using the extracted first depth map and the extracted second depth map of the scene together to make a 3D measurement of the scene.

Claims

1. A method of capturing three-dimensional information of a scene on a structure, comprising the steps of:

positioning at least two cameras and a light pattern projector in a known position and orientation with respect to each other;

operating the light pattern projector to project a known light pattern onto the scene on the structure;

using a first of the cameras, taking a first image of the scene, the first image being a two-dimensional image;

using a second of the cameras, taking a second image of the scene, the second image being a two-dimensional image, wherein the first and second images are taken chronologically synchronous;

using the chronologically synchronously taken first image and second image, and the known position and orientation of the cameras with respect to each other, extracting a first depth map of the scene;

using i) the first image, ii) the known light pattern, and iii) the known position and orientation of the first camera and the light pattern projector with respect to each other, extracting a second depth map of the scene; and

using the extracted first depth map and the extracted second depth map of the scene together to make a 3D measurement of the scene.

2. The method of claim 1, wherein the method comprises the step of:

using the second image, the known light pattern, and the known position and orientation of the second camera and the light pattern projector with respect to each other, extracting a third depth map of the scene.

3. The method of claim 1, wherein each depth map is added separately to a three-dimensional model of the object.

4. The method of claim 1, wherein the depth maps are first combined to create a further denser depth map and wherein said denser depth map is then added to a three-dimensional model of the object.

5. The method of claim 1, wherein the first depth map is extracted by stereometric means.

6. The method of claim 1, wherein the second depth map is extracted by structured light means.

7. The method of claim 1, wherein the structure is a natural structure.

8. The method of claim 1, wherein the structure is an artificial structure.

9. The method of claim 1, wherein the structure is an artificial intra-oral structure.

10. The method of claim 1, wherein light pattern is known random light pattern.

11. The method of claim 1, wherein light pattern is known static random light pattern.

12. The method of claim 1, wherein the first and second cameras are positioned respectively to each other in different first and second locations such that the first and second images are different from each other.

25. A non-transitory computer readable medium containing computer code that when executed on a computer causes the computer to execute the method of claim 1.

13. A system for capturing three-dimensional information of a scene on a structure, comprising:

a first camera, of at least two cameras,

a second camera, of said at least two cameras,

a light pattern projector having a known light pattern, the light pattern projector positioned to project the known light pattern onto the scene on the structure, wherein, the first and second cameras and the light pattern projector are in a known position and orientation with respect to each other; and

a computerized controller that;

i) operates the light pattern projector to project the known light pattern onto the scene on the structure;

ii) uses the first camera to take a first image of the scene having the projected known light pattern thereon, the first image being a two-dimensional image;

iii) uses the second camera to take a second image of the scene having the projected known light pattern thereon, the second image being a two-dimensional image, whereby the first and second images are chronologically synchronously taken stereo image pairs of the scene;

iv) uses the chronologically synchronously taken first image and second images, extracting a first depth map of the scene;

v) uses i) the first image, ii) the known light pattern, and iii) the known position and orientation of the first camera and the light pattern projector with respect to each other, extracting a second depth map of the scene; and

vi) uses the extracted first depth map and the extracted second depth map of the scene together to make a 3D measurement of the scene.

14. The system of claim 13, wherein the system uses the second image, the known light pattern, and the known position and orientation of the second camera and the light pattern projector with respect to each other, to extract a third depth map of the scene.

15. The system of claim 13, wherein the system adds each depth map separately to a three-dimensional model of the object.

16. The system of claim 13, wherein the system first combines the depth maps to create a further denser depth map and which is then added to a three-dimensional model of the object.

17. The system of claim 13, wherein the system extracts the first depth map employing stereometric means.

18. The system of claim 13, wherein the system extracts the second depth map employing structured light means.

19. The system of claim 13, wherein the structure is a natural structure.

20. The system of claim 13, wherein the structure is an artificial structure.

21. The system of claim 13, wherein the structure is an artificial intra-oral structure.

22. The system of claim 13, wherein light pattern is known random light pattern.

23. The system of claim 13, wherein light pattern is known static random light pattern.

24. The system of claim 13, wherein the first and second cameras are positioned respectively in different first and second locations such that the first and second images are different from each other.