Method and apparatus for colour imaging a three-dimensional structure

• US 9,404,740 B2
• Filed: 06/30/2015
• Issued: 08/02/2016
• Est. Priority Date: 06/17/2004
• Status: Active Grant

• Current Assignee: Align Technology Incorporated
• Original Assignee: Align Technology Incorporated
• Inventors: Babayoff, Noam
• Primary Examiner(s): Geisel, Kara E
• Assistant Examiner(s): ALLI, IYABO
• Application Number:
  US14/755,171
• Publication Number:
  US 20150297329A1
• Time in Patent Office: 399 Days
• Field of Search: 2505594-55949, 356/479, 356601-609
• US Class Current: 1/1
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/00096 Optical elements
• CPC Class Codes: A61B 1/0615 for radial illumination
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/0646 with illumination filters
• CPC Class Codes: A61B 1/0676 at distal tip of an endoscope
• CPC Class Codes: A61B 1/0684 using light emitting diodes [LED]
• CPC Class Codes: A61B 1/24 for the mouth, i.e. stomatoscopes, e.g. with tongue depressors tongue depressors per se A61B13/00; Instruments for opening or keeping open the mouth combined with saliva removers A61C17/00; mouth openers for animals A61D15/00
• CPC Class Codes: A61B 1/247 with means for viewing areas outside the direct line of sight, e.g. dentists' mirrors
• CPC Class Codes: A61B 5/0068 Confocal scanning
• CPC Class Codes: A61B 5/0088 for oral or dental tissue
• CPC Class Codes: A61B 5/1077 Measuring of profiles
• CPC Class Codes: A61B 5/1079 using optical or photographic means
• CPC Class Codes: A61C 19/04 Measuring instruments specially adapted for dentistry A61C5/44 takes precedence; testing vitality of teeth A61B5/0534; radiation diagnosis specially adapted for dentistry A61B6/51
• CPC Class Codes: A61C 9/0053 Optical means or methods, e.g. scanning the teeth by a laser or light beam
• CPC Class Codes: A61C 9/0066 Depth determination through adaptive focusing
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01J 3/02 Details
• CPC Class Codes: G01J 3/0205 Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
• CPC Class Codes: G01J 3/0208 : using focussing or collimating elements, e.g. lenses or mirrors; performing aberration correction
• CPC Class Codes: G01J 3/0216 : using light concentrators or collectors or condensers
• CPC Class Codes: G01J 3/0218 : using optical fibers
• CPC Class Codes: G01J 3/0224 : using polarising or depolarising elements
• CPC Class Codes: G01J 3/0243 : having a through-hole enabling the optical element to fulfil an additional optical function, e.g. a mirror or grating having a throughhole for a light collecting or light injecting optical fiber
• CPC Class Codes: G01J 3/0256 : Compact construction
• CPC Class Codes: G01J 3/10 : Arrangements of light sources specially adapted for spectrometry or colorimetry
• CPC Class Codes: G01J 3/462 : Computing operations in or between colour spaces; Colour management systems
• CPC Class Codes: G01J 3/50 : using electric radiation detectors
• CPC Class Codes: G01J 3/501 : Colorimeters using spectrally-selective light sources, e.g. LEDs
• CPC Class Codes: G01J 3/508 : measuring the colour of teeth
• CPC Class Codes: G01J 3/51 : using colour filters
• CPC Class Codes: G01N 21/255 : Details, e.g. use of specially adapted sources, lighting or optical systems
• CPC Class Codes: G06T 2207/10024 : Color image
• CPC Class Codes: G06T 2207/10028 : Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30036 : Dental; Teeth
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection
• CPC Class Codes: G06T 7/12 : Edge-based segmentation
• CPC Class Codes: G06T 7/90 : Determination of colour characteristics
• CPC Class Codes: H01L 27/14868 : CCD or CID colour imagers
• CPC Class Codes: H04N 13/15 : for colour aspects of image signals
• CPC Class Codes: H04N 13/207 : using a single 2D image sensor
• CPC Class Codes: H04N 13/257 : Colour aspects
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 : Synchronisation thereof; Control thereof

Abstract

A device for determining the surface topology and associated color of a structure, such as a teeth segment, includes a scanner for providing depth data for points along a two-dimensional array substantially orthogonal to the depth direction, and an image acquisition means for providing color data for each of the points of the array, while the spatial disposition of the device with respect to the structure is maintained substantially unchanged. A processor combines the color data and depth data for each point in the array, thereby providing a three-dimensional color virtual model of the surface of the structure. A corresponding method for determining the surface topology and associate color of a structure is also provided.

Claims

1. A system for determining surface topology and associated color of a structure portion of a three-dimensional structure, the system comprising:

an imaging device configured to generate three-dimensional depth data and two-dimensional color data of the structure portion; and

one or more processors operably coupled to the imaging device, the one or more processors configured to cause the system to;

generate the three-dimensional depth data of the structure portion using the imaging device when the imaging device is at a first spatial disposition relative to the structure portion,

generate the two-dimensional color data of the structure portion using the imaging device when the imaging device is at a second spatial disposition relative to the structure portion,

determine a spatial transformation between the three-dimensional depth data and the two-dimensional color data, and

map the two-dimensional color data to the three-dimensional depth data using the determined spatial transformation.

2. The system of claim 1, wherein the second spatial disposition differs from the first spatial disposition by one or more of a translation along a depth direction or a rotation about the depth direction.

3. The system of claim 1, wherein the spatial transformation is determined by performing an alignment procedure.

4. The system of claim 3, wherein the alignment procedure comprises an optical character recognition technique.

5. The system of claim 3, wherein the alignment procedure comprises:

determining a first shape profile based on the three-dimensional depth data;

determining a second shape profile based on the two-dimensional color data; and

aligning the first shape profile to the second shape profile.

6. The system of claim 5, wherein aligning the first shape profile to the second shape profile comprises performing one or more of a coplanar translation or a coplanar rotation.

7. The system of claim 5, wherein aligning the first shape profile to the second shape profile comprises determining a best fit between the first shape profile and the second shape profile.

8. The system of claim 1, wherein the three-dimensional depth data and the two-dimensional color data are generated by the imaging device within a time interval of less than about 100 milliseconds.

9. The system of claim 1, wherein the imaging device comprises a monochromatic image sensor used to generate the three-dimensional depth data and the two-dimensional color data.

10. The system of claim 1, wherein the imaging device comprises a color image sensor used to generate the three-dimensional depth data and the two-dimensional color data.

11. A method for determining surface topology and associated color of a structure portion of a three-dimensional structure, the method comprising:

generating, using an imaging device, three-dimensional depth data of the structure portion when the imaging device is at a first spatial disposition relative to the structure portion;

generating, using the imaging device, two-dimensional color data of the structure portion when the imaging device is at a second spatial disposition relative to the structure portion;

determining, using one or more processors, a spatial transformation between the three-dimensional depth data and the two-dimensional color data; and

mapping, using the one or more processors, the two-dimensional color data to the three-dimensional depth data using the determined spatial transformation.

12. The method of claim 11, wherein the second spatial disposition differs from the first spatial disposition by one or more of a translation along a depth direction or a rotation about the depth direction.

13. The method of claim 11, wherein determining the spatial transformation comprises performing an alignment procedure.

14. The method of claim 13, wherein the alignment procedure comprises an optical character recognition technique.

15. The method of claim 13, wherein performing the alignment procedure comprises:

determining a first shape profile based on the three-dimensional depth data;

determining a second shape profile based on the two-dimensional color data; and

aligning the first shape profile to the second shape profile.

16. The method of claim 15, wherein aligning the first shape profile to the second shape profile comprises performing one or more of a coplanar translation or a coplanar rotation.

17. The method of claim 15, wherein aligning the first shape profile to the second shape profile comprises determining a best fit between the first shape profile and the second shape profile.

18. The method of claim 11, wherein the three-dimensional depth data and the two-dimensional color data are generated by the imaging device within a time interval of less than about 100 milliseconds.

19. The method of claim 11, wherein the imaging device comprises a monochromatic image sensor used to generate the three-dimensional depth data and the two-dimensional color data.

20. The method of claim 11, wherein the imaging device comprises a color image sensor used to generate the three-dimensional depth data and the two-dimensional color data.