Method and apparatus for colour imaging a three-dimensional structure

• US 10,750,151 B2
• Filed: 02/14/2020
• Issued: 08/18/2020
• Est. Priority Date: 06/17/2004
• Status: Active Grant

• Current Assignee: Align Technology Incorporated
• Original Assignee: Align Technology Incorporated
• Inventors: Babayoff, Noam
• Primary Examiner(s): Volentine, Rebecca A
• Application Number:
  US16/791,994
• Publication Number:
  US 20200186777A1
• Time in Patent Office: 186 Days
• Field of Search: None
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/00096 Optical elements
• CPC Class Codes: A61B 1/0615 for radial illumination
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/0646 with illumination filters
• CPC Class Codes: A61B 1/0676 at distal tip of an endoscope
• CPC Class Codes: A61B 1/0684 using light emitting diodes [LED]
• CPC Class Codes: A61B 1/24 for the mouth, i.e. stomatoscopes, e.g. with tongue depressors tongue depressors per se A61B13/00; Instruments for opening or keeping open the mouth combined with saliva removers A61C17/00; mouth openers for animals A61D15/00
• CPC Class Codes: A61B 1/247 with means for viewing areas outside the direct line of sight, e.g. dentists' mirrors
• CPC Class Codes: A61B 5/0068 Confocal scanning
• CPC Class Codes: A61B 5/0088 for oral or dental tissue
• CPC Class Codes: A61B 5/1077 Measuring of profiles
• CPC Class Codes: A61B 5/1079 using optical or photographic means
• CPC Class Codes: A61C 19/04 Measuring instruments specially adapted for dentistry A61C5/44 takes precedence; testing vitality of teeth A61B5/0534; radiation diagnosis specially adapted for dentistry A61B6/51
• CPC Class Codes: A61C 9/0053 Optical means or methods, e.g. scanning the teeth by a laser or light beam
• CPC Class Codes: A61C 9/0066 Depth determination through adaptive focusing
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01J 3/02 Details
• CPC Class Codes: G01J 3/0205 Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
• CPC Class Codes: G01J 3/0208 : using focussing or collimating elements, e.g. lenses or mirrors; performing aberration correction
• CPC Class Codes: G01J 3/0216 : using light concentrators or collectors or condensers
• CPC Class Codes: G01J 3/0218 : using optical fibers
• CPC Class Codes: G01J 3/0224 : using polarising or depolarising elements
• CPC Class Codes: G01J 3/0243 : having a through-hole enabling the optical element to fulfil an additional optical function, e.g. a mirror or grating having a throughhole for a light collecting or light injecting optical fiber
• CPC Class Codes: G01J 3/0256 : Compact construction
• CPC Class Codes: G01J 3/10 : Arrangements of light sources specially adapted for spectrometry or colorimetry
• CPC Class Codes: G01J 3/462 : Computing operations in or between colour spaces; Colour management systems
• CPC Class Codes: G01J 3/50 : using electric radiation detectors
• CPC Class Codes: G01J 3/501 : Colorimeters using spectrally-selective light sources, e.g. LEDs
• CPC Class Codes: G01J 3/508 : measuring the colour of teeth
• CPC Class Codes: G01J 3/51 : using colour filters
• CPC Class Codes: G01N 21/255 : Details, e.g. use of specially adapted sources, lighting or optical systems
• CPC Class Codes: G06T 2207/10024 : Color image
• CPC Class Codes: G06T 2207/10028 : Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30036 : Dental; Teeth
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection
• CPC Class Codes: G06T 7/12 : Edge-based segmentation
• CPC Class Codes: G06T 7/90 : Determination of colour characteristics
• CPC Class Codes: H01L 27/14868 : CCD or CID colour imagers
• CPC Class Codes: H04N 13/15 : for colour aspects of image signals
• CPC Class Codes: H04N 13/207 : using a single 2D image sensor
• CPC Class Codes: H04N 13/257 : Colour aspects
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 : Synchronisation thereof; Control thereof

Abstract

A device for determining the surface topology and associated color of a structure, such as a teeth segment, includes a scanner for providing depth data for points along a two-dimensional array substantially orthogonal to the depth direction, and an image acquisition means for providing color data for each of the points of the array, while the spatial disposition of the device with respect to the structure is maintained substantially unchanged. A processor combines the color data and depth data for each point in the array, thereby providing a three-dimensional color virtual model of the surface of the structure. A corresponding method for determining the surface topology and associate color of a structure is also provided.

Claims

1. A method for determining surface topology and associated color of an intraoral structure, the method comprising:

illuminating the intraoral structure using white light passing through focusing optics;

capturing depth image data and color image data of the intraoral structure from the white light illuminating the intraoral structure using an image sensor of an imaging device, wherein the capturing includes scanning a focal plane of the imaging device over a range of depths and wherein returned lights beams returning from the intraoral structure pass through the focusing optics;

generating, using the imaging device and the depth image data, depth data of the intraoral structure;

generating, using the imaging device and the color image data, color data of the intraoral structure; and

providing, using one or more processors, a color three-dimensional numerical entity based on the depth data and the color data.

2. The method of claim 1, further comprising determining, using one or more processors, a spatial transformation between the depth data and the color data, wherein determining the spatial transformation comprises performing an alignment procedure.

3. The method of claim 2, wherein performing the alignment procedure comprises:

determining a first shape profile based on the depth data;

determining a second shape profile based on the color data; and

aligning the first shape profile to the second shape profile.

4. The method of claim 1, wherein the image sensor is a color image sensor.

5. The method of claim 1, wherein illuminating the intraoral structure using the white light passing through the focusing optics further comprises passing the white light through a plurality of color filters.

6. The method of claim 5, wherein the plurality of color filters are a tri-color filter.

7. The method of claim 5, wherein a filter wheel includes the plurality of color filters.

8. The method of claim 6, further comprising rotating the filter wheel to sequentially present each color of the filter wheel to the white light.

9. The method of claim 5, wherein the image sensor is a monochrome image sensor.

10. A method for determining surface topology and associated color of an intraoral structure, the method comprising:

illuminating the intraoral structure with white light;

capturing depth image data and color image data of the intraoral structure from the white light illuminating the intraoral structure using an image sensor of an imaging device, wherein the capturing includes scanning, using focusing optics, a focal plane of the imaging device over a range of depths and wherein the capturing of the depth image data and the color image data occurs at the same first angle and orientation with respect to the intraoral structure, and wherein the capturing includes capturing returning light passing though the focusing optics from the intraoral structure;

generating, using the imaging device and the depth image data, depth data at the first angle and orientation with respect to the intraoral structure;

generating, using the imaging device and the color image data, color data at the first angle and orientation with respect to the intraoral structure; and

providing, using one or more processors, a color three-dimensional numerical entity based on the depth data and the color data.

11. The method of claim 10, wherein providing the color three-dimensional numerical entity based on the depth data and the color data comprises generating a plurality of data points of the color three-dimensional numerical entity, each of the plurality of data points comprising three-dimensional surface coordinate data and color data associated therewith.

12. The method of claim 10, wherein the providing comprises performing an alignment procedure comprising:

determining a first shape profile based on the depth data;

determining a second shape profile based on the color data; and

aligning the first shape profile to the second shape profile.

13. The method of claim 10, wherein the image sensor is a color image sensor.

14. The method of claim 10, wherein illuminating the intraoral structure with the white light passing through the focusing optics further comprises passing the white light through a plurality of color filters.

15. The method of claim 14, wherein a filter wheel includes the plurality of color filters.

16. The method of claim 15, further comprising rotating the filter wheel to sequentially present each color of the filter wheel to the white light.

17. The method of claim 14, wherein the image sensor is a monochrome image sensor.

18. A system for determining surface topology and associated color of an intraoral structure, the system comprising:

a handheld imaging device comprising a probe, focusing optics configured to scan a focal plane of the handheld imaging device over a range of depths, a white light illuminator, and an image sensor configured to capture depth image data over the range of depths and color image data of the intraoral structure, wherein incident light beams from the white light illuminator and returned lights beams returning from the intraoral structure pass through the focusing optics; and

one or more processors operably coupled to the handheld imaging device, the one or more processors configured to cause the system to;

generate depth data of the intraoral structure using the depth image data from the handheld imaging device,

generate color data of the intraoral structure using the color image data from the handheld imaging device, and

provide a color three-dimensional numerical entity based on the depth data and the color data.

19. The system of claim 18, wherein the image sensor is a color image sensor.

20. The system of claim 18, further comprising a plurality of color filters positioned between the white light source and the intraoral structure.

21. The system of claim 20, wherein the plurality of color filters are a tri-color filter.

22. The system of claim 20, wherein a filter wheel includes the plurality of color filters.

23. The system of claim 21, wherein the handheld imaging device is configured to rotate the filter wheel to sequentially present each color of the filter wheel to the light.

24. The system of claim 20, wherein the image sensor is a monochrome image sensor.

25. A system for determining surface topology and associated color of an intraoral structure, the system comprising:

a handheld imaging device comprising a probe, focusing optics configured to scan a focal plane of the handheld imaging device over a range of depths, a white light illuminator, and an image sensor configured to capture depth image data over the range of depths and color image data of the intraoral structure by being configured to capture returning light passing through the focusing optics from the intraoral structure; and

one or more processors operably coupled to the handheld imaging device, the one or more processors configured to cause the system to;

capture the depth image data and the color image data of the intraoral structure over the range of depths and wherein the capturing of the depth image data and the color image data occurs at the same first angle and orientation with respect to the intraoral structure;

generate, using the imaging device and the depth image data, depth data at the first angle and orientation with respect to the intraoral structure;

generate, using the imaging device and the color image data, color data at the first angle and orientation with respect to the intraoral structure; and

provide, using one or more processors, a color three-dimensional numerical entity based on the depth data and the color data.

26. The system of claim 25, further comprising a plurality of color filters between the white light illuminator and the intraoral structure.

27. The system of claim 26, wherein a filter wheel includes the plurality of color filters.

28. The system of claim 27, wherein the handheld imaging device is configured to rotate the filter wheel to sequentially present each color of the filter wheel to the white light illuminator.

29. The system of claim 26, wherein the image sensor is a monochrome image sensor.

30. The system of claim 25, wherein the image sensor is a color image sensor.