Method and apparatus for colour imaging a three-dimensional structure

• US 10,728,519 B2
• Filed: 04/02/2019
• Issued: 07/28/2020
• Est. Priority Date: 06/17/2004
• Status: Active Grant

• Current Assignee: Align Technology Incorporated
• Original Assignee: Align Technology Incorporated
• Inventors: Babayoff, Noam
• Primary Examiner(s): Volentine, Rebecca A
• Application Number:
  US16/373,482
• Publication Number:
  US 20190230336A1
• Time in Patent Office: 483 Days
• Field of Search: None
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/00096 Optical elements
• CPC Class Codes: A61B 1/0615 for radial illumination
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/0646 with illumination filters
• CPC Class Codes: A61B 1/0676 at distal tip of an endoscope
• CPC Class Codes: A61B 1/0684 using light emitting diodes [LED]
• CPC Class Codes: A61B 1/24 for the mouth, i.e. stomatoscopes, e.g. with tongue depressors tongue depressors per se A61B13/00; Instruments for opening or keeping open the mouth combined with saliva removers A61C17/00; mouth openers for animals A61D15/00
• CPC Class Codes: A61B 1/247 with means for viewing areas outside the direct line of sight, e.g. dentists' mirrors
• CPC Class Codes: A61B 5/0068 Confocal scanning
• CPC Class Codes: A61B 5/0088 for oral or dental tissue
• CPC Class Codes: A61B 5/1077 Measuring of profiles
• CPC Class Codes: A61B 5/1079 using optical or photographic means
• CPC Class Codes: A61C 19/04 Measuring instruments specially adapted for dentistry A61C5/44 takes precedence; testing vitality of teeth A61B5/0534; radiation diagnosis specially adapted for dentistry A61B6/51
• CPC Class Codes: A61C 9/0053 Optical means or methods, e.g. scanning the teeth by a laser or light beam
• CPC Class Codes: A61C 9/0066 Depth determination through adaptive focusing
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01J 3/02 Details
• CPC Class Codes: G01J 3/0205 Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
• CPC Class Codes: G01J 3/0208 : using focussing or collimating elements, e.g. lenses or mirrors; performing aberration correction
• CPC Class Codes: G01J 3/0216 : using light concentrators or collectors or condensers
• CPC Class Codes: G01J 3/0218 : using optical fibers
• CPC Class Codes: G01J 3/0224 : using polarising or depolarising elements
• CPC Class Codes: G01J 3/0243 : having a through-hole enabling the optical element to fulfil an additional optical function, e.g. a mirror or grating having a throughhole for a light collecting or light injecting optical fiber
• CPC Class Codes: G01J 3/0256 : Compact construction
• CPC Class Codes: G01J 3/10 : Arrangements of light sources specially adapted for spectrometry or colorimetry
• CPC Class Codes: G01J 3/462 : Computing operations in or between colour spaces; Colour management systems
• CPC Class Codes: G01J 3/50 : using electric radiation detectors
• CPC Class Codes: G01J 3/501 : Colorimeters using spectrally-selective light sources, e.g. LEDs
• CPC Class Codes: G01J 3/508 : measuring the colour of teeth
• CPC Class Codes: G01J 3/51 : using colour filters
• CPC Class Codes: G01N 21/255 : Details, e.g. use of specially adapted sources, lighting or optical systems
• CPC Class Codes: G06T 2207/10024 : Color image
• CPC Class Codes: G06T 2207/10028 : Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30036 : Dental; Teeth
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection
• CPC Class Codes: G06T 7/12 : Edge-based segmentation
• CPC Class Codes: G06T 7/90 : Determination of colour characteristics
• CPC Class Codes: H01L 27/14868 : CCD or CID colour imagers
• CPC Class Codes: H04N 13/15 : for colour aspects of image signals
• CPC Class Codes: H04N 13/207 : using a single 2D image sensor
• CPC Class Codes: H04N 13/257 : Colour aspects
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 : Synchronisation thereof; Control thereof

Abstract

A device for determining the surface topology and associated color of a structure, such as a teeth segment, includes a scanner for providing depth data for points along a two-dimensional array substantially orthogonal to the depth direction, and an image acquisition means for providing color data for each of the points of the array, while the spatial disposition of the device with respect to the structure is maintained substantially unchanged. A processor combines the color data and depth data for each point in the array, thereby providing a three-dimensional color virtual model of the surface of the structure. A corresponding method for determining the surface topology and associate color of a structure is also provided.

Claims

1. A system for determining surface topology and associated color of at least a portion of a three-dimensional structure, comprising:

a hand-held device comprising;

a scanning system configured to provide depth data of said portion, the depth data corresponding to a plurality of data points defined on a plane substantially orthogonal to a depth direction;

an imaging system configured to provide two-dimensional color image data of said portion associated with said plurality of data points;

an illumination unit configured to transmit a first array of incident light along a path towards the three-dimensional structure; and

a detector configured to measure intensity of each of a plurality of returned light that returns along the path and from the three-dimensional structure; and

a processor operably coupled to the hand-held device and configured to associate the depth data with the two-dimensional color image data.

2. The system according to claim 1, wherein the imaging system is configured to provide the tow-dimensional color image data independently from the depth data.

3. The system according to claim 1, wherein the plurality of data points is associated with a two-dimensional reference array.

4. The system according to claim 1, wherein the operation of the scanning system is based on confocal imaging techniques, and

wherein the scanning system is configured to:

cause the illumination system to transmit the first array of incident light along the path towards the three-dimensional structure;

cause the detector to measure intensity of each of the plurality of returned light that return along the path and from the three-dimensional structure.

5. The system according to claim 1, wherein the depth data and the two-dimensional color image data are associated by aligning the plurality of data points and the two-dimensional color image data in the same frame of reference.

6. The system according to claim 1, wherein the associated depth data and the two-dimensional color image data represent the color and surface topology of the portion of the three-dimensional structure.

7. The system according to claim 1, wherein the imaging system is configured to:

cause the illumination system to illuminate the portion with a set of three or more, differently-colored illumination radiations, the differently-colored illumination radiations being combinable to provide white light;

cause the detector to capture a monochromatic image of the portion for each of the three or more, differently-colored illumination radiations; and

combine each of the monochromatic images to create the two-dimensional color image data.

8. The system according to claim 7, comprising a color illumination system configured to provide the set of three or more, differently-colored illumination radiations.

9. The system according to claim 8, wherein the color illumination system comprises one of the following:

(a) a first illumination means for providing the set of three or more, differently-colored illumination radiations;

or

(b) a first illumination means for providing two, differently-colored illumination radiations, and a second illumination means for providing at least one, differently-colored illumination radiation.

10. The system according to claim 9, further comprising:

a probing member comprising a sensing face;

the illumination unit configured to transmit a first array of incident light beams towards the three-dimensional structure and through the probing member and the sensing face to generate illuminated spots on the portion of the structure;

an optical system configured to focus the incident light beams at each of a plurality of focal planes located between the sensing face and the structure; and

the detector configured to measure intensity of each of a plurality of returned light beams originating from the illuminated spots; and

wherein the processor is operably coupled to the detector and further configured to;

determine for each of the returned light beams a spot specific position, each spot specific position corresponding to a position of a respective focal plane in the plurality of focal planes, each respective focal plane corresponding to a maximum intensity of each respective returned light beam in the plurality of returned light beams; and

generate, based on the determined spot specific positions, data representative of a topology of the portion.

11. The system according to claim 10, wherein the first illumination means comprises radiation transmission elements located out of the path of the incident light beams or the returned light beams.

12. The system according to claim 10, wherein the probing member comprises a light transmissive material having an optical interface with light focusing optics in the optical system and a reflective face configured to reflect light between the optical interface and the sensing face.

32. The system accordingly to claim 1, wherein the two-dimensional color image data is captured in response to illumination of the portion by a first wavelength.

13. A system for determining surface topology and associated color of at least a portion of a three-dimensional dental structure, comprising:

a hand-held device comprising;

a scanning system configured to provide depth data of the portion, the depth data corresponding to a plurality of data points defined on a plane substantially orthogonal to a depth direction; and

an imaging system configured to provide color image data of the portion associated with said plurality of data points;

an illumination unit configured to transmit a first array of incident light along a path towards the three-dimensional structure; and

a detector configured to measure intensity of each of a plurality of returned light that return along the path and from the three-dimensional structure; and

a processor operably coupled with the hand-held device and configured to associate the depth data with the color image data, wherein the depth data and the color image data represent the surface topology and the color of the portion of the three-dimensional dental structure.

14. The system according to claim 13, wherein the imaging system is configured to provide the color image data independently from the depth data.

15. The system according to claim 13, wherein the plurality of data points is associated with a two-dimensional reference array.

16. The system according to claim 13, wherein the operation of the scanning system is based on confocal imaging techniques, and

wherein the scanning system is configured to:

cause the illumination system to transmit the first array of incident light along the path towards the three-dimensional structure;

cause the detector to measure intensity of each of the plurality of returned light that return along the path and from the three-dimensional structure.

17. The system according to claim 13, wherein the depth data and the color image data are associated by aligning the plurality of data points and the color image data in the same frame of reference.

18. The system according to claim 17, wherein the frame of reference comprises a two-dimensional reference array.

19. The system according to claim 13, wherein the depth data and the color image data are associated by using an alignment procedure comprising an optical character recognition technique.

20. The system according to claim 13, comprising a color illumination system having at least one illumination source configured to transmit light beams having different wavelengths in the system.

21. The system according to claim 13, wherein the system is configured to provide the depth data and the color image data based on the measured intensity of the light beams having different wavelengths.

22. The system according to claim 20, wherein the color illumination system comprises one of the following:

(a) a single illumination source configured to provide three or more, illumination beams having different wavelengths;

or

(b) a first illumination source configured to provide two illumination beams having different wavelengths, and a second illumination source configured to provide an illumination beam having a different wavelength than the wavelengths of the two illumination beams from the first illumination source.

23. The system according to claim 13, wherein the system comprises:

a probing member having a sensing face;

the illumination unit configured to transmit an array of incident light beams towards the three-dimensional dental structure and through the probing member and the sensing face to generate illuminated spots on the portion of the structure;

an optical system configured to focus the incident light beams at each of a plurality of focal planes located between the sensing face and the structure; and

the detector configured to measure intensity of each of a plurality of returned light beams originating from the illuminated spots; and

wherein the processor is operably coupled to the detector and is further configured to;

determine for each of the returned light beams a spot specific position, each spot specific position corresponding to a position of a respective focal plane in the plurality of focal planes, each respective focal plane corresponding to a maximum intensity of each respective returned light beam in the plurality of returned light beams; and

generate, based on the determined spot specific positions, data representing the surface topology of the portion of the three-dimensional dental structure.

30. The system according to claim 13, wherein the scanning system is configured to:

cause the illumination system to transmit the first array of incident light along the path towards the three-dimensional structure;

cause the detector to measure intensity of each of the plurality of returned light that return along the path and from the three-dimensional structure.

31. The method according to claim 17, wherein the scanning system is configured to:

cause the illumination system to transmit the first array of incident light along the path towards the three-dimensional structure;

cause the detector to measure intensity of each of the plurality of returned light that return along the path and from the three-dimensional structure.

33. The system accordingly to claim 13, wherein the color image data is captured in response to illumination of the portion by a first wavelength.

24. A method for determining surface topology and associated color of at least a portion of a three-dimensional dental structure, the method comprising:

providing a hand-held device comprising;

a scanning system configured to provide depth data of the portion, the depth data corresponding to a plurality of data points defined on a plane substantially orthogonal to a depth direction; and

an imaging system configured to provide color image data of the portion associated with said plurality of data points;

an illumination unit configured to transmit a first array of incident light along a path towards the three-dimensional structure; and

a detector configured to measure intensity of each of a plurality of returned light that return along the path and from the three-dimensional dental structure; and

a processor operably coupled to the hand-held device and configured to associate the depth data with the color image data, wherein the depth data and the color image data represent the surface topology and the color of the portion of the three-dimensional dental structure; and

operating the hand-held device.

25. The method according to claim 24, wherein the imaging system is configured to provide the color image data independently from the depth data.

26. The method according to claim 24, wherein the plurality of data points is associated with a two-dimensional reference array.

27. The method according to claim 24, wherein the operation of the scanning system is based on confocal imaging techniques, and

wherein the scanning system is configured to:

cause the illumination system to transmit the first array of incident light along the path towards the three-dimensional structure;

cause the detector to measure intensity of each of the plurality of returned light that return along the path and from the three-dimensional structure.

28. The method according to claim 24, wherein the depth data and the color image data are associated by aligning the plurality of data points and the color image data in the same frame of reference.

29. The method according to claim 28, wherein the frame of reference comprises a two-dimensional reference array.

34. The method accordingly to claim 24, wherein the color image data is captured in response to illumination of the portion by a first wavelength.