Method and apparatus for structure imaging a three-dimensional structure

• US 10,750,152 B2
• Filed: 04/03/2020
• Issued: 08/18/2020
• Est. Priority Date: 06/17/2004
• Status: Active Grant

• Current Assignee: Align Technology Incorporated
• Original Assignee: Align Technology Incorporated
• Inventors: Babayoff, Noam
• Primary Examiner(s): Volentine, Rebecca A
• Application Number:
  US16/840,267
• Publication Number:
  US 20200236336A1
• Time in Patent Office: 137 Days
• Field of Search: None
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/00096 Optical elements
• CPC Class Codes: A61B 1/0615 for radial illumination
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/0646 with illumination filters
• CPC Class Codes: A61B 1/0676 at distal tip of an endoscope
• CPC Class Codes: A61B 1/0684 using light emitting diodes [LED]
• CPC Class Codes: A61B 1/24 for the mouth, i.e. stomatoscopes, e.g. with tongue depressors tongue depressors per se A61B13/00; Instruments for opening or keeping open the mouth combined with saliva removers A61C17/00; mouth openers for animals A61D15/00
• CPC Class Codes: A61B 1/247 with means for viewing areas outside the direct line of sight, e.g. dentists' mirrors
• CPC Class Codes: A61B 5/0068 Confocal scanning
• CPC Class Codes: A61B 5/0088 for oral or dental tissue
• CPC Class Codes: A61B 5/1077 Measuring of profiles
• CPC Class Codes: A61B 5/1079 using optical or photographic means
• CPC Class Codes: A61C 19/04 Measuring instruments specially adapted for dentistry A61C5/44 takes precedence; testing vitality of teeth A61B5/0534; radiation diagnosis specially adapted for dentistry A61B6/51
• CPC Class Codes: A61C 9/0053 Optical means or methods, e.g. scanning the teeth by a laser or light beam
• CPC Class Codes: A61C 9/0066 Depth determination through adaptive focusing
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01J 3/02 Details
• CPC Class Codes: G01J 3/0205 Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
• CPC Class Codes: G01J 3/0208 : using focussing or collimating elements, e.g. lenses or mirrors; performing aberration correction
• CPC Class Codes: G01J 3/0216 : using light concentrators or collectors or condensers
• CPC Class Codes: G01J 3/0218 : using optical fibers
• CPC Class Codes: G01J 3/0224 : using polarising or depolarising elements
• CPC Class Codes: G01J 3/0243 : having a through-hole enabling the optical element to fulfil an additional optical function, e.g. a mirror or grating having a throughhole for a light collecting or light injecting optical fiber
• CPC Class Codes: G01J 3/0256 : Compact construction
• CPC Class Codes: G01J 3/10 : Arrangements of light sources specially adapted for spectrometry or colorimetry
• CPC Class Codes: G01J 3/462 : Computing operations in or between colour spaces; Colour management systems
• CPC Class Codes: G01J 3/50 : using electric radiation detectors
• CPC Class Codes: G01J 3/501 : Colorimeters using spectrally-selective light sources, e.g. LEDs
• CPC Class Codes: G01J 3/508 : measuring the colour of teeth
• CPC Class Codes: G01J 3/51 : using colour filters
• CPC Class Codes: G01N 21/255 : Details, e.g. use of specially adapted sources, lighting or optical systems
• CPC Class Codes: G06T 2207/10024 : Color image
• CPC Class Codes: G06T 2207/10028 : Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30036 : Dental; Teeth
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection
• CPC Class Codes: G06T 7/12 : Edge-based segmentation
• CPC Class Codes: G06T 7/90 : Determination of colour characteristics
• CPC Class Codes: H01L 27/14868 : CCD or CID colour imagers
• CPC Class Codes: H04N 13/15 : for colour aspects of image signals
• CPC Class Codes: H04N 13/207 : using a single 2D image sensor
• CPC Class Codes: H04N 13/257 : Colour aspects
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 : Synchronisation thereof; Control thereof

Abstract

A system for determining the surface topology and associated color of at least a portion of a three-dimensional structure includes a hand-held device. A scanning system may be configured to provide depth data of the portion. An imaging system may be configured to provide two-dimensional color image data of the portion associated with the plurality of data points. A processor may be operably coupled to the hand-held device and configured to associate the depth data with the color image data.

Claims

1. A system for generating a 3D model of an intraoral structure portion, the system comprising:

a hand-held imaging device comprising a probe, focusing optics configured to scan a focal plane of the hand-held imaging device over a range of depths, an illuminator, and an image sensor configured to capture image data over the range of depths of the intraoral structure portion, wherein light from the illuminator passes through the focusing optics to the intraoral structure portion and light from the intraoral structure portion passes through the focusing optics to the image sensor and wherein the image data includes first image data captured in response to illuminating the intraoral structure portion with light from the illuminator; and

one or more processors operably coupled to the hand-held imaging device, the one or more processors configured to cause the system to;

generate depth data of the intraoral structure portion using the first image data from the hand-held imaging device;

generate color data of the intraoral structure portion using the first image data from the hand-held imaging device; and

provide, using one or more processors, a three-dimensional numerical entity based on the depth data and the color data.

2. The system of claim 1, wherein the first image data is captured over the range of focal lengths.

3. The system of claim 1, wherein the illuminator generates light of a first wavelength.

4. The system of claim 3, wherein the illuminator that generates light of the first wavelength comprises a first illuminator and wherein the system further comprises a second illuminator that generates light of at least a second wavelength.

5. The system of claim 4, wherein the image sensor captures second image data in response to illuminating the intraoral structure portion with light from the second illuminator, and

wherein the one or more processors are further configured to cause the system to:

generate the color data of the intraoral structure portion using the first image data and the second image data from the hand-held imaging device.

6. The system of claim 4, wherein the second illuminator generates light of the second wavelength and a third wavelength.

7. The system of claim 1, wherein the depth data comprises a 3D surface model of the intraoral structure portion.

8. The system of claim 6, wherein the first wavelength is red, the second wavelength is green, and the third wavelengths is blue.

9. A method for determining surface characteristics of an intraoral structure, the method comprising:

illuminating the intraoral structure with light of a light source passing through focusing optics and light of a second light source;

capturing depth image data and color image data of returning light from the intraoral structure over a range of focal lengths during the illumination with light of the light source and the light of the second light source, the returning light passing through the focusing optics;

generating depth data of the intraoral structure using the depth image data; and

generating color data of the intraoral structure using the color image data; and

combining the depth data with the color data to create a three-dimensional numerical entity.

10. The method of claim 9, wherein the light source generates light of a first wavelength.

11. The method of claim 10, wherein the light source is a first light source and further comprises illuminating the intraoral structure with light of a second wavelength from a second light source.

12. The method of claim 11, wherein the color data is first color data and further comprising:

capturing second color data in response to illuminating the intraoral structure with light from the second light source, and wherein combining the depth data with the first color data to create the three-dimensional numerical entity further comprises combining the second color data with the depth data and the first color data to generate the three-dimensional numerical entity.

13. The method of claim 12, wherein the second light source generates light of the second wavelength and a third wavelength.

14. The method of claim 9, wherein the depth data comprises a 3D surface model of the intraoral structure.

15. The method of claim 13, wherein the first wavelength is red, the second wavelength is green, and the third wavelengths is blue.

16. A method for determining surface topology and associated color of an intraoral structure, the method comprising:

illuminating the intraoral structure with light from a light source passing though focusing optics;

capturing image data of the intraoral structure from light returned passing through the focusing optics from the intraoral structure in response to the illumination of the intraoral structure with the light source using an image sensor of an imaging device, wherein the capturing includes scanning a focal plane of the imaging device over a range of depths;

generating, using the imaging device and the image data, depth data of the intraoral structure;

generating, using the imaging device and the image data, color data of the intraoral structure; and

providing a color three-dimensional numerical entity based on the depth data and the color data.

17. The method of claim 16, further comprising illuminating the intraoral structure with light of a first wavelength and a second wavelength.

18. The method of claim 17, wherein the image data comprises light captured in response to illuminating the intraoral structure with light of the first wavelength and in response to illuminating the intraoral structure with light of the second wavelength, and wherein the color three-dimensional numerical entity is also based on the color data.

19. The method of claim 18, wherein the light source comprises a first light source and a second light source and illuminating the intraoral structure with light of the first wavelength includes illuminating the intraoral structure with light of the first wavelength from the first light source and illuminating the intraoral structure with light of the second wavelength includes illuminating the intraoral structure with light of the second wavelength from the second light source.

20. The method of claim 16, wherein the depth data comprises a 3D surface model of the intraoral structure.

21. The method of claim 18, wherein the first wavelength is red and the second wavelength is green or blue.

22. The method of claim 20, wherein the image sensor is a monochrome image sensor.

23. A system for determining surface topology and associated color of an intraoral structure, the system comprising:

a hand-held imaging device comprising a probe, focusing optics configured to scan a focal plane of the hand-held imaging device over a range of depths, an illuminator configured to illuminate the intraoral structure through the focusing optics, and an image sensor configured to capture image data of the intraoral structure over the range of depths from light returned through the focusing optics; and

one or more processors operably coupled to the hand-held imaging device, the one or more processors configured to cause the system to;

generate depth data of the intraoral structure using the image data from the hand-held imaging device,

generate color data of the intraoral structure using the image data from the hand-held imaging device, and

provide a color three-dimensional numerical entity comprising the depth data and the color data associated with the intraoral structure, wherein the image data used for generating the depth data is also used for the color data of the color three-dimensional numerical entity.

24. The system of claim 23, where the illuminator is a first illuminator and further comprises a second illuminator configured to illuminate the intraoral structure with light.

25. The system of claim 24, wherein the image sensor is configured to capture the color data in response to illuminating the intraoral structure with light of the second illuminator, and wherein the color three-dimensional numerical entity is also based on the color data.

26. The system of claim 23, wherein a time interval for capturing the image data over the range of depths is within an interval between about 0 to about 100 milliseconds.

27. The system of claim 23, wherein the depth data comprises a 3D surface model of the intraoral structure.

28. The system of claim 24, wherein the first illuminator illuminates with red light and the second illuminator illuminates with green or blue light.

29. The system of claim 23, wherein the image sensor is a monochrome image sensor.