Methods and computer readable storage media for hyperspectral or multispectral imaging of multiple wavelengths

• US 9,107,624 B2
• Filed: 03/15/2013
• Issued: 08/18/2015
• Est. Priority Date: 06/05/2012
• Status: Active Grant

• Current Assignee: Hypermed Imaging Incorporated
• Original Assignee: Hypermed Imaging Incorporated
• Inventors: Darty, Mark Anthony
• Primary Examiner(s): LAURITZEN, AMANDA L
• Application Number:
  US14/372,711
• Publication Number:
  US 20150051498A1
• Time in Patent Office: 886 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61B 2090/309 using white LEDs
• CPC Class Codes: A61B 2090/3618 with a mirror
• CPC Class Codes: A61B 2560/0425 Ergonomically shaped housings A61B2560/0418 takes precedence
• CPC Class Codes: A61B 2562/0233 Special features of optical sensors or probes classified in A61B5/00
• CPC Class Codes: A61B 2562/028 Microscale sensors, e.g. electromechanical sensors [MEMS]
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/443 Evaluating skin constituents, e.g. elastin, melanin, water
• CPC Class Codes: A61B 5/447 specially adapted for aiding the prevention of ulcer or pressure sore development, i.e. before the ulcer or sore has developed
• CPC Class Codes: A61B 5/4869 Determining body composition
• CPC Class Codes: A61B 90/30 Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G01J 2003/1213 Filters in general, e.g. dichroic, band
• CPC Class Codes: G01J 2003/2826 Multispectral imaging, e.g. filter imaging
• CPC Class Codes: G01J 3/0208 using focussing or collimating elements, e.g. lenses or mirrors; performing aberration correction
• CPC Class Codes: G01J 3/021 using plane or convex mirrors, parallel phase plates, or particular reflectors
• CPC Class Codes: G01J 3/0224 using polarising or depolarising elements
• CPC Class Codes: G01J 3/0291 Housings; Spectrometer accessories; Spatial arrangement of elements, e.g. folded path arrangements
• CPC Class Codes: G01J 3/10 Arrangements of light sources specially adapted for spectrometry or colorimetry
• CPC Class Codes: G01J 3/12 Generating the spectrum; Monochromators
• CPC Class Codes: G01J 3/2803 using photoelectric array detector
• CPC Class Codes: G01J 3/2823 : Imaging spectrometer
• CPC Class Codes: G01J 3/36 : Investigating two or more bands of a spectrum by separate detectors
• CPC Class Codes: G01J 3/447 : Polarisation spectrometry
• CPC Class Codes: G01N 2021/4709 : Backscatter
• CPC Class Codes: G01N 21/255 : Details, e.g. use of specially adapted sources, lighting or optical systems
• CPC Class Codes: G01N 21/31 : Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry G01N21/72 takes precedence
• CPC Class Codes: G01N 21/47 : Scattering, i.e. diffuse reflection G01N21/25, G01N21/41 take precedence G01N21/55 takes precedence
• CPC Class Codes: G06F 18/2135 : based on approximation criteria, e.g. principal component analysis
• CPC Class Codes: G06T 2207/10036 : Multispectral image; Hyperspectral image
• CPC Class Codes: G06T 2207/30064 : Lung nodule
• CPC Class Codes: G16H 50/50 : for simulation or modelling of medical disorders

Abstract

A hyperspectral/multispectral imager comprising a housing is provided. At least one light source is attached to the housing. An objective lens, in an optical communication path comprising originating and terminating ends, is further attached to the housing and causes light to (i) be backscattered by the tissue of a subject at the originating end and then (ii) pass through the objective lens to a beam steering element at the terminating end of the communication path inside the housing. The beam steering element has a plurality of operating modes each of which causes the element to be in optical communication with a different optical detector in a plurality of optical detectors offset from the optical communication path. Each respective detector filter in a plurality of detector filters covers a corresponding optical detector in the plurality of optical detectors thereby filtering light received by the corresponding detector from the beam steering element.

Claims

1. A method for directing one of a hyperspectral and a multispectral imager to acquire one of a hyperspectral and a multispectral image of a tissue of a subject, each of the hyperspectral and multispectral images comprising a plurality of sub-images of the tissue of the subject, each respective sub-image in the plurality of sub-images acquired at a corresponding wavelength or wavelength band in a plurality of wavelengths or wavelength bands, the one of the hyperspectral and multispectral imager comprising:

i.) a housing having an exterior and an interior;

ii.) at least one incoherent light source disposed on the exterior of the housing;

iii.) at least one objective lens attached to or within the housing, the at least one objective lens disposed in an optical communication path, the optical communication path comprising an originating end and a terminating end, wherein light from the at least one incoherent light source is (i) first backscattered by a tissue of a subject and (ii) then passed from the originating end of the optical communication path, through the at least one objective lens, and to the terminating end of the optical communication path;

iv.) a beam steering element within the interior of the housing, the beam steering element in optical communication with the terminating end of the optical communication path, the beam steering element characterized by a plurality of operating modes, the beam steering element comprising a minor or an array of mirrors;

v.) a plurality of optical detectors offset from the optical communication path, each respective optical detector in the plurality of optical detectors in optical communication with a corresponding operating mode of the beam steering element; and

vi.) a plurality of detector filters within the housing, each respective detector filter in the plurality of detector filters covering a corresponding optical detector in the plurality of optical detectors thereby filtering light received by the corresponding optical detector from the beam steering element so that each optical detector in the plurality of optical detectors is configured for recording a respective sub-image in the plurality of sub-images at the wavelength or wavelength band of the respective sub-image, the method comprising;

a.) identifying a plurality of baseline exposure times, each respective baseline exposure time in the plurality of baseline exposure times representing an exposure time for resolving a respective sub-image, in the plurality of sub-images, of the tissue of the subject at the wavelength or wavelength band of the respective sub-image, wherein a first baseline exposure time for a first sub-image is different than a second baseline exposure time of a second sub-image in the plurality of sub-images; and

b.) cycling the beam steering element through the plurality of operating modes, wherein the beam steering element is retained in each respective operating mode for the baseline exposure time corresponding to the wavelength or wavelength band collected by the detector filter corresponding to the respective operating mode so that a sub-image is recorded on the optical detector corresponding to the respective operating mode, thereby collecting the hyperspectral or multispectral image of the tissue.

2. The method of claim 1, wherein each optical detector in the plurality of optical detectors is configured to detect a different frequency band of radiation.

3. The method of claim 1, wherein the at least one incoherent light source disposed on the exterior of the housing is a broadband light source.

4. The method of claim 3, wherein the broadband light source is a white light-emitting diode.

5. The method of claim 3, further comprising a bandpass filter covering the broadband light source.

6. The method of claim 1, wherein the at least one incoherent light source disposed on the exterior of the housing comprises a first incoherent light source configured to emit radiation of a first spectral bandwidth and a second incoherent light source configured to emit radiation of a second spectral bandwidth, wherein said first and second spectral bandwidths are different.

7. The method of claim 6, wherein (b) cycling the beam steering element through the plurality of operating modes comprises:

1) turning on the first incoherent light source;

2) while the first incoherent light source is on;

placing the beam steering element in a first operating mode in the plurality of operating modes that causes the beam steering element to be in optical communication with a first optical detector in the plurality of optical detectors, the first optical detector configured to detect light having a wavelength in the first spectral bandwidth;

3) turning off the first incoherent light source;

4) turning on the second incoherent light source; and

5) while the second incoherent light source is on;

placing the beam steering element in a second operating mode in the plurality of operating modes that causes the beam steering element to be in optical communication with a second optical detector in the plurality of optical detectors, the second optical detector configured to detect light having a wavelength in the second spectral bandwidth.

8. The method of claim 1, wherein the beam steering element within the interior of the housing comprises the mirror, and wherein the mirror is mounted on an actuator, the actuator having the plurality of operating modes.

9. The method of claim 8, wherein the mirror is a single-surface mirror.

10. The method of claim 8, wherein the mirror is a two-axis micro electro-mechanical (MEMS) mirror.

11. The method of claim 1, wherein the beam steering element within the interior of the housing comprises the array of mirrors, wherein the array of mirrors is an array of micromirrors.

12. The method of claim 11, wherein the array of micromirrors is mounted on an actuator, the actuator having the plurality of operating modes.

13. The method of claim 1, wherein at least one optical detector in the plurality of optical detectors is not covered by a detector filter in the plurality of detector filters.

14. The method of claim 13, wherein the at least one optical detector that is not covered by a detector filter is configured for focusing an image of said surface of the subject acquired by at least one respective optical detector in said plurality of optical detectors.

15. A non-transitory computer readable storage medium storing one or more programs executable by one of a hyperspectral and multispectral imaging device, with a central processing unit configured to execute the one or more programs and an optical acquisition camera configured to acquire one of a hyperspectral and a multispectral image of a tissue of a subject, the one of the hyperspectral and multispectral imaging device comprising:

i.) a housing having an exterior and an interior;

ii.) at least one incoherent light source disposed on the exterior of the housing;

iii.) at least one objective lens attached to or within the housing, the at least one objective lens disposed in an optical communication path, the optical communication path comprising an originating end and a terminating end, wherein light from the at least one incoherent light source is (i) first backscattered by a tissue of a subject and (ii) then passed from the originating end of the optical communication path, through the at least one objective lens, and to the terminating end of the optical communication path;

iv.) a beam steering element within the interior of the housing, the beam steering element in optical communication with the terminating end of the optical communication path, the beam steering element characterized by a plurality of operating modes, the beam steering element comprising a minor or an array of minors;

v.) a plurality of optical detectors offset from the optical communication path, each respective optical detector in the plurality of optical detectors in optical communication with a corresponding operating mode of the beam steering element; and

vi.) a plurality of detector filters within the housing, each respective detector filter in the plurality of detector filters covering a corresponding optical detector in the plurality of optical detectors thereby filtering light received by the corresponding optical detector from the beam steering element so that each optical detector in the plurality of optical detectors is configured for recording a respective sub-image in a plurality of sub-images at a wavelength or wavelength band of the respective sub-image, the one or more programs comprising instructions for

a.) identifying a plurality of baseline exposure times, each respective baseline exposure time in the plurality of baseline exposure times representing an exposure time for resolving the respective sub-image, in the plurality of sub-images, of the tissue of the subject at the wavelength or wavelength band of the respective sub-image, wherein a first baseline exposure time for a first sub-image is different than a second baseline exposure time of a second sub-image in the plurality of sub-images; and

b.) cycling the beam steering element through the plurality of operating modes, wherein the beam steering element is retained in each respective operating mode for the baseline exposure time corresponding to the wavelength or wavelength band collected by the detector filter corresponding to the respective operating mode so that a sub-image is recorded on the optical detector corresponding to the respective operating mode, thereby collecting the hyperspectral or multispectral image of the tissue.

16. The non-transitory computer readable storage medium of claim 15, wherein each optical detector in the plurality of optical detectors is configured to detect a different frequency band of radiation.

17. The non-transitory computer readable storage medium of claim 15, wherein the respective baseline exposure time for the respective sub-image in the plurality of sub-images is determined by:

(1) a factor affecting a baseline illumination of the tissue of the subject; and

(2) a sensitivity of a corresponding optical detector in the plurality of optical detectors in the optical acquisition camera used to acquire the sub-image at the wavelength or wavelength band corresponding to the sub-image.

18. The non-transitory computer readable storage medium of claim 17, wherein the factor is (i) an amount of illumination of the tissue of the subject provided by an illumination subsystem comprising lights mounted on a printed circuit board of one of the hyperspectral and multispectral imaging device, (ii) an amount of ambient light, or (iii) a concentration of melanin in the tissue of the subject.

19. The non-transitory computer readable storage medium of claim 15, wherein the at least one incoherent light source disposed on the exterior of the housing is a broadband light source.

20. The non-transitory computer readable storage medium of claim 19, wherein the broadband light source is a white light-emitting diode.

21. The non-transitory computer readable storage medium of claim 19, further comprising a bandpass filter covering the broadband light source.

22. The non-transitory computer readable storage medium of claim 15, wherein the at least one incoherent light source disposed on the exterior of the housing comprises a first incoherent light source configured to emit radiation of a first spectral bandwidth and a second incoherent light source configured to emit radiation of a second spectral bandwidth, wherein said first and second spectral bandwidths are different.

23. The non-transitory computer readable storage medium of claim 22, wherein (b) cycling the beam steering element through the plurality of operating modes comprises:

1) turning on the first incoherent light source;

2) while the first incoherent light source is on;

placing the beam steering element in a first operating mode in the plurality of operating modes that causes the beam steering element to be in optical communication with a first optical detector in the plurality of optical detectors, the first optical detector configured to detect light having a wavelength in the first spectral bandwidth;

3) turning off the first incoherent light source;

4) turning on the second incoherent light source; and

5) while the second incoherent light source is on;

placing the beam steering element in a second operating mode in the plurality of operating modes that causes the beam steering element to be in optical communication with a second optical detector in the plurality of optical detectors, the second optical detector configured to detect light having a wavelength in the second spectral bandwidth.

24. The non-transitory computer readable storage medium of claim 15, wherein the beam steering element within the interior of the housing comprises the mirror, and wherein the mirror is mounted on an actuator, the actuator having the plurality of operating modes.

25. The non-transitory computer readable storage medium of claim 24, wherein the mirror is a single-surface mirror.

26. The non-transitory computer readable storage medium of claim 24, wherein the mirror is a two-axis micro electro-mechanical (MEMS) mirror.

27. The non-transitory computer readable storage medium of claim 15, wherein the beam steering element within the interior of the housing comprises the array of minors, wherein the array of mirrors is an array of micromirrors.

28. The non-transitory computer readable storage medium of claim 27, wherein the array of micromirrors is mounted on an actuator, the actuator having the plurality of operating modes.

29. The non-transitory computer readable storage medium of claim 15, wherein at least one optical detector in the plurality of optical detectors is not covered by a detector filter in the plurality of detector filters.

30. The non-transitory computer readable storage medium of claim 29, wherein the at least one optical detector that is not covered by a detector filter is configured for focusing an image of said surface of the subject acquired by at least one respective optical detector in said plurality of optical detectors.