Single-sensor hyperspectral imaging device

• US 10,018,758 B2
• Filed: 08/18/2017
• Issued: 07/10/2018
• Est. Priority Date: 06/05/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Hypermed Imaging Incorporated
• Inventors: Darty, Mark
• Primary Examiner(s): Giles, Nicholas G
• Application Number:
  US15/681,265
• Publication Number:
  US 20170371079A1
• Time in Patent Office: 326 Days
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/14546 for measuring analytes not otherwise provided for, e.g. ions, cytochromes
• CPC Class Codes: A61B 5/14552 Details of sensors specially adapted therefor
• CPC Class Codes: A61B 5/6889 Rooms
• CPC Class Codes: A61B 5/743 Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots
• CPC Class Codes: G01J 2003/2806 Array and filter array
• CPC Class Codes: G01J 2003/2826 Multispectral imaging, e.g. filter imaging
• CPC Class Codes: G01J 3/0264 Electrical interface; User interface
• CPC Class Codes: G01J 3/2803 using photoelectric array detector
• CPC Class Codes: G01J 3/2823 Imaging spectrometer
• CPC Class Codes: G01J 3/36 Investigating two or more bands of a spectrum by separate detectors
• CPC Class Codes: G02B 5/20 Filters polarising elements G02B5/30
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/75 by influencing optical camera components

Abstract

The present disclosure generally relates to hyperspectral spectroscopy, and in particular, to systems, methods and devices enabling a single-sensor hyperspectral imaging device. Hyperspectral (also known as “multispectral”) spectroscopy is an imaging technique that integrates multiples images of an object resolved at different narrow spectral bands (i.e., narrow ranges of wavelengths) into a single data structure, referred to as a three-dimensional hyperspectral data cube. Data provided by hyperspectral spectroscopy allow for the identification of individual components of a complex composition through the recognition of spectral signatures of individual components within the three-dimensional hyperspectral data cube.

Claims

1. A computer implemented method, comprising:

at a hyperspectral imaging device comprising (i) a photo-sensor array including a plurality of photo-sensors, a spectral filter array having a plurality of filter elements, wherein each respective filter in the plurality of filter elements is arranged to filter light received by a corresponding one or more of the plurality of photo-sensors, each respective filter element in the plurality of filter elements is one of a plurality of filter-types, and each respective filter-type of the plurality of filter-types is characterized by a corresponding single spectral pass-band, wherein each of the single spectral pass-band of each respective filter-type is a different single spectral pass-band, (ii) an interface module, and (iii) a control module;

(A) exposing the photo-sensor array to a common exposure of light to concurrently generate a plurality of photo-sensor outputs, wherein each photo-sensor output in the plurality of photo-sensor outputs corresponds to a respective photo-sensor in the plurality of photo-sensors,

(B) selecting, using the interface module, a plurality of sub-sets of photo-sensor outputs from the plurality of photo-sensor outputs, wherein each sub-set of photo-sensor outputs in the plurality of photo-sensor outputs corresponds to a respective filter-type in the plurality of filter-types;

(C) generating, using the control module, a plurality of images for a hyperspectral data cube, wherein each respective image in the plurality of images is generated from a corresponding sub-set of photo-sensor outputs in the plurality of sub-sets of photo-sensor outputs; and

(D) assembling a hyperspectral data cube comprising a plurality of planes, wherein each plane in the plurality of planes corresponds to a respective image in the plurality of images.

2. The computer implemented method of claim 1, wherein the method further comprises assembling a hyperspectral image based on the hyperspectral data cube.

3. The computer implemented method of claim 1, wherein each of the plurality of images is generated by applying an interpolation process to the respective sub-set of photo-sensor outputs for the respective filter-type.

4. The computer implemented method of claim 1, wherein the center-to-center distance between two filter elements of the same type in the plurality of filter elements is less than 250 microns.

5. The computer implemented method of claim 1, wherein the center-to-center distance between two filter elements of the same type in the plurality of filter elements is less than 150 microns.

6. The computer implemented method of claim 1, wherein the filter elements of at least one particular filter-type in the plurality of filter elements are spatially distributed throughout the spectral filter array.

7. The computer implemented method of claim 6, wherein the spatial distribution of the filter elements of the at least one particular filter-type in the plurality of filter elements is characterized by a substantially uniform distribution of the filter elements throughout the spectral filter array.

8. The computer implemented method of claim 1, wherein a spatial distribution of the plurality of filter elements is characterized by a repeating pattern of one or more filter-types.

9. The computer implemented method of claim 1, wherein the plurality of filter-types includes at least three filter-types.

10. The computer implemented method of claim 1, wherein one or more sub-sets of photo-sensor outputs from the plurality of sub-sets of photo-sensor outputs is selected by circuitry within the interface module.

11. The computer implemented method of claim 1, wherein the interface module comprises one or more registers configured to receive the plurality of photo-sensor outputs, and wherein the selecting, using the interface module, a plurality of sub-sets of photo-sensor outputs includes:

identifying which registers correspond to filter elements of a particular filter-type from a look-up table; and

selecting one or more sub-sets of photo-sensor outputs from the one or more registers based on the identification of the registers that correspond to filter elements of the particular filter-type.

12. The computer implemented method of claim 11, the method further comprising bundling photo-sensor outputs from the particular filter-type into data packets, wherein the data packets each include at least a corresponding register value.

13. The computer implemented method of claim 12, wherein the hyperspectral imaging device further comprises a transceiver and a receiver, the method further comprising:

transmitting, using the transceiver, the data packets to a server, and

receiving, using the receiver, an image for each filter-type from the server based on the transmitted data packets.

14. The computer implemented method of claim 1, wherein the hyperspectral imaging device comprises a hand-held medical imaging device.

15. The computer implemented method of claim 2, wherein the hyperspectral imaging device further comprises a display control module and a display.

16. The computer implemented method of claim 15, the method further comprising displaying the assembled hyperspectral image on the display.

17. The computer implemented method of claim 16, wherein the common exposure of light is from a region of interest on a subject, the method further comprising:

displaying a probability that the region of interest on the subject has a particular medical condition, a category of condition, a probable age of the condition, a boundary of the condition, or information about treatment of a condition.

18. The computer implemented method of claim 16, wherein the display is a touchscreen display configured to manipulate a displayed image or to control the hyperspectral imaging device.

19. The computer implemented method of claim 1, wherein hyperspectral imaging device further comprises a communication interface, the method further comprising:

communicating the plurality of respective images to a remote device across a communication network.

20. The computer implemented method of claim 19, wherein the communication interface uses a communication protocol selected from the group consisting of cellular, WiFi, ZigBee, BlueTooth, IEEE 802.11b, 802.11a, 802.11g, and 802.11n to communicate the plurality of images.