Single-sensor hyperspectral imaging device

• US 10,534,116 B2
• Filed: 06/07/2018
• Issued: 01/14/2020
• Est. Priority Date: 06/05/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Hypermed Imaging Incorporated
• Inventors: Darty, Mark
• Primary Examiner(s): Giles, Nicholas G
• Application Number:
  US16/003,036
• Publication Number:
  US 20180275325A1
• Time in Patent Office: 586 Days
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/14546 for measuring analytes not otherwise provided for, e.g. ions, cytochromes
• CPC Class Codes: A61B 5/14552 Details of sensors specially adapted therefor
• CPC Class Codes: A61B 5/6889 Rooms
• CPC Class Codes: A61B 5/743 Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots
• CPC Class Codes: G01J 2003/2806 Array and filter array
• CPC Class Codes: G01J 2003/2826 Multispectral imaging, e.g. filter imaging
• CPC Class Codes: G01J 3/0264 Electrical interface; User interface
• CPC Class Codes: G01J 3/2803 using photoelectric array detector
• CPC Class Codes: G01J 3/2823 Imaging spectrometer
• CPC Class Codes: G01J 3/36 Investigating two or more bands of a spectrum by separate detectors
• CPC Class Codes: G02B 5/20 Filters polarising elements G02B5/30
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/75 by influencing optical camera components

Abstract

The present disclosure generally relates to hyperspectral spectroscopy, and in particular, to systems, methods and devices enabling a single-sensor hyperspectral imaging device. Hyperspectral (also known as “multispectral”) spectroscopy is an imaging technique that integrates multiples images of an object resolved at different narrow spectral bands (i.e., narrow ranges of wavelengths) into a single data structure, referred to as a three-dimensional hyperspectral data cube. Data provided by hyperspectral spectroscopy allow for the identification of individual components of a complex composition through the recognition of spectral signatures of individual components within the three-dimensional hyperspectral data cube.

Claims

1. A hyperspectral imaging device comprising:

a photo-sensor array including a plurality of photo-sensors, each photo-sensor in the plurality of photo-sensors providing a respective output;

a spectral filter array having a plurality of filter elements, wherein each filter in the filter elements is arranged to filter light received by a respective one or more of the plurality of photo-sensors, and wherein each filter element is one of a plurality of filter-types, wherein each filter-type is characterized by a spectral pass-band different from the other filter-types, wherein the filter elements of at least one discrete filter-type in the plurality of filter-types are distributed across the spectral filter array in a non-uniform distribution;

one or more processors; and

memory including instructions, the instructions, when executed by the one or more processors, cause the hyperspectral imaging device to perform operations comprising;

selecting one or more sub-sets of photo-sensor outputs, wherein each sub-set of photo-sensor outputs is associated with a single respective filter-type; and

forming a hyperspectral data cube from the one or more sub-sets of photo-sensor outputs by generating a plurality of respective images, wherein each respective image is produced from a single respective sub-set of photo-sensor outputs so that each of the plurality of respective images is associated with a particular filter-type.

2. The hyperspectral imaging device of claim 1, wherein the instructions, when executed by the one or more processors, cause the hyperspectral imaging device to:

capture single frame image data by controlling exposure of the photo-sensor array and spectral filter array to light, wherein the hyperspectral data cube is generated from the single frame image data.

3. The hyperspectral image device of claim 2, wherein each of the plurality of images is generated by applying an interpolation process to the respective sub-set of photo-sensor outputs for the one respective filter-type.

4. The hyperspectral imaging device of claim 1, wherein a center-to-center distance between two filter elements of the same type is less than 250 microns.

5. The hyperspectral imaging device of claim 1, wherein a center-to-center distance between two filter elements of the same type is less than 150 microns.

6. The hyperspectral imaging device of claim 1, wherein the filter elements of at least one particular filter-type are spatially distributed across throughout the spectral filter array.

7. The hyperspectral imaging device of claim 6, wherein the at least one particular filter-type is different than the at least one discrete filter-type, and the spatial distribution of the filter elements of the at least one particular filter-type is characterized by a substantially uniform distribution of the filter elements throughout the spectral filter array.

8. The hyperspectral imaging device of claim 1, wherein a spatial distribution of the filter elements is characterized by a repeating pattern of one or more filter-types.

9. The hyperspectral imaging device of claim 1, wherein the plurality of filter-types includes at least three filter-types.

10. The hyperspectral imaging device of claim 1, further comprising circuitry configured to select the one or more sub-sets of photo-sensor outputs.

11. The hyperspectral imaging device of claim 1, wherein the instructions, when executed by the one or more processors, cause the hyperspectral imaging device to:

receive the output of the photo-sensor array at one or more registers;

identify which registers correspond to filter elements of a particular filter-type from a look-up table; and

select one or more sub-sets of photo-sensor outputs from the one or more registers based on the identification of the registers that correspond to filter elements of the particular filter-type.

12. The hyperspectral imaging device of claim 11, wherein the instructions, when executed by the one or more processors, cause the hyperspectral imaging device to:

bundle photo-sensor outputs from the particular filter-type into data packets, wherein the data packets include at least the corresponding register values.

13. The hyperspectral imaging device of claim 12, further comprising a transceiver to transmit the data packets to a server and receive an image for each filter-type from the server based on the transmitted data packets.

14. The hyperspectral imaging device of claim 1, comprising a handheld medical imaging device.

15. The hyperspectral imaging device of claim 1, further comprising a display screen.

16. The hyperspectral imaging device of claim 15, wherein the instructions, when executed by the one or more processors, cause the hyperspectral imaging device to:

display a hyperspectral image formed from the hyperspectral data cube on the display screen.

17. The hyperspectral imaging device of claim 16, wherein the instructions, when executed by the one or more processors, cause the hyperspectral imaging device to:

display information about a region of interest imaged by the plurality of respective images, the information selected from the group consisting of a probability that the region of interest is afflicted by a particular medical condition, a category of medical condition that the region of interest is afflicted with, a probable age of a medical condition that the region of interest is afflicted with, a boundary of a medical condition that the region of interest is afflicted with, or information about treatment of a medical condition that the region of interest is afflicted with.

18. The hyperspectral imaging device of claim 15, wherein the display is a touchscreen display that is used to manipulate a displayed image or to control the hyperspectral imaging device.

19. The hyperspectral imaging device of claim 1, further comprising a communication interface for communicating the plurality of respective images to a remote device across a communication network.

20. The hyperspectral imaging device of claim 19, wherein the communication interface uses a communication protocol selected from the group consisting of cellular, WiFi, ZigBee, BlueTooth, IEEE 802.11b, 802.11a, 802.11g, and 802.11n to communicate the plurality of images.