Method and apparatus for imaging unsectioned tissue specimens

• US 10,416,434 B2
• Filed: 01/26/2018
• Issued: 09/17/2019
• Est. Priority Date: 02/12/2016
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Fujimoto, James G., Giacomelli, Michael G., Yoshitake, Tadayuki, Cahill, Lucas C.
• Primary Examiner(s): Le, Peter D
• Application Number:
  US15/881,224
• Publication Number:
  US 20180259762A1
• Time in Patent Office: 599 Days
• CPC Class Codes: G01N 2021/0106 General arrangement of respective parts
• CPC Class Codes: G01N 21/01 Arrangements or apparatus for facilitating the optical investigation
• CPC Class Codes: G01N 21/6408 with measurement of decay time, time resolved fluorescence
• CPC Class Codes: G01N 21/6428 Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes" in vivo A61B5/00; immunoassay G01N33/53
• CPC Class Codes: G01N 21/6458 Fluorescence microscopy fluorescence microscopes per se G02B21/0076 and G02B21/16
• CPC Class Codes: G01N 21/6486 Measuring fluorescence of biological material, e.g. DNA, RNA, cells G01N21/6428 takes precedence
• CPC Class Codes: G02B 21/0088 Inverse microscopes
• CPC Class Codes: G02B 21/086 for transillumination only
• CPC Class Codes: G02B 21/16 adapted for ultraviolet illumination ; Fluorescence microscopes G02B21/0076 takes precedence
• CPC Class Codes: G02B 21/26 Stages; Adjusting means therefor
• CPC Class Codes: G02B 21/361 Optical details, e.g. image relay to the camera or image sensor G02B21/364 takes precedence; illumination details G02B21/06 and subgroups
• CPC Class Codes: G02B 21/367 providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison

Abstract

An apparatus and method for real-time optical imaging of a tissue specimen. The apparatus comprises a primary imaging system configured to use an illumination source to acquire images of a tissue specimen through one or more spectrally separated channels, and configured to perform optical depth sectioning; an auxiliary imaging system, configured to acquire an auxiliary image of the tissue specimen; a specimen holder having a transparent window therewithin, window, the specimen holder comprising one or more position sensors, wherein the specimen holder is configured to be translatable in the specimen plane; a user input device configured to accept user input, wherein the specimen holder is configured to translate in response to the user input in real-time; a processing unit configured to execute a sequence of instructions on the sequence of images acquired by the primary imaging system, the auxiliary image, and at least one specimen holder position to generate a composite representation of the tissue specimen that includes a representation of cell nuclei in the specimen; and a display device configured to display the composite representation of the tissue specimen in real-time.

Claims

1. An apparatus for real-time optical imaging of a tissue specimen, the apparatus comprising:

a primary imaging system configured to use an illumination source to acquire images of a tissue specimen through one or more spectrally separated channels, at least one of the one or more spectrally separated channels is configured to detect a range of wavelengths distinct from the range of wavelengths of the illumination source, the primary imaging system being an inverted microscope having a frame acquisition rate and configured to perform optical depth sectioning, the primary image system being configured to acquire a sequence of images;

an auxiliary imaging system, wherein the auxiliary imaging system is configured to acquire an auxiliary image of the tissue specimen, wherein the area of the auxiliary image is greater than the area of each image of the sequence of images acquired by the primary imaging system;

a specimen holder having a transparent window therewithin, the specimen holder being disposed in a specimen plane at a focal plane of the primary imaging system, the specimen holder being configured to hold the tissue specimen on the transparent window, the specimen holder comprising one or more position sensors, wherein the specimen holder is configured to be translatable in the specimen plane, the one or more position sensors being configured to measure a specimen holder position, and wherein the specimen holder is configured to be translatable to a focal plane of the auxiliary imaging system;

a user input device configured to accept user input, wherein the specimen holder is configured to translate in response to the user input in real-time;

a processing unit in electrical communication with the primary imaging system, the auxiliary imaging system, and the position sensors, wherein the processing unit is configured to execute a sequence of instructions on the sequence of images acquired by the primary imaging system, the auxiliary image, and at least one specimen holder position to generate a composite representation of the tissue specimen that includes a representation of cell nuclei in the specimen; and

a display device in electrical communication with the processing unit, the display device being configured to display the composite representation of the tissue specimen in real-time.

2. The apparatus of claim 1 wherein the representation of cell nuclei comprises virtual transillumination microscopy.

3. The apparatus of claim 2 wherein the primary imaging system is a multiphoton, confocal, light sheet, ultraviolet surface excitation, or structured illumination microscope.

4. The apparatus of claim 2, wherein each image of the sequence of images acquired by the primary imaging system is acquired through two or more spectrally separated channels;

and wherein at least one of the two or more spectrally separated channels is configured to detect light emitted by a nuclear contrast agent, and wherein at least one of the two or more spectrally separated channels is configured to detect light emitted from a complementary source of contrast.

5. The apparatus of claim 4 wherein the processing unit includes computer executable instructions for virtual transillumination microscopy processing on the sequence of images using the nonlinear process:

R=(1/(1−

exp(−

k)){circumflex over ( 

)}2)*(exp(−

B_Hematoxylin,red*I_nuclear*k)−

exp(−

k))*(exp(−

B_Eosin,red*I_secondary*k)−

exp(−

k))

G=(1/(1−

exp(−

k)){circumflex over ( 

)}2)*(exp(−

B_Hematoxylin,green*I_nuclear*k)−

exp(−

k))*(exp(−

B_Eosin,green*I_secondary*k)−

exp(−

k))

B=(1/(1−

exp(−

k)){circumflex over ( 

)}2)*(exp(−

B_Hematoxylin,blue*I_nuclear*k)−

exp(−

k))*(exp(−

B_Eosin,blue*I_secondary*k)−

exp(−

k))

where R, G, and B are the red, green and blue intensities of the virtual transillumination images respectively, B_Hematoxylin,red, B_Hematoxylin,green and B_Hematoxylin,blue are the absorption of hematoxylin for red, green and blue light respectively, B_Eosin,red, B_Eosin,green, and B_Eosin,blue are the absorption of eosin for red, green and blue light respectively, while I_nuclear is the intensity of the contrast agent has specificity for cell nuclei or components of cell nuclei, I_secondary is the intensity of the complementary source of contrast, and k is an arbitrary scaling constant.

6. The apparatus of claim 2 wherein the specimen holder further comprises one or more actuators for translation of the specimen holder.

7. The apparatus of claim 2 wherein the user input device incorporates mechanical transmission of force from the user to the specimen holder to move the specimen holder.

8. The apparatus of claim 2 wherein the auxiliary imaging system comprises a one dimensional digital array of photosensors.

9. The apparatus of claim 2 wherein the auxiliary imaging system comprises two or more spectrally distinct channels.

10. The apparatus of claim 9, wherein the processing unit is configured to compare the two or more spectrally distinct channels of the auxiliary imaging system to identify the location of the surgical inks, sutures or exogenous markings in the tissue samples.

11. The apparatus of claim 10, wherein the composite representation of the tissue specimen includes one or more locations of the surgical inks, sutures or exogenous markings on the tissue specimen.

12. The apparatus of claim 2 wherein the auxiliary imaging system includes a narrow band illuminator.

13. The apparatus of claim 2 wherein the specimen holder further includes a removable transparent window.

14. The apparatus of claim 13 wherein the removable transparent window incorporates a fluid retaining structure surrounding the transparent window.

15. The apparatus of claim 13, wherein the auxiliary imaging system is configured to be spatially separated from the primary imaging system.

16. The apparatus of claim 2 wherein the specimen holder further includes one or more of a specimen divider or a lid.

17. The apparatus of claim 15 wherein the auxiliary imaging system further includes an illumination source.

18. The apparatus of claim 17 wherein the illumination source is configured to be attached to the lid.

19. The apparatus of claim 15 wherein the lid further includes a specimen guide configured to hold the tissue specimen against the transparent window.

20. The apparatus of claim 2 wherein the processing unit further includes a graphics processing unit having hardware for parallel processing, and further wherein the graphics processing unit includes computer executable instructions for a series of parallel processing operations on the sequence of images acquired by the primary imaging system.

21. The apparatus of claim 20, wherein the graphics processing unit includes computer executable instructions for a series of parallel processing operations for virtual transillumination microscopy processing.

22. The apparatus of claim 2 wherein the processing unit is configured to update the composite representation of the tissue specimen after translation of the specimen holder in less than 250 milliseconds.

23. The apparatus of claim 2 wherein the composite representation of the tissue specimen comprises, for each image from the sequence of images acquired by the primary imaging system:

a representation of the image from the sequence of images; and

a representation of the auxiliary image with a subregion of the auxiliary image indicating the location on the specimen holder where the image from the sequence of images was acquired, wherein the subregion of the auxiliary image is computed using one or more specimen holder positions from the one or more position sensors.

24. The apparatus of claim 2, wherein the primary imaging system is configured to perform subsurface imaging.

25. The apparatus of claim 24 wherein the primary imaging system comprises an illumination source configured to emit light having a red or an infrared wavelength.

26. The apparatus of claim 25, wherein the illumination source is configured to emit light having a wavelength from 600 nm to 2000 nm.

27. The apparatus of claim 2, wherein each image in the sequence of images acquired by the primary imaging system is acquired in response to a measurement by the one or more position sensors, and wherein the processing unit is configured to generate an averaged or a non-averaged image of the images in the sequence of images.

28. A kit, comprising:

the apparatus of claim 1; and

a primary fluorescent nuclear contrast agents absorbing light emitted by the primary imaging system, and wherein a fluorescent emission wavelength of the primary fluorescent nuclear contrast agent corresponds to at least one of the spectrally separated channels.

29. The apparatus of claim 28, further comprising:

a secondary fluorescent contrast agent having a fluorescent emission wavelength different from the fluorescent emission wavelength of the primary fluorescent nuclear contrast agent, wherein the secondary fluorescent contrast agent has specificity for a structure selected from cytoplasm, stroma, collagen, or muscle.

30. The apparatus of claim 29, wherein the secondary fluorescent contrast agent is eosin-Y.

31. The apparatus of claim 29, further including a fluid medium in which at least one of the primary fluorescent nuclear contrast agent or the secondary fluorescent contrast agent is soluble.

32. The apparatus of claim 28, wherein the apparatus further comprises an identifier reader and wherein the kit further comprises an identifier.

33. The kit of claim 28, further comprising:

one or more fluorescent signal source selected from a surgical ink, a suture, or an exogenous marker, the fluorescent signal source having a fluorescence emission signal different from that of the primary fluorescent contrast agent.

34. The kit of claim 33, wherein the fluorescent signal source emits at a wavelength different from the fluorescent emission wavelength of the primary fluorescent contrast agent.

35. The kit of claim 33, wherein the fluorescent signal source emits at a range of wavelengths that overlaps the fluorescent emission wavelength of the primary fluorescent contrast agent.

36. The kit of claim 33, wherein the fluorescent signal source includes one or more of fluorescent microspheres, microcrystals, fluorescent dyes, or fluorescent plastics.

37. The apparatus of claim 1, wherein each image in the sequence of images acquired by the primary imaging system is acquired through one or more spectrally separated channels;

and

wherein at least one of the one or more spectrally separated channels is configured to detect light emitted by one or more sources selected from a surgical ink, a suture, and an exogenous marking.

38. The apparatus of claim 37, wherein:

at least one of the two or more spectrally separated channels is configured to detect light emitted by a nuclear contrast agent; and

at least one of the two or more spectrally separated channels is configured to detect light emitted by one or more sources selected from a surgical ink, a suture, and an exogenous marking.

39. A method of real-time optical imaging of a tissue specimen comprising the steps of:

applying one or more fluorescent contrast agents to a tissue specimen, wherein at least one of the one or more fluorescent contrast agents is a nuclear contrast agent;

providing an apparatus, comprising;

a primary imaging system configured to use an illumination source to acquire images of a tissue specimen through one or more spectrally separated channels, at least one of the one or more spectrally separated channels is configured to detect a range of wavelengths distinct from the wavelength of the illumination source, the primary imaging system being an inverted microscope having a frame acquisition rate and configured to perform optical depth sectioning, the primary image system being configured to acquire a sequence of images;

an auxiliary imaging system, wherein the auxiliary imaging system is configured to acquire an auxiliary image of the tissue specimen, wherein the area of the auxiliary image is greater than the area of each image of the sequence of images acquired by the primary imaging system;

a specimen holder having a transparent window therewithin, the specimen holder disposed in a specimen plane at a focal plane of the primary imaging system, the specimen holder being configured to hold the tissue specimen on the transparent window, the specimen holder comprising one or more position sensors, wherein the specimen holder is configured to be translatable in the specimen plane, the one or more position sensors being configured to measure a specimen holder position, and wherein the specimen holder is configured to be translatable to the focal plane of the auxiliary imaging system;

a user input device configured to accept user input, wherein the specimen holder is configured to translate in response to the user input in real-time;

a processing unit in electrical communication with the primary imaging system, the auxiliary imaging system, and the position sensors, wherein the processing unit is configured to execute a sequence of instructions on the sequence of images acquired by the primary imaging system, the auxiliary image, and at least one specimen holder position to form a representation of cell nuclei and to generate a composite representation of the tissue specimen; and

a display device in electrical communication with the processing unit, the display device being configured to display the composite representation of the tissue specimen in real-time,

situating the tissue specimen in the specimen holder;

positioning the specimen holder at the focal plane of the auxiliary imaging system;

causing the auxiliary imaging system to acquire the auxiliary image;

positioning the specimen holder at the focal plane of the primary imaging system;

causing the primary imaging system to acquire the sequence of images;

causing a processing unit to detect cell nuclei within the tissue specimen and to generate a composite representation of the tissue specimen;

causing the display device to display the composite representation of the tissue specimen in real-time; and

causing the specimen holder to translate in the specimen plane using the user input device.

40. The method of claim 39, wherein acquiring the sequence of images comprises using multiphoton, confocal, light sheet, ultraviolet surface excitation, or structured illumination microscopy.

41. The method of claim 39, wherein the primary imaging system is configured to adjust magnification by varying the area scanned, the method further including adjusting magnification of the primary imaging system.

42. The method of claim 39, wherein:

each image of the sequence of images acquired by the primary imaging system is acquired through two or more spectrally separated channels;

and wherein at least one of the two or more spectrally separated channels is detecting light emitted by the nuclear contrast agent, and wherein at least one of the two or more spectrally separated channels is detecting light emitted from a complementary source of contrast.

43. The method of claim 39, wherein the nuclear contrast agent is a rapid diffusion agent.

44. The method of claim 39, wherein the nuclear contrast agent is an agent having enhancement of fluorescent emission by associating with cell nuclei.

45. The method of claim 39, wherein the specimen holder includes a removable window, the method further comprising the steps of:

removing the transparent window from the specimen holder;

situating one or more tissue specimens onto the transparent window; and

inserting the transparent window back into the specimen holder.

46. The method of claim 39, wherein the specimen holder includes a specimen divider configured to separate two or more tissue specimens.

47. The method of claim 39, wherein the specimen holder includes a lid, the method further including covering the tissue specimen with the lid.

48. The method of claim 47, wherein the lid includes a specimen guide, the method further including compressing the tissue specimen against the transparent window using the specimen guide.

49. The method of claim 39, further including identifying surgical inks, sutures, or exogenous markings on the tissue specimen created during a medical procedure using the auxiliary image.

50. The method of claim 49, further including measuring the distance between a selected location in any one of the sequence of images acquired by the primary imaging system and the surgical inks, sutures or exogenous markings on the tissue specimen using the representation of the auxiliary image.

51. The method of claim 49, further including illuminating the tissue specimen with a narrow band illumination and identifying surgical inks, sutures, or exogenous markings from known absorption spectra.

52. The method of claim 49, further including trans-illuminating the tissue specimen with the illumination source and identifying surgical inks, sutures, or exogenous markings.

53. The method of claim 39, further including causing the processing unit to record at least one of:

the sequence of images acquired by the primary imaging system, the specimen holder position, the auxiliary image, or the composite representation.

54. The method of claim 39, further including:

causing the graphics processing unit to execute computer executable instructions for a series of parallel processing operations on the sequence of images.

55. The method of claim 54, wherein the graphics processing unit includes computer executable instructions for a series of parallel processing operations for virtual transillumination microscopy processing.

56. The method of claim 39, further including causing the processing unit to generate the composite representation of the tissue specimen, said composite representation comprising, for each image from the sequence of images acquired by the primary imaging system:

a representation of the image from the sequence of images; and

a representation of the auxiliary image with a subregion of the auxiliary image indicating the location on the specimen holder where the image from the sequence of images was acquired, wherein the subregion of the auxiliary image is computed using one or more specimen holder positions.

57. The method of claim 39, wherein the composite representation comprises, for at least one image in the sequence of images acquired by the primary imaging system:

a representation of the image indicating two or more fiducial marks;

a representation of the auxiliary image; and

a representation of a distance between the two or more fiducial marks.

58. The method of claim 39, wherein the processing unit is configured to generate a first and a second composite representations, wherein:

the first composite representation comprises;

a representation of a first image from the sequence of images acquired by the primary imaging system, the representation of the first image indicating a first fiducial mark, recorded in response to user input; and

a representation of the auxiliary image;

the second composite representation comprises;

a representation of a second image from the sequence of images acquired by the primary imaging system, the representation of the second image indicating a second fiducial mark, recorded in response to user input; and

a representation of the auxiliary image; and

a representation of a distance between the first and the second fiducial marks.