MULTI-MODALITY CONTRAST AND BRIGHTFIELD CONTEXT RENDERING FOR ENHANCED PATHOLOGY DETERMINATION AND MULTI-ANALYTE DETECTION IN TISSUE

• US 20120200694A1
• Filed: 10/07/2010
• Published: 08/09/2012
• Est. Priority Date: 10/12/2009
• Status: Active Grant

• Current Assignee: Ventana Medical Systems Incorporated (Roche Holding AG)
• Original Assignee: Ventana Medical Systems Incorporated (Roche Holding AG)
• Inventors: Pestano, Gary, Nagle, Ray B., Miller, Phillip, Garsha, Karl, Otter, Michael, Nagy, Alexandra Dea, Froehlich, Jan, Day, William
• Granted Patent: US 9,310,302 B2
• Time in Patent Office: Days
• US Class Current: 348/79
• CPC Class Codes: G01N 21/6456 Spatial resolved fluorescence measurements; Imaging

Abstract

Multiple modality contrast can be used to produce images that can be combined and rendered to produce images similar to those produced with wavelength absorbing stains viewed under transmitted white light illumination. Images obtained with other complementary contrast modalities can be presented using engineered color schemes based on classical contrast methods used to reveal the same anatomical structures and histochemistry, thereby providing relevance to medical training and experience. Dark-field contrast images derived from refractive index and fluorescent DAPI counterstain images are combined to produce images similar to those obtained with conventional H&E staining for pathology interpretation. Such multi-modal image data can be streamed for live navigation of histological samples, and can be combined with molecular localizations of genetic DNA probes (FISH), sites of mRNA expression (mRNA-ISH), and immunohistochemical (IHC) probes localized on the same tissue sections, used to evaluate and map tissue sections prepared for imaging mass spectrometry.

Claims

1. A method, comprising:

generating at least one first image and at least one second image of a specimen, wherein the generated first and second images comprise complementary contrasting image parameters; and

producing the contrasting images for observation.

2. The method of claim 1, further comprising obtaining the at least one first image and the at least one second image by subjecting the specimen to a first stimulus beam and a second stimulus beam, respectively.

3. The method of claim 2, wherein the first and second images are combined to produce a combined image.

4. The method of claim 2 or 3, wherein the first and second stimulus beams are applied to the specimen simultaneously, and the associated contrasting complementary images are obtained simultaneously.

5. The method of any one of claims 1-4, further comprising displaying the first and second images side-by-side.

6. The method of claim 2, further comprising displaying the combined image.

7. The method of claim 2, further comprising recording the first and second contrasting images as corresponding recorded images.

8. The method of any one of claims 2-7, wherein the specimen is fluorescently stained and the first beam is selected to produce fluorescence by the fluorescent stain so that the first image is a fluorescence image of the specimen, and further wherein the second stimulus beam is applied to the specimen so that the second image is a refractive dark field image.

9. The method of claim 8, further comprising recording the fluorescence image and the refractive dark field image as corresponding recorded images.

10. The method of claim 9, further comprising:

applying a color mapping to the recorded dark field image to produce a pseudo-color dark field image; and

combining the pseudo-color dark field recorded image and the recorded fluorescence image to produce a combined recorded image.

11. The method of claim 10, wherein the color mapping is based on a color lookup table associated with at least one absorptive stain.

12. The method of claim 11, wherein the stain is an eosin stain.

13. The method of claim 10, further comprising applying a color lookup table to the fluorescence image, wherein the color lookup table is associated with at least one absorptive stain.

14. The method of claim 13, wherein the fluorescence is based on DAPI fluorescence, and the color lookup table associated with the fluorescence image is based on a hematoxylin stain.

15. The method of claim 10, further comprising producing a pseudo-brightfield recorded image based on the combined recording image.

16. The method of claim 15, further comprising applying color lookup tables to the refractive dark field image and the fluorescence image so as to produce an image having image contrast associated with hematoxylin and eosin staining.

17. The method of claim 2, further comprising generating a mass spectroscopic image of the specimen.

18. An imaging apparatus, comprising:

a first imaging system configured to produce a first image of a specimen; and

a second imaging system configured to produce a second image of the specimen, wherein the first and second images are complementary images.

19. The imaging apparatus of claim 18, wherein the first image system is a refractive dark field optical system configured to produce a the first image as a refractive dark field image and the second imaging system is a fluorescence optical system configured to produce the second image as a fluorescence image.

20. The imaging apparatus of claim 18, further comprising at least one image capture device coupled to receive the first and second images, and an image processor configured to record the first and second images.

21. The imaging apparatus of claim 20, wherein the image processor is configured to produce a combined image based on the first and second images.

22. The imaging apparatus of claim 21, wherein the image processor is configured to apply a color lookup table to at least one of the first and second recorded images to produce the combined image based on a pseudo-colored rendering of at least one of the first and second recorded images.

23. The imaging apparatus of claim 22, further comprising rendering the combined image as a pseudo-brightfield image.

24. The imaging apparatus of claim 23, wherein the image processor is configured to process the first image based on a color lookup table associated with an eosin stain and to process the second image based on color lookup table associated with a hematoxylin stain, wherein the combined image is based on the processed first and second images.

25. The imaging apparatus of claim 20, wherein the image capture device is configured to receive the first image and the second image as side by side images.

26. The imaging apparatus of claim 21, wherein the image processor is configured to overlay the first and second images to produce the combined image.

27. The imaging apparatus of claim 26, further comprising a display configured to receive and display the combined image.

28. At least one computer readable storage media comprising computer-executable instructions for:

receiving a first image and a complementary second image associated with common portion of a specimen section; and

combining the first and second images to produce a combined image.

29. The at least one computer readable storage media of claim 28, further comprising computer-executable instructions for producing a pseudo-color image based on at least one of the first image and the second image, wherein the combined image is based on the at least one pseudo-color image.

30. The at least one computer readable storage media of claim 29, further comprising computer executable instructions for inverting the combined image to produce a brightfield rendered image.

31. The at least one computer readable storage media of claim 28, further comprising computer executable instructions for receiving a mass spectroscopic image of the common specimen section, and including the mass spectroscopic image in the combined image.

32. The at least one computer readable storage media of any one of claims 28-31, wherein the first image and the second image are a dark field refraction contrast image and a fluorescence image, respectively.

33. The at least one computer readable storage media of claim 32, wherein the first image is processed based on a color lookup table associated with an eosin stain and the second image is processed based on color lookup table associated with a hematoxylin stain, wherein the combined image is based on the processed first and second images.

34. An image processor, comprising:

image inputs configured to receive at least a first image and a complementary second image;

a color lookup table input configured to receive at least one color lookup table;

an image combiner configured to process at least one of the first image and the complementary second image based on the at least one color lookup table and produce a pseudo-color image, and to combine the pseudo-color image with one of the first image and the second image.

35. The image processor of claim 34, wherein the image combiner is configured to produce a brightfield rendering based on the combined image.

36. The image processor of claim 35, wherein the color lookup table input is configured to receive a first color lookup table associated with an absorptive stain and a second color lookup table associated with a fluorescent stain, and the image combiner is configured to process the first and second images based on the first and second color lookup tables, respectively, and wherein the combined image is based on the processed first and second images.

37. The image processor of claim 36, wherein the absorptive stain is an eosin stain and the fluorescent stain is a hematoxylin stain.