Generating artificial hyperspectral images using correlated analysis of co-registered images

• US 8,699,769 B2
• Filed: 07/11/2012
• Issued: 04/15/2014
• Est. Priority Date: 07/12/2011
• Status: Active Grant

• Current Assignee: Definiens AG (Astrazeneca PLC)
• Original Assignee: Definiens AG (Astrazeneca PLC)
• Inventors: Schoenmeyer, Ralf, Ellenberg, Peter, Binnig, Gerd, Athelogou, Maria, Schmidt, Guenter
• Primary Examiner(s): Abdi, Amara
• Application Number:
  US13/546,182
• Publication Number:
  US 20130016886A1
• Time in Patent Office: 643 Days
• Field of Search: None
• US Class Current: 382/128
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 2207/10061 from scanning electron microscope
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 7/0012 Biomedical image inspection

Abstract

High-resolution digital images of adjacent slices of a tissue sample are acquired, and tiles are defined in the images. Values associated with image objects detected in each tile are calculated. The tiles in adjacent images are co-registered. A first hyperspectral image is generated using a first image, and a second hyperspectral image is generated using a second image. A first pixel of the first hyperspectral image has a first pixel value corresponding to a local value obtained using image analysis on a tile in the first image. A second pixel of the second hyperspectral image has a second pixel value corresponding to a local value calculated from a tile in the second image. A third hyperspectral image is generated by combining the first and second hyperspectral images. The third hyperspectral image is then displayed on a computer monitor using a false-color encoding generated using the first and second pixel values.

Claims

1. A method comprising:

(a) defining a first tile on a first digital image of a first slice of a tissue sample, wherein the first tile has a first position in the first digital image;

(b) defining a second tile on a second digital image of a second slice of the tissue sample, wherein the second tile has a second position in the second digital image, and wherein the second slice originated from the tissue sample adjacent to the first slice;

(c) calculating a first value associated with image objects detected in the first tile;

(d) calculating a second value associated with image objects detected in the second tile;

(e) generating a first hyperspectral image using the first digital image, wherein a first pixel of the first hyperspectral image has a first pixel value corresponding to the first value and a first pixel location corresponding to the first position;

(f) generating a second hyperspectral image using the second digital image, wherein a second pixel of the second hyperspectral image has a second pixel value corresponding to the second value and a second pixel location corresponding to the second position;

(g) generating a third hyperspectral image using the first hyperspectral image and the second hyperspectral image; and

(h) displaying the third hyperspectral image on a computer monitor using a false-color encoding generated using the first pixel value and the second pixel value.

2. The method of claim 1, wherein the first slice is stained with a first biomarker, wherein the first value corresponds to a histopathological score of the first biomarker, wherein the second slice is stained with a second biomarker, and wherein the second value corresponds to a histopathological score of the second biomarker.

3. The method of claim 2, wherein the false-color encoding is based on a difference between the first value and the second value.

4. The method of claim 2, wherein the third hyperspectral image depicts a heterogeneity of a tumor in the tissue sample.

5. The method of claim 1, wherein the third hyperspectral image is generated in (g) by co-registering the first hyperspectral image with the second hyperspectral image.

6. The method of claim 1, further comprising:

(i) generating an image object by segmenting the third hyperspectral image using an image analysis process.

7. The method of claim 6, further comprising:

(j) determining a value of a property of the image object of the third hyperspectral image; and

(k) storing the value of the property in a non-volatile memory.

8. The method of claim 1, wherein the first slice is stained with an H&

E biomarker, and wherein the first value corresponds to a number of mitotic objects stained by the H&

E biomarker.

9. The method of claim 1, wherein the second slice is immunohistochemically (IHC) stained, and wherein the second value corresponds to a first Allred score.

10. The method of claim 9, wherein the second slice is stained using an estrogen receptor antibody.

11. The method of claim 10, wherein the first slice is immunohistochemically (IHC) stained using a progesterone receptor antibody, and wherein the first value corresponds to a second Allred score.

12. The method of claim 1, wherein the second slice is processed with in-situ hybridization, and wherein the second value indicates gene amplification.

13. The method of claim 1, wherein the first value is calculated using a factor, and wherein the factor is taken from the group consisting of:

a mean, a median, a minimum, a maximum, a quantile, and a standard deviation of a property of a subset of the image objects detected in the first tile.

14. The method of claim 13, wherein the first value is calculated also using the property of image objects detected in tiles adjacent to the first tile.

15. The method of claim 13, wherein the subset of the image objects detected in the first tile are those image objects classified as nuclei of tumor cells, and wherein the property is an intensity of staining of the subset of the image objects.

16. The method of claim 1, further comprising:

(i) selecting a pixel of the third hyperspectral image; and

(j) displaying on the computer monitor a tile of the first digital image that corresponds to the selected pixel of the third hyperspectral image.

17. The method of claim 1, further comprising:

(i) selecting the first tile of the first digital image; and

(j) displaying the first pixel value on the computer monitor.

18. The method of claim 17, wherein the first pixel value is displayed as part of a bar chart.

19. The method of claim 13, wherein the tissue sample is taken from a patient, further comprising:

(i) generating a therapy response for the patient based on the first value.

20. The method of claim 13, wherein the tissue sample is taken from a patient with cancer, further comprising:

(i) determining a probability of recurrence of the cancer based on the first value.

21. The method of claim 1, wherein the first value is calculated using a distance weighted mean of a property of a subset of the image objects detected in the first tile, and wherein the distance weighted mean is based on distances from a center of the first tile to a center of each of the image objects in the subset.