Context driven image mining to generate image-based biomarkers

• US 9,245,337 B2
• Filed: 10/30/2013
• Issued: 01/26/2016
• Est. Priority Date: 10/15/2002
• Status: Active Grant

• Current Assignee: Definiens AG (Astrazeneca PLC)
• Original Assignee: Definiens AG (Astrazeneca PLC)
• Inventors: Schmidt, Guenter, Binnig, Gerd, Schoenmeyer, Ralf, Schaepe, Arno
• Primary Examiner(s): Newman, Michael A
• Application Number:
  US14/067,932
• Publication Number:
  US 20140050384A1
• Time in Patent Office: 818 Days
• Field of Search: 382/132, 382/128, 382/130, 382/131, 382/155, 382/158, 382/181
• US Class Current: 1/1
• CPC Class Codes: G01N 2800/00 Detection or diagnosis of diseases
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30061 Lung
• CPC Class Codes: G06T 2207/30068 Mammography; Breast
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/0014 using an image reference approach
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes

Abstract

An image-based biomarker is generated using image features obtained through object-oriented image analysis of medical images. The values of a first subset of image features are measured and weighted. The weighted values of the image features are summed to calculate the magnitude of a first image-based biomarker. The magnitude of the biomarker for each patient is correlated with a clinical endpoint, such as a survival time, that was observed for the patient whose medical images were analyzed. The correlation is displayed on a graphical user interface as a scatter plot. A second subset of image features is selected that belong to a second image-based biomarker such that the magnitudes of the second image-based biomarker for the patients better correlate with the clinical endpoints observed for those patients. The second biomarker can then be used to predict the clinical endpoint of other patients whose clinical endpoints have not yet been observed.

Claims

1. A method comprising:

generating a plurality of image objects by segmenting a digital image using object-oriented image analysis;

selecting selected image objects from among the plurality of image objects;

displaying one of the selected image objects on a graphical user interface;

determining a value of each of a subset of image features of the plurality of image objects, wherein the subset of image features constitutes a custom image-based biomarker;

calculating magnitudes of the custom image-based biomarker for each of the plurality of image objects; and

changing the image features included in the subset of image features such that magnitudes of the custom image-based biomarker for the selected image objects are greater than the magnitudes of the custom image-based biomarker for image objects other than the selected image objects.

2. The method of claim 1, further comprising:

searching a database of digital images for which magnitudes of the customer image-based biomarker have been calculated for image objects in the digital images and identifying identified digital images that include matched image objects for which the magnitudes of the custom image-based biomarker are greater than the magnitudes of the custom image-based biomarker for image objects in the identified digital images other than the matched image objects.

3. The method of claim 2, further comprising:

retrieving an indication of the medical condition of patients associated with the identified digital images.

4. The method of claim 2, further comprising:

displaying a list of the identified digital images.

5. The method of claim 1, wherein the calculating the magnitudes of the custom image-based biomarker involves weighting each value of each image feature of each subset and summing all of the weighted values of the image features in the subset.

6. The method of claim 1, wherein the digital image is a medical image of a patient.

7. The method of claim 1, wherein the selected image objects are manually selected by a clinical expert.

8. The method of claim 1, wherein the selected image objects are generated from within regions of interest in the digital image, and wherein the regions of interest depict cancerous tissue identified by a pathologist.

9. The method of claim 1, wherein the selected image objects correspond to human tissue taken from the group consisting of:

tumor cells, lung cells and lymph nodes.

10. The method of claim 1, wherein the digital image is acquired by a computed tomography (CT) system.

11. The method of claim 1, wherein the digital image is acquired by a digital pathology slide scanner.

12. The method of claim 1, wherein the digital image depicts tissue stained using immunohistochemistry, and wherein antibodies are used to measure protein expression in the stained tissue.

13. The method of claim 1, wherein the digital image depicts tissue stained using fluorescence in situ hybridization (FISH), and wherein genetic markers are used to measure gene amplification in the tissue.

14. A method comprising:

generating image objects by segmenting first digital images of tissue using object-oriented image analysis, wherein the tissue is taken from a first patient;

determining a value of each of a first subset of image features of the image objects, wherein the image features of the first subset belong to a first custom biomarker;

calculating a magnitude of the first custom biomarker for each of the first digital images based on the values of the first subset of image features for the image objects in the first digital images; and

selecting a second subset of image features that belong to a second custom biomarker such that the magnitude of the second custom biomarker for each of the first digital images is greater than the magnitude of the first custom biomarker for each of the first digital images.

15. The method of claim 14, wherein the calculating the magnitude of the first custom biomarker involves weighting each value of each image feature of the first subset and summing all of the weighted values of the image features in the first subset.

16. The method of claim 14, further comprising:

modifying how the value of each image feature in the second subset is weighted such that the magnitude of the second custom biomarker for each of the first digital images increases.

17. The method of claim 14, further comprising:

searching a database of second digital images for which magnitudes of the second customer biomarker have been calculated and identifying matched digital images for which the magnitudes of the second custom biomarker are greater than the magnitudes of the second custom biomarker for the first digital images.

18. The method of claim 17, further comprising:

displaying a list of the matched digital images.

19. The method of claim 17, further comprising:

retrieving an indication of the medical condition of patients associated with the matched digital images.

20. The method of claim 14, further comprising:

saving the second custom biomarker on a computer-readable medium.