Graphical user interface for interpreting the results of image analysis

• US 8,711,149 B2
• Filed: 12/07/2011
• Issued: 04/29/2014
• Est. Priority Date: 12/08/2010
• Status: Active Grant

• Current Assignee: Definiens AG (Astrazeneca PLC)
• Original Assignee: Definiens AG (Astrazeneca PLC)
• Inventors: Schmidt, Guenter, Schaepe, Arno, Binnig, Gerd
• Primary Examiner(s): McDowell, Jr., Maurice L
• Application Number:
  US13/313,685
• Publication Number:
  US 20120147010A1
• Time in Patent Office: 874 Days
• Field of Search: 345/440, 715/771, 382/128
• US Class Current: 345/440
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 2207/30072 Microarray; Biochip, DNA array; Well plate
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

A method of intuitively displaying values obtained from analyzing bio-medical images includes displaying a table of the values in a first pane of a graphical user interface. The table contains a user selectable row that includes a reference value and two numerical values. The reference value refers to an image of a tissue slice. The first numerical value is generated by performing image analysis on the image, and the second numerical value indicates a health state of the tissue. The image is displayed in a second pane of the graphical user interface in response to the user selecting the user selectable row. A graphical plot with a selectable symbol associated with the image is displayed in a third pane. The symbol has a position in the plot defined by the values. Alternatively, in response to the user selecting the selectable symbol, the image is displayed in the second pane.

Claims

1. A method for displaying values of a table and images associated with one or more rows of the values on a graphical user interface, the method comprising:

displaying the table in a first pane of the graphical user interface, wherein the table includes a user selectable row, wherein the user selectable row includes a reference value, a first numerical value and a second numerical value, wherein the reference value refers to a digital image of a slice of tissue, wherein the first numerical value is generated by performing image analysis on the digital image, and wherein the second numerical value indicates a health state of the slice of tissue; and

displaying the digital image in a second pane of the graphical user interface in response to the user selecting the user selectable row.

2. The method of claim 1, wherein the digital image depicts only a portion the slice of tissue.

3. The method of claim 1, wherein the displayed image shows a classification of an object in the digital image.

4. The method of claim 1, wherein the first numerical value is taken from the group consisting of:

an Allred score, a Gleason score, an Elston-Ellis score, and a HercepTest score.

5. The method of claim 1, wherein the health state of the slice of tissue is indicative of a disease free survival time.

6. The method of claim 1, wherein identified cells are indicated in the digital image displayed in the second pane, and wherein the identified cells are taken from the group consisting of:

mitotic cells, tumor cells, stroma cells, cells exhibiting an overexpression of a protein, cells exhibiting an overamplification of a gene, and inflamed cells.

7. The method of claim 1, wherein an identified cell component is indicated in the digital image displayed in the second pane, and wherein the identified cell component is taken from the group consisting of:

a nucleus, a cytoplasm and a membrane.

8. The method of claim 1, wherein an identified tissue region is indicated in the digital image displayed in the second pane, and wherein the identified tissue region is taken from the group consisting of:

a tumor region, a stroma region, an inflamed region, an epithelial region and a glandular region.

9. The method of claim 1, wherein the user selectable row is marked with a color that corresponds to a classification of the digital image.

10. The method of claim 1, wherein the health state is determined by a statistical method using the first numerical value, and wherein the statistical method is taken from the group consisting of:

applying a decision tree, applying a linear regression plot, applying a support vector machine, applying a Bayesian classifier.

11. The method of claim 1, further comprising:

displaying a Kaplan Meier plot in a third pane of the graphical user interface, wherein a symbol is displayed in the Kaplan Meier plot in response to the user selecting the user selectable row, and wherein the symbol has a position in the Kaplan Meier plot defined by the health state.

12. The method of claim 1, further comprising:

displaying a second digital image together with the digital image in an image gallery manner in the second pane in response to the user selecting a second user selectable row, wherein a second reference value in the second user selectable row refers to the second digital image.

13. The method of claim 12, wherein the second digital image displayed in the image gallery manner includes an outline of an object identified in the second digital image.

14. The method of claim 12, further comprising:

ordering digital images displayed in the image gallery manner in the second pane to correspond to an order of user selectable rows in the table that correspond to the digital images.

15. The method of claim 14, wherein the ordering of the digital images occurs as the user selectable rows are selected.

16. The method of claim 14, wherein the ordered digital images depict portions of digital images of slices of tissue.

17. A computer program product tangibly embodied on a non-transitory computer-readable storage medium and comprising computer readable and executable program instructions that when executed by a processor provide a visual display of a user'"'"'s graphical user interface on an interconnected display device, the graphical user interface comprising:

a first pane displaying a graphical plot including a user selectable symbol, wherein the user selectable symbol is associated with a digital image of a slice of tissue, wherein the user selectable symbol has a position in the graphical plot defined by a first numerical value and a second numerical value, wherein a third numerical value is indicative of a health state of the slice of tissue, wherein the first and second numerical values are generated by image analysis of the digital image, and wherein the symbol represents the health state; and

a second pane displaying images, wherein the second pane displays the digital image in response to the user selecting the selectable symbol.

18. The computer program product of claim 17, wherein the third numerical values is generated by image analysis of the digital image of the slice of tissue.

19. The computer program product of claim 17, wherein the first numerical value indicates an intensity of immunohistochemical staining of cancer cells in the digital image of the slice of tissue.

20. The computer program product of claim 17, wherein the second numerical value indicates a proportion of stained cells in a portion of the digital image.

21. The computer program product of claim 17, wherein the health state is a disease free survival time.

22. The computer program product of claim 17, wherein the third numerical value is indicative of the health state of a portion of the slice of tissue, and wherein the portion of the slice of tissue is taken from the group consisting of:

normal tissue, in-situ cancer tissue, and distant metastasis tissue.

23. The computer program product of claim 17, wherein the health state of the slice of tissue is determined by image analysis by an expression status of a protein taken from the group consisting of:

estrogen receptor protein (ER), progesterone receptor protein (PR), KI67 protein, and HER2neu protein.

24. The computer program product of claim 17, wherein the graphical plot is taken from the group consisting of:

a scatter plot, a line plot, and a histogram plot.

25. A method comprising:

displaying a first image analysis value in a table in a first pane of a graphical user interface, wherein the first image analysis value is generated by analyzing a digital image of a slice of tissue using a first parameter;

displaying a list of selectable statistical methods in a second pane of the graphical user interface, wherein the list includes a statistical method;

computing a second parameter using the statistical method after the user has selected the statistical method; and

displaying a second image analysis value in the table after the user has selected the statistical method, wherein the second image analysis value is generated by analyzing the digital image using the second parameter.

26. The method of claim 25, further comprising:

displaying a new version of the digital image in a third pane of the graphical user interface after the user has selected the statistical method, wherein the new version of the digital image is generated by analyzing the digital image using the second parameter.

27. The method of claim 25, wherein the statistical method uses the first image analysis value to analyze the digital image.

28. The method of claim 25, wherein the first parameter is a decision threshold, and wherein the statistical method involves generating a decision tree.

29. The method of claim 25, wherein the first parameter is a segmentation threshold, and wherein the statistical method involves performing an aggregation method taken from the group consisting of:

determining an average value, determining a standard deviation value, determining an absolute deviation value, and determining a quantile value.

30. The method of claim 25, wherein the statistical method is taken from the group consisting of:

generating a decision tree, generating a linear regression plot, generating a support vector machine, generating a Bayes classifier.

31. The method of claim 25, wherein the slice of tissue includes tumor tissue.

32. The method of claim 25, wherein the first image analysis value is generated by analyzing only a portion of the digital image.

33. A method comprising:

displaying a table in a first pane of a graphical user interface, wherein the table includes a user selectable row, wherein the user selectable row includes a reference value, a first numerical value and a second numerical value, wherein the reference value refers to a digital image of a slice of tissue of a patient, wherein the first numerical value is generated by performing image analysis on the digital image, and wherein the second numerical value indicates a biological state; and

displaying the digital image in a second pane of the graphical user interface in response to a user selecting the user selectable row.

34. The method of claim 33, wherein the biological state corresponds to a health state of the patient.

35. The method of claim 33, wherein the biological state corresponds of to a health state of the slice of tissue.

36. The method of claim 33, wherein the health state of the slice of tissue is determined using gene expression information.

37. The method of claim 33, wherein the reference value includes a grouping of reference values.

38. The method of claim 33, wherein the displayed digital image includes an outline of an object in the digital image.

39. The method of claim 33, wherein the displayed digital image includes an outline in a common shape that encloses an object in the digital image, and wherein the common shape is taken from the group consisting of:

a polygon, a circle and an oval.