Systems and methods for performing quality review scoring of biomarkers and image analysis methods for biological tissue

• US 9,036,888 B2
• Filed: 04/30/2012
• Issued: 05/19/2015
• Est. Priority Date: 04/30/2012
• Status: Active Grant

• Current Assignee: Leica Microsystems Gmbh (Danaher Corporation)
• Original Assignee: General Electric Company
• Inventors: Kamath, Vidya Pundalik, Santamaria-Pang, Alberto, Sarachan, Brion Daryl, Rittscher, Jens
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Kholdebarin, Iman K
• Application Number:
  US13/460,081
• Publication Number:
  US 20130287283A1
• Time in Patent Office: 1,114 Days
• Field of Search: None
• US Class Current: 382/133
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G06F 18/2178 based on feedback of a supervisor
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06V 10/7784 based on feedback from supervisors
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 2380/08 Biomedical applications
• CPC Class Codes: G09G 5/026 Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence

Abstract

Exemplary embodiments include methods, systems, and devices for enabling users to provide quality scores for indicating the quality of image analysis methods performed on images of biological tissue. An exemplary user interface displays results of an image analysis method performed on an image of biological tissue in an overlaid manner on an image of biological tissue. The exemplary user interface enable a user to provide, directly on the user interface, one or more quality scores to indicate the user'"'"'s assessment of the quality of the image analysis performed on the image. Exemplary embodiments store the quality scores provided by the user in association with the image analysis method and the image of biological tissue.

Claims

1. A computer-implemented method for performing quality review of analysis of biological tissue, the method comprising:

rendering a graphical user interface on a visual display device;

rendering, on the graphical user interface, a field of view selection component allowing a user to select a field of view from a data set comprising tissue profile data including multiplexed biomarker images capturing expression of a plurality of biomarkers in a plurality of fields of view of biological tissue, the field of view selection component allowing the user to configure a color and/or a transparency of a representation on the graphical user interface of biological tissue corresponding to a selected field of view;

in response to user input selecting a field of view corresponding to a biological tissue at the field of view selection component, rendering, on the graphical user interface, a first image of the selected field of view corresponding to the biological tissue;

rendering, on the graphical user interface, a morphological analysis selection component allowing a user to select a morphological analysis of multiplexed biomarker image data corresponding to the selected field of view, the morphological analysis selection component allowing the user to configure a color and/or a transparency of a representation of a result of a selected morphological analysis;

in response to user input selecting a first morphological analysis at the morphological analysis selection component, overlaying, on the first image of the biological tissue, a representation of a first result of the selected first morphological analysis, the first result of the first morphological analysis indicating one or more morphological features of interest in the first image, wherein the first image of the biological tissue and the representation of the first result are displayed in a visually distinguishable manner; and

rendering, on the graphical user interface, a quality review selection component allowing the user to select a quality review score based on a comparison between the first image and the representation of the first result of the first morphological analysis rendered on the graphical user interface.

2. The method of claim 1, further comprising:

receiving a quality review score from the user, at the quality review selection component on the user interface, indicating one or more quality aspects of the first result of the first morphological analysis.

3. The method of claim 2, further comprising:

sending instructions to store, on a storage device, the quality review score in association with the multiplexed biomarker image data corresponding to the selected field of view of the biological tissue and the selected first morphological analysis.

4. The method of claim 2, wherein the instructions further indicate that the quality review score is to be stored in association with an identification of the user providing the quality review score.

5. The method of claim 1, further comprising:

receiving user input to change the first image to a second image of the selected field of view; and

in response to the user input, removing the first image from the user interface and rendering the second image of the selected field of view on the graphical user interface, while continuing to render the representation of the first result of the first morphological analysis such that the second image is overlaid by the representation of the first result of the first morphological analysis.

6. The method of claim 1, further comprising:

receiving user input to change the first result of the first morphological analysis to a second result of a second morphological analysis performed on the multiplexed biomarker image data, the second result of the second morphological analysis indicating one or more morphological features of interest in the first image; and

in response to the user input, removing the representation of first result of the first morphological analysis from the user interface and overlaying, on the first image, a representation of the second result of the second morphological analysis.

7. The method of claim 1, further comprising:

overlaying, on the first image and the representation of the first result, a representation of a second result of a second morphological analysis performed on the multiplexed biomarker image data, the second result of the second morphological analysis indicating one or more morphological features of interest in the first image.

8. The method of claim 1, wherein the first morphological analysis includes running an image segmentation algorithm implemented on a computing device on the multiplexed biomarker image data, and wherein the first result of the first morphological analysis includes a set of one or more segments delineating one or more biological units of interest in the first image.

9. The method of claim 8, wherein a biological unit of interest is a cell, and wherein the one or more segments trace cell membranes displayed on the first image.

10. The method of claim 8, wherein a biological unit of interest is a collection of cells.

11. The method of claim 8, wherein a biological unit of interest is a sub-cellular component of a cell.

12. The method of claim 8, wherein the quality review score includes a segmentation quality score that indicates an evaluation of a performance of the image segmentation algorithm.

13. The method of claim 12, further comprising:

automatically determining that one or more segmentation quality scores, corresponding to results generated by a single image segmentation algorithm run on one or more images of a type of biological tissue, are below a predefined quality threshold; and

automatically determining that the image segmentation algorithm is unsuitable for processing the type of the biological tissue.

14. The method of claim 8, wherein the quality review score includes a marker quality score that indicates an evaluation of a quality of a marker used to treat the biological tissue prior to capturing the first image of the biological tissue.

15. The method of claim 14, further comprising:

automatically determining that one or more marker quality scores, corresponding to one or more images of a type of biological tissue treated with a single marker, are below a predefined quality threshold; and

automatically determining that the marker is unsuitable for staining the type of the biological tissue.

16. The method of claim 8, further comprising:

receiving user input, at the graphical user interface, selecting one or more morphological units of the biological tissue on the first image to indicate that the selected units are incorrectly identified by one or more segments generated by the image segmentation algorithm; and

adjusting the rendering of the selected units on the graphical user interface to indicate that the selected units are incorrectly segmented.

17. The method of claim 16, further comprising:

removing representations of the selected units of the biological tissue from the first image.

18. The method of claim 1, wherein the first morphological analysis includes running a segment ranking algorithm on the multiplexed biomarker image data, and wherein the first result of the first morphological analysis includes one or more probability indicators corresponding to one or more segments delineating one or more biological units of interest in the biological tissue, each probability indicator indicating whether a corresponding segment correctly identifies the biological unit of interest.

19. The method of claim 1, wherein the biological unit of interest is a cell.

20. The method of claim 18, wherein the first result is represented as numerical rankings represented on the one or more segments on the first image, and wherein higher rankings indicate higher probabilities that the corresponding segments correctly identify the biological units of interest.

21. The method of claim 18, further comprising:

receiving user input, at the user interface, selecting one or more segments represented on the first image to indicate that the selected segments incorrectly identify the biological units of interest; and

adjusting one or more probability indicators corresponding to the selected segments to indicate that the selected segments incorrectly identify the biological units of interest.

22. The method of claim 18, further comprising:

automatically determining that one or more probability indicators corresponding to the segments in the first image are below a predefined threshold; and

recommending an improvement to the user regarding the first morphological analysis.

23. The method of claim 1, further comprising:

receiving user input, at the user interface, to adjust a transparency of the representation of the first result of the first morphological analysis; and

in response to the user input, adjusting the transparency of the representation of the first result to facilitate quality review of the first result.

24. The method of claim 1, further comprising:

receiving user input, at the user interface, to adjust a contrast and/or a brightness of the first image; and

in response to the user input, adjusting the contrast and/or the brightness of the first image.

25. The method of claim 1, further comprising:

receiving user input, at the user interface, to change a color for representing the first result of the first morphological analysis; and

in response to the user input, changing the color in which the first result is represented on the graphical user interface to facilitate quality review of the first result.

26. The method of claim 1, further comprising:

overlaying, on the first image, a second image of the selected field of view corresponding to the biological tissue such that the first result of the first morphological analysis is overlaid on both the first and second images, the first image of the biological tissue obtained by staining the biological tissue with a first marker prior to capturing the first image, and the second image of the biological tissue obtained by staining the biological tissue with a second marker prior to capturing the second image.

27. The method of claim 26, wherein the rendering of the first and second images creates a composite image on the graphical user interface, and wherein a first contribution of the first image to the composite image is configurable and/or a second contribution of the second image to the composite image is configurable.

28. A computer system for performing quality review of analysis of biological tissue, the system comprising:

a visual display device;

a data storage device for storing a data set comprising tissue profile data including multiplexed biomarker images capturing expression of a plurality of biomarkers in a plurality of fields of view of biological tissue; and

a computer processor coupled to the visual display device and the data storage device, the computer processor programmed to;

render a graphical user interface on the visual display device,

render, on the graphical user interface, a field of view selection component allowing a user to select a field of view from the data set stored on the data storage device, the field of view selection component allowing the user to configure a color and/or a transparency of a representation on the graphical user interface of biological tissue corresponding to a selected field of view,

in response to user input selecting a field of view corresponding to a biological tissue at the field of view selection component, render, on the graphical user interface, a first image of the selected field of view corresponding to the biological tissue,

render, on the graphical user interface, a morphological analysis selection component allowing a user to select a morphological analysis of multiplexed biomarker image data corresponding to the selected field of view, the morphological analysis selection component allowing the user to configure a color and/or a transparency of a representation of a result of a selected morphological analysis,

in response to user input selecting a first morphological analysis at the morphological analysis selection component, overlay, on the first image of the biological tissue, a representation of a first result of the selected first morphological analysis, the first result of the first morphological analysis indicating one or more morphological features of interest in the first image, wherein the first image of the biological tissue and the representation of the first result are displayed in a visually distinguishable manner, and

render, on the graphical user interface, a quality review selection component allowing the user to select a quality review score based on a comparison between the first image and the representation of the first result of the first morphological analysis rendered on the graphical user interface.