Medical Image Rendering

• US 20110228997A1
• Filed: 03/17/2010
• Published: 09/22/2011
• Est. Priority Date: 03/17/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Siddiqui, Khan Mohammad, Criminisi, Antonio, Sharp, Toby
• Granted Patent: US 9,256,982 B2
• Time in Patent Office: Days
• US Class Current: 382/131
• CPC Class Codes: G06T 15/30 Clipping
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2210/12 Bounding box
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes

Abstract

Medical image rendering is described. In an embodiment a medical image visualization engine receives results from an organ recognition system which provide estimated organ centers, bounding boxes and organ classification labels for a given medical image. In examples the visualization engine uses the organ recognition system results to select appropriate transfer functions, bounding regions, clipping planes and camera locations in order to optimally view an organ. For example, a rendering engine uses the selections to render a two-dimensional image of medical diagnostic quality with minimal user input. In an embodiment a graphical user interface populates a list of organs detected in a medical image and a clinician is able to select one organ and immediately be presented with the optimal view of that organ. In an example opacity of background regions of the medical image may be adjusted to provide context for organs presented in a foreground region.

Claims

1. A computer-implemented method comprising:

at an input receiving at least one estimated organ center obtained from a medical image, each estimated organ center having an associated organ classification label;

using a user interface to display an organ name for each estimated organ center;

at the user interface, receiving user input specifying at least one of the organ names;

arranging a visualization engine to select a transfer function according to the specified organ name;

arranging the visualization engine to select a camera location according to the estimated organ center of the specified organ name;

using a rendering engine to render a two-dimensional image from the medical image using the selected camera location and transfer function; and

using the user interface to display the two-dimensional image such that a foreground region associated with the specified organ name is visualized.

2. A method as claimed in claim 1 comprising at the input, receiving an organ classification for each image element of the medical image;

arranging the visualization engine to select at least one other transfer function using the image element organ classifications; and

using the rendering engine to render the two-dimensional image using the transfer functions according to the image element organ classifications.

3. A method as claimed in claim 1 comprising at the input, receiving an organ classification and associated probability for each image element of the medical image;

arranging the rendering engine to render the two-dimensional image such that the opacity of each image element of the two-dimensional image is related to the organ classification probabilities of the associated medical image elements.

4. A method as claimed in claim 1 comprising at the input, receiving an organ classification and associated probability for each image element of the medical image and arranging the rendering engine to render the two-dimensional image such that the color of each image element of the two-dimensional image is related to the organ classifications of the associated medical image elements.

5. A method as claimed in claim 1 comprising arranging the visualization engine to select a bounding region and to center that bounding region on the estimated organ center of the specified organ name;

and to provide that centered bounding region to the rendering engine for use when rendering the two-dimensional image.

6. A method as claimed in claim 5 comprising arranging the visualization engine to select the bounding region using the specified organ name and training data.

7. A method as claimed in claim 1 comprising arranging the visualization engine to select a clipping plane and to provide that clipping plane to the rendering engine for use when rendering the two-dimensional image.

8. A method as claimed in claim 1 wherein using the rendering engine comprises using a ray-casting engine to render the two-dimensional image by computing rays extending from the camera location through the two-dimensional image and into the medical image and carrying out integration along those rays.

9. A method as claimed in claim 1 wherein receiving user input comprises receiving a single-click specifying at least one of the organ names.

10. A method as claimed in claim 1 wherein the medical image is selected from any of:

a magnetic resonance image scan, a computed tomography scan, a single photon emission computed tomography image, positron emission tomography image, DTI tractography image and ultra-sound scan.

11. A medical image visualization engine comprising:

an input arranged to receive a plurality of estimated organ centers obtained from a medical image of at least three dimensions, each estimated organ center having an associated organ classification label;

a user interface arranged to display an organ name for each estimated organ center;

the user interface also arranged to receive user input specifying at least one of the organ names;

arranging a visualization engine to select a transfer function according to the specified organ name;

a processor arranged to select a camera location according to the estimated organ center of the specified organ name;

a rendering engine arranged to render a two-dimensional image from the medical image using the selected camera location and transfer function; and

the user interface being arranged to display the two-dimensional image such that a foreground region associated with the specified organ name is visualized.

12. An apparatus as claimed in claim 11 wherein the user interface is arranged to display the two-dimensional image in a display window that is substantially centered on the estimated organ center of the specified organ name.

13. An apparatus as claimed in claim 11 wherein the input is arranged to receive an organ classification for each image element of the medical image;

the visualization engine is arranged to select at least one other transfer function using the image element organ classifications; and

the rendering engine is arranged to render the two-dimensional image using the transfer functions according to the image element organ classifications.

14. An apparatus as claimed in claim 11 wherein the input is arranged to receive an organ classification and associated probability for each image element of the medical image;

and the rendering engine is arranged to render the two-dimensional image such that the opacity of each image element of the two-dimensional image is related to the organ classification probabilities of the associated medical image elements.

15. An apparatus as claimed in claim 11 wherein the input is arranged to receive an organ classification and associated probability for each image element of the medical image;

and the rendering engine is arranged to render the two-dimensional image such that the color of each image element of the two-dimensional image is related to the organ classifications of the associated medical image elements.

16. An apparatus as claimed in claim 11 wherein the visualization engine is arranged to select a bounding region and to center that bounding region on the estimated organ center of the specified organ name;

and to provide that centered bounding region to the rendering engine for use when rendering the two-dimensional image.

17. An apparatus as claimed in claim 11 wherein the visualization engine is arranged to select a clipping plane and to provide that clipping plane to the rendering engine for use when rendering the two-dimensional image.

18. An apparatus as claimed in claim 11 wherein the rendering engine comprises a ray-casting engine arranged to render the two-dimensional image by computing rays extending from the camera location through the two-dimensional image and into the medical image and carrying out integration along those rays.

19. A computer-implemented method comprising:

at an input receiving at least one estimated organ center obtained from a medical image of at least three dimensions, each estimated organ center having an associated organ classification label; and

also receiving an organ classification probability for each image element of the medical image;

using a user interface to display an organ name for each estimated organ center;

at the user interface, receiving user input specifying at least one of the organ names;

arranging a visualization engine to select a transfer function according to the specified organ name;

arranging the visualization engine to select a camera location according to the estimated organ center of the specified organ name; and

using a rendering engine to render a two-dimensional image from the medical image using the selected camera location and transfer function and such that the opacity of the image elements of the two-dimensional image are related to the organ classification probabilities.

20. A method as claimed in claim 19 which comprises arranging the user interface to display the two-dimensional image such that it is substantially centered on the estimated organ center of the specified organ name.