Method, apparatus and computer program product for the interactive rendering of multivalued volume data with layered complementary values

• US 7,355,597 B2
• Filed: 05/06/2003
• Issued: 04/08/2008
• Est. Priority Date: 05/06/2002
• Status: Active Grant

• Current Assignee: Brown University
• Original Assignee: Brown University
• Inventors: Laidlaw, David H, Wenger, Andreas
• Primary Examiner(s): Tung; Kee M
• Assistant Examiner(s): YANG, ANDREW GUS
• Application Number:
  US10/431,130
• Publication Number:
  US 20030234781A1
• Time in Patent Office: 1,799 Days
• Field of Search: None
• US Class Current: 345/419
• CPC Class Codes: G06T 15/08 Volume rendering

Abstract

Described is a system, method and computer program product for rendering volumetric multivalued primary data. The system includes a rendering engine having an input coupled to a source of multivalued primary data and an output coupled to a display. The rendering engine includes a data processor for calculating additional data values from the primary data, deriving at least one visual representation from the primary data and the additional data values, and mapping the derived visual representation through transfer functions to hardware primitives for volumetrically rendering the derived visual representation to provide a visualization. The system further includes a user interface for interacting with the visualization. The source of primary data can be a magnetic resonance imaging system, and the primary data may be a multivalued combination obtained from T1 weighted data, T2 weighted data, diffusion data, velocity data, magnetization transfer data, perfusion data, data derived from other imaging modalities (e.g., PET), and simulation data. The primary data can be diffusion tensor data generated by the MRI system from tissue, and the data processor operates to identify directed diffusion paths and to render the directed diffusion paths as thread-like structures. Indications of at least flowing blood velocity and vorticity can also be rendered from MRI data or from simulations.

Claims

1. A method, comprising:

calculating additional data values from multivalued primary data;

deriving at least one volume-based visual representation from the multivalued primary data and the calculated additional data values, the derived at least one volume-based visual representation comprising a layer representing a computed volume-based visual representation that elucidates the underlying data; and

mapping the derived at least one volume-based visual representation through transfer functions to hardware primitives for volumetrically rendering the at least one volume-based visual representation;

where the multivalued primary data comprises diffusion tensor data generated by a magnetic resonance imaging system from tissue, and where deriving comprises identifying directed diffusion paths and rendering the directed diffusion paths as thread-like structures that are made visually distinct from another and from other tissue by the use of a complementary volume disposed around the thread-like structures.

2. A method as in claim 1, further comprising interacting with the volumetrically rendered at least one volume-based visual representation.

3. A method as in claim 1, where the multivalued primary data comprises a multivalued combination obtained from T1 weighted data, T2 weighted data, diffusion data, velocity data, magnetization transfer data, perfusion data and simulation data.

8. A method as in claim 2, where interacting comprises using an exploratory culling volume to interactively control the complexity of a visualized layer.

9. A method as in claim 2, where interacting comprises using at least one of one-dimensional transfer function editing, two-dimensional transfer function editing, and two-dimensional barycentric transfer function editing.

12. A method as in claim 1, where calculating and deriving comprise identifying flow paths, and where mapping comprises visualizing flow streamlines and vorticity lines.

13. A method as in claim 1, where the multivalued primary data further comprises data generated by at least one of a computed tomography scanning system, a positron emission tomography system, a functional MRI, a multi-electrode EEG or MEG, a confocal microscope, a multi-photon microscope and an optical projection tomography system.

14. A method as in claim 1, where mapping comprises generating a plurality of superimposed visual layers from the primary and calculated additional data values, further comprising interactively modifying a displayed representation by varying at least one parameter in at least one of the layers.

15. A method as in claim 1, where mapping comprises generating an ordered sequence of view aligned slices, where the slices are arranged in order from back to front, and where each slice is rendered a plurality of times, once for each volume layer.

16. A method as in claim 15, where a direct-volume-rendered layer is rendered first, followed by other layers containing additional structure.

4. A method, comprising:

calculating additional data values from primary data;

deriving at least one visual representation from the primary data and the additional data values; and

mapping the derived visual representations through transfer functions to hardware primitives for volumetrically rendering the derived at least one visual representation,

where the primary data comprises diffusion tensor data generated by a magnetic resonance imaging system from tissue, where calculating and deriving comprise identifying directed diffusion paths, and where mapping comprises visualizing the directed diffusion paths as thread-like structures.

5. A method as in claim 4, where the tissue comprises a blood vessel, and where at least flowing blood velocity and vorticity are visualized.

6. A method as in claim 4, where the tissue comprises brain tissue.

7. A method as in claim 4, further comprising rendering the thread-like structures so as to make them visually distinct from another and from other tissue by the use of a complementary volume disposed around individual ones of the thread-like structures.

10. A method as in claim 4, where a first lighting model is used to visualize volume layers containing the thread-like structures, and where a second lighting model is used to visualize other volume layers.

11. A method, comprising:

calculating additional data values from the primary data;

deriving at least one visual representation from the primary data and the additional data values;

mapping the derived visual representations through transfer functions to hardware primitives for volumetrically rendering the derived at least one visual representation; and

interacting with the visualization representation,

where the primary data comprises diffusion tensor data generated by a magnetic resonance imaging system from tissue, where calculating and deriving comprise identifying directed diffusion paths, where mapping comprises visualizing the directed diffusion paths as thread-like structures, and where interacting comprises varying a density volume of visualized thread-like structures by varying at least one of a thread-length criterion and a diffusion rate criterion.

17. A system, comprising a rendering engine having an input coupled to a source of multivalued primary data and an output coupled to a display means, said rendering engine comprising a data processor configured to calculate additional data values from the primary data, said data processor further configured to derive at least one volume-based visual representation from the primary data and the additional data values, the derived at least one volume-based visual representation comprising a layer representing a computed volume-based visual representation that elucidates the underlying data, said data processor further configured to map the derived volume-based visual representation through transfer functions to hardware primitives for volumetrically rendering the derived at least one volume-based visual representation to provide a visualization, further comprising a user interface for interacting with the visualization, where the multivalued primary data comprises diffusion tensor data generated by a magnetic resonance imagine system from tissue, and where said data processor is further configured to identify directed diffusion paths and to render the directed diffusion paths as thread-like structures that are made visually distinct from another and from other tissue by the use of a complementary volume disposed around the thread-like structures.

18. A system as in claim 17, where the multivalued primary data comprises a multivalued combination obtained from T1 weighted data, T2 weighted data, diffusion data, velocity data, magnetization transfer data, perfusion data and simulation data.

23. A system as in claim 17, where said user interface employs a exploratory culling volume to interactively control the complexity of a visualized layer.

24. A system as in claim 17, where said user interface displays a plurality of widgets to a user for enabling the user to perform at least one of one-dimensional transfer function editing, two-dimensional transfer function editing, and two-dimensional barycentric transfer function editing of a rendered image.

27. A system as in claim 17, where said processor operates to identify flow paths and to render flow streamlines and vorticity lines.

28. A system as in claim 17, where the primary data further comprises data generated by at least one of a computed tomography scanning system, a positron emission tomography system, a functional MRI, a multi-electrode EEG or MEG, a confocal microscope, a multi-photon microscope and an optical projection tomography system.

29. A system as in claim 17, where said processor operates to generate a plurality of superimposed visual layers from the primary and calculated additional data values, and where said user interface operates with a user to interactively modify a displayed representation by varying at least one parameter in at least one of the layers.

30. A system as in claim 17, where said processor operates to generate an ordered sequence of view aligned slices, where the slices are arranged in order from back to front, and where each slice is rendered a plurality of times, once for each volume layer.

31. A system as in claim 30, where a direct-volume-rendered layer is rendered first, followed by other layers containing finer structure.

32. A system as in claim 17, where said system comprises a computer containing at least one graphics card comprising texture memory and means for providing three dimensional texture processing, texture compression, and programmable texture blending.

19. A system to render volumetric multivalued primary data, comprising a rendering engine having an input coupled to a source of multivalued primary data and an output coupled to a display means, said rendering engine comprising a data processor configured to calculate additional data values from the primary data, to derive at least one visual representation from the primary data and the additional data values and to map the derived visual representation through transfer functions to hardware primitives for use in volumetrically rendering the derived visual representation to provide a visualization,

where the primary data comprises diffusion tensor data generated by a magnetic resonance imaging system from tissue, where said data processor is further configured to identify directed diffusion paths and render the directed diffusion paths as thread-like structures.

20. A system as in claim 19, where the tissue comprises a blood vessel, and where indications of at least flowing blood velocity and vorticity are rendered.

21. A system as in claim 19, where the tissue comprises brain tissue.

22. A system as in claim 19, where said rendering engine renders the thread-like structures so as to make them visually distinct from another and from other tissue by the use of a complementary volume disposed around individual ones of the thread-like structures.

25. A system as in claim 19, where said data processor uses a first lighting model to visualize volume layers containing the thread-like structures and a second lighting model to visualize other volume layers.

26. A system as in claim 19, further comprising a user interface that comprises means for varying a density volume of visualized thread-like structures by varying at least one of a thread-length criterion and a diffusion rate criterion.

33. A computer program product embodied on or in a computer-readable medium, the computer program product comprising computer readable program code comprising:

computer readable program code for calculating additional data values from multivalued primary data;

computer readable program code for deriving at least one volume-based visual representation from the multivalued primary data and the additional data values;

computer readable program code for mapping the derived at least one volume-based visual representation through transfer functions to hardware primitives for volumetrically rendering the derived at least one volume-based visual representation, the derived at least one volume-based visual representation comprising a layer representing a computed volume-based visual representation that elucidates the underlying data; and

computer readable program code providing a user interface function for enabling a user to interact with the volumetrically rendered derived at least one volume-based visual representation;

where the multivalued primary data comprises diffusion tensor data generated by a magnetic resonance imaging system from tissue, and where deriving comprises identifying directed diffusion paths and rendering the directed diffusion paths as thread-like structures that are made visually distinct from another and from other tissue by the use of a complementary volume disposed around the thread-like structures.

34. A computer program product as in claim 33, where the primary data comprises a multivalued combination obtained from T1 weighted data, T2 weighted data, diffusion data, velocity data, magnetization transfer data, perfusion data and simulation data.

39. A computer program product as in claim 33, where said user interface function is operable for enabling a user to employ an exploratory culling volume to interactively control the complexity of a visualized layer.

40. A computer program product as in claim 33, where said user interface function is operable for enabling a user to employ at least one of one-dimensional transfer function editing, two-dimensional transfer function editing, and two-dimensional barycentric transfer function editing.

43. A computer program product as in claim 33, where said computer program code for calculating and deriving comprises code for identifying flow paths, and where said computer program code for mapping comprises code for rendering flow streamlines and vorticity lines.

44. A computer program product as in claim 33, where the primary data further comprises data generated by at least one of a computed tomography scanning system, a positron emission tomography system, a functional MRI, a multi-electrode EEG or MEG, a confocal microscope, a multi-photon microscope and an optical projection tomography system.

45. A computer program product as in claim 33, where said computer program code for mapping comprises code for generating a plurality of superimposed visual layers from the primary and calculated additional data values, further comprising computer program code for interactively modifying a displayed representation by varying at least one parameter in at least one of the layers.

46. A computer program product as in claim 33, where said computer program code for mapping comprises code for generating an ordered sequence of view aligned slices, where the slices are arranged in order from back to front, and where each slice is rendered a plurality of times, once for each volume layer.

47. A computer program product as in claim 46, where a direct-volume-rendered layer is rendered first, followed by other layers containing finer structure.

35. A computer program embodied on or in a computer-readable medium, the computer program product comprising computer readable program code comprising:

computer readable program code for calculating additional data values from the primary data;

computer readable program code for deriving at least one visual representation from the primary data and the additional data values;

computer readable program code for mapping the derived visual representation through transfer functions to hardware primitives for volumetrically rendering the derived visual representation; and

computer readable program code providing a user interface function for enabling a user to interact with the volumetrically rendered derived visual representation,

where the primary data comprises diffusion tensor data generated by a magnetic resonance imaging system from tissue, where said computer program code for calculating and deriving comprises code for identifying directed diffusion paths, and where said computer program code for mapping comprises code for rendering the directed diffusion paths as thread-like structures.

36. A computer program as in claim 35, where the tissue comprises a blood vessel, and where at least flowing blood velocity and vorticity are rendered.

37. A computer program as in claim 35, where the tissue comprises brain tissue.

38. A computer program as in claim 35, operating to render the thread-like structures so as to make them visually distinct from another and from other tissue by the use of a complementary volume disposed around individual ones of the thread-like structures.

41. A computer program as in claim 35, where a first lighting model is used to render volume layers containing the thread-like structures, and where a second lighting model is used to render other volume layers.

42. A computer program as in claim 35, where said user interface function is operable for enabling a user to vary a density volume of rendered thread-like structures by varying at least one of a thread-length criterion and a diffusion rate criterion.

48. A system, comprising an input configured to receive multivalued primary data and an output to provide data for a visual display, further comprising a processor disposed between the input and the output and configured to calculate additional data values from the multivalued primary data, the processor further configured to generate a volume-based representation of the primary data and the additional data values, where the volume-based visual representation comprises a layer representing a computed volume-based visual representation that elucidates the underlying data, the processor further configured to send the volume-based representation to the output to be volumetrically rendered for presentation by the visual display, the processor configured to operate with a user interface for enabling user interaction with the presentation, where the primary data comprises diffusion tensor data generated by a magnetic resonance imaging system from tissue, and where the processor is further configured to identify directed diffusion paths and to render the directed diffusion paths as thread-like structures that are made visually distinct from another and from other tissue by the use of a complementary volume disposed around the thread-like structures.

49. The system as in claim 48, where the processor uses a first lighting model to visualize volume layers containing the thread-like structures and a second lighting model to visualize other volume layers.

50. The system as in claim 48, the user interface enabling control over a density volume of visualized thread-like structures by varying at least one of a thread-length criterion and a diffusion rate criterion.

51. The system as in claim 48, where said user interface comprises a plurality of widgets for enabling the user to perform at least one of one-dimensional transfer function editing, two-dimensional transfer function editing, and two-dimensional barycentric transfer function editing of a rendered image.

52. The system as in claim 48, where the primary data further comprises data generated by at least one of a computed tomography scanning system, a positron emission tomography system, a functional MRI, a multi-electrode EEG or MEG, a confocal microscope, a multi-photon microscope and an optical projection tomography system.

53. The system as in claim 48, where the volume-based representation comprises at least one layer representing a mapping of the additional data values to a volume.

54. The system as in claim 48, where the multivalued primary data comprises flow data, and where said processor is configured at least to identify flow paths and to map identified flow paths to visualize at least one of streamlines and vorticity.