Method and apparatus for visualizing three-dimensional and higher-dimensional image data sets

• US 8,189,002 B1
• Filed: 10/31/2005
• Issued: 05/29/2012
• Est. Priority Date: 10/29/2004
• Status: Active Grant

• Current Assignee: PME IP Pty., Ltd.
• Original Assignee: PME IP Australia Pty Ltd. (Pro Medicus Limited)
• Inventors: Westerhoff, Malte, Stalling, Detlev, Thieret, Scott A.
• Primary Examiner(s): Hajnik, Daniel F
• Assistant Examiner(s): Prendergast, Roberta D
• Application Number:
  US11/263,577
• Time in Patent Office: 2,402 Days
• Field of Search: 345/426, 345/589, 345/506
• US Class Current: 345/426
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06T 2210/52 Parallel processing

Abstract

In one aspect, the invention provides improvements in a digital data processor of the type that renders a three-dimensional (3D) volume image data into a two-dimensional (2D) image suitable for display. The improvements include a graphics processing unit (GPU) that comprises a plurality of programmable vertex shaders that are coupled to a plurality of programmable pixel shaders, where one or more of the vertex and pixel shaders are adapted to determine intensities of a plurality of pixels in the 2D image as an iterative function of intensities of sample points in the 3D image through which a plurality viewing rays associated with those pixels are passed. The pixel shaders compute, for each ray, multiple iteration steps of the iterative function prior to computing respective steps for a subsequent ray.

Claims

1. An apparatus for computed tomography wherein a three-dimensional (3D) volume image data set is rendered into a two-dimensional (2D) displayed image, comprising

a graphics processing unit (GPU) comprising a plurality of programmable vertex shaders that are coupled to a plurality of programmable pixel shaders,

one or more of the vertex shaders and the pixel shaders being adapted to determine a color of each pixel in the 2D image as an iterative function of intensities and gradients at a plurality of sample points in the 3D image through which a ray associated with a particular pixel is passed, where the calculation of the iterative function for a given sample point along a ray associated with that particular pixel is based on a result of calculations of the iterative function for other sample points on that ray,

wherein at least one of the pixel shaders computes a gradient at a sample point along a ray based on differences in intensities at a plurality of points in the 3D image data neighboring that sample point;

said at least one of the pixel shaders computes the gradient at the sample point along the ray in a second coordinate system rotated relative to a first coordinate system in which the 3D image data is represented, wherein at least one axis of the second coordinate system aligned along the ray; and

said at least one of the pixel shaders subsequently rotates the gradient computed in the second coordinate system back to the first coordinate system.

2. The apparatus of claim 1, the further improvement wherein the at least one pixel shader computes the gradient by employing any of central differences or one-sided differences computational techniques.

3. The apparatus of claim 1, wherein the one or more pixel shaders store the 2D image in an off-screen buffer.

4. The apparatus of claim 3, wherein the one or more pixel shaders effect the display of the buffered 2D image.

5. The apparatus of claim 4, wherein the one or more pixel shaders apply selected filtering operations to the buffered 2D image.

6. The apparatus of claim 1, wherein the at least one of the pixel shaders computes the gradient “

on-the-fly”

while rendering the 3D image.

7. The apparatus of claim 1, wherein at least one of the pixel shaders determines the intensity of at least one of the sample point along the ray by interpolating values of a plurality of neighboring points in the 3D image data.

8. A digital data processor wherein a three-dimensional (3D) volume image data represented in an (x, y, z) coordinate system is rendered into a two-dimensional (2D) displayed image represented in a (u, v) coordinate system, comprising

a graphics processing unit (GPU) comprising a plurality of programmable vertex shaders that are coupled to a plurality of programmable pixel shaders,

one or more of the pixel shaders being adapted to determine a color of each pixel in the 2D image as an iterative function of intensities and gradients at a plurality of sample points in the 3D image through which a ray associated with that pixel is passed, where the calculation of the iterative function for a given sample point along a ray associated with that particular pixel is based on a result of calculations of the iterative function for other sample points on that ray,

for each ray, the pixel shaders rotating the (x, y, z) coordinate system so as to align an axis thereof with that ray and generating gradients at the sample points along that ray in the rotated coordinate system relative to the (x, y, z) coordinate system,

said pixel shaders transform the gradients generated in the rotated coordinate system back to the (x, y, z) coordinate system.

9. The apparatus of claim 8, wherein the one or more vertex shaders compute each ray based on a viewing point and a viewing direction.

10. The apparatus of claim 8, wherein for at least one ray, a single invocation of a pixel shader computes a full set of the iteration steps corresponding to all sample points along that ray.

11. The apparatus of claim 8, wherein for each ray, one or more pixel shaders test whether sample points along one portion of that ray lie in the 3D image and one or more pixel shaders compute iteration steps at sample points along another portion of the ray without such testing.

12. A digital data processor wherein a three-dimensional (3D) volume image is rendered into a two-dimensional (2D) displayed image, comprising

a graphics processing unit (GPU) comprising a plurality of programmable vertex shaders that are coupled to a plurality of programmable pixel shaders,

one or more of the pixel shaders being adapted to determine intensities of a plurality of pixels in the 2D image as an iterative function of intensities of sample points in the 3D image through which a plurality of rays associated with those pixels are passed, where, for each pixel, the calculation of the iterative function for a given sample point along a ray associated with that particular pixel is based on a result of calculations of the iterative function for other sample points on that ray,

wherein the GPU stores at least a portion of data representing the 2D image in an off-screen buffer,

for each ray, the pixel shaders rotating a first coordinate system in which the 3D image is represented so as to align an axis thereof with that ray and computing gradients at sample points along that ray in the rotated coordinate system relative to the first coordinate system;

said pixel shaders transform the gradients computed in the rotated coordinate system back to the first coordinate system in which the 3D image is represented.

13. The digital data processor of claim 12, wherein the GPU effects display of the 2D image data stored in the off-screen buffer.

14. The digital data processor of claim 12, wherein the GPU generates the 2D image intensities by executing instructions implemented on the GPU via a first application programming interface (API).

15. The digital data processor of claim 14, wherein the GPU generates the 2D image effects display of the stored 2D image data by executing instructions implemented thereon via a second application programming interface (API).

16. A method for rendering a three-dimensional (3D) volume image data set into a two-dimensional (2D) displayed image, comprising:

determining by one or more pixel shaders in a graphics processing unit (GPU) a color of each pixel in the 2D image as an iterative function of intensities and gradients at a plurality of sample points in the 3D image through which a ray associated with that pixel is passed, wherein the iterative function for a given sample point along the ray associated with that particular pixel is based on a result of calculations of the iterative function for other sample points on that ray;

for each ray, rotating by one or more pixel shaders in the GPU a first coordinate system in which the 3D image is represented so as to align an axis thereof with that ray;

computing by one or more pixel shaders in the GPU a gradient at a sample point along that ray in the rotated coordinate system relative to the first coordinate system; and

transforming by one or more pixel shaders in the GPU the gradient computed in the rotated coordinate system back to the first coordinate system in which the 3D image is represented.

17. The method of claim 16, further comprising storing the 2D image in an off-screen buffer.

18. The method of claim 17, further comprising effecting the display of the buffered 2D image.

19. The method of claim 18, further comprising applying selected filtering operations to the buffered 2D image.