Mutual information based registration of 3D-image volumes on GPU using novel accelerated methods of histogram computation

• US 7,609,884 B1
• Filed: 12/20/2005
• Issued: 10/27/2009
• Est. Priority Date: 12/23/2004
• Status: Active Grant

• Current Assignee: PME IP Pty., Ltd.
• Original Assignee: PME IP Australia Pty Ltd. (Pro Medicus Limited)
• Inventors: Stalling, Detlev, Westerhoff, Malte
• Primary Examiner(s): Hung; Yubin
• Application Number:
  US11/312,837
• Time in Patent Office: 1,407 Days
• Field of Search: 382/131, 382/168, 382/170, 382/171, 382/294, 345/629
• US Class Current: 382/168
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06T 7/35 using statistical methods
• CPC Class Codes: G06V 10/758 Involving statistics of pixels or of feature values, e.g. histogram matching

Abstract

The invention provides, in some aspects, methods and apparatus for registering 3D image volumes by utilizing a graphics processing unit (GPU) to facilitate maximizing the “mutual information” of the two image volumes being registered. Particularly, the GPU is used to compute a histogram representing a probability that a pixel value in one of the images being registered coincides with a pixel value in a transform of the other image. In one aspect, this histogram is generated by a two-step approach including (i) “drawing” a polygon covering a two-component render target having as many pixels as there are samples, where the value of each pixel in the target includes the intensities of the respective pixels of the first and second images, (ii) interpreting that render target as a buffer of vertex positions and, for each sample, rendering into a new render target, a single vertex using coordinates from that buffer.

Claims

1. A digital data processing method of determining mutual information of two images by executing software using a central processing unit (CPU), comprising:

computing a histogram as part of a process of registering a first image and a second image, said histogram representing a probability that pixel values in the first image coincide with pixel values in the second image, said computing step being performed on a graphics processing unit (GPU) using a two-pass approach, including;

in a first pass, drawing a polygon covering a first render target comprising pixels each of which has (i) a first channel representing an image intensity of a corresponding pixel from the first image and (ii) a second channel representing an image intensity of a corresponding pixel from the second image, and

in a second pass, rendering a second render target at vertices corresponding to selected values in pixels of the first render target.

2. The method of claim 1, wherein, at completion of the second pass, the second render target represents the histogram.

3. The method of claim 1, wherein there are as many pixels in the first render target as there are samples over which the probability and the histogram are to be computed.

4. The method of claim 1, wherein the first pass includes storing in a texture on the GPU locations of the samples in the first and second images.

5. The method of claim 4, wherein the texture has a same resolution as the first render target.

6. The method of claim 1, wherein the first pass includes storing the first and second images in respective textures on the GPU.

7. The method of claim 1, wherein the first pass includes utilizing the GPU to draw the polygon by the steps of identifying, for each pixel in the first render target, the location of a corresponding sample point in each of the first and second images, and writing to that pixel in the first render target intensities of those sample points.

8. The method of claim 1, wherein the second pass includes the step of interpreting the first render target as a buffer of vertices.

9. The method of claim 8, wherein the second pass includes setting up the second render target with N×

N pixels and initializing them with zeros, where N represents a maximum binning value for the histogram.

10. The method of claim 9, wherein the second pass further includes rendering, for each sample point over which the probability and histogram are to be computed, a single vertex in the second render target using coordinates from the buffer of vertices.

11. The method of claim 10, comprising enabling additive blending and incrementing by one a pixel value at each rendered vertex.

12. A digital data processing method of determining mutual information of two images by executing software using a central processing unit (CPU), comprising

computing a histogram representing a probability that pixel values in a first image coincide with pixel values in a transformation of a second image by a two-pass approach that includes:

A. a first pass that includes utilizing the graphics processing unit (GPU) that is coupled to the CPU to draw a polygon covering a two-channel first render target comprising pixels each of which has (i) a first channel representing an image intensity of a corresponding pixel from the first image and (ii) a second channel representing an image intensity of a corresponding pixel from the transformation of the second image,

B. the first pass further including (i) storing in a texture on the GPU locations of the samples in the first image and the transformation of the second image, (ii) storing the first image and the transformation of the second image in respective textures on the GPU, (iii) utilizing the GPU to draw the polygon by the steps of identifying, for each pixel in the two-channel first render target, the location of a corresponding sample point in each of the first and second images, and writing to that pixel in the two-channel first render target intensities of those sample points,

C. a second pass that includes utilizing the GPU to render a second render target at vertices corresponding to values in pixels of the two-channel first render target,

D. the second pass further including (i) interpreting the two-channel first render target as a buffer of vertices, (ii) setting up the second render target with N×

N pixels and initializing them with zeros, where N represents a maximum binning value for the histogram, and (iii) rendering, for each sample point over which the probability and histogram are to be computed, rendering a single vertex in the second render target using coordinates from the buffer of vertices, and

E. returning the second render target as representing the histogram.

13. The method of claim 12, wherein the histogram represents a probability that can be expressed as follows:

14. The method of claim 13, comprising determining the mutual information H(A,B∘

T) as a function of the relation;

15. A digital data processing method of determining mutual information of two images by executing software using a central processing unit (CPU), comprising

computing a histogram representing a probability that pixel values in a first image coincide with pixel values in a transformation of a second image by a two-pass approach that includes:

A. a first pass that includes utilizing the graphics processing unit (GPU) that is coupled to the CPU to draw a polygon covering a two-channel first render target comprising pixels each of which has (i) a first channel representing an image intensity of a corresponding pixel from the first image and (ii) a second channel representing an image intensity of a corresponding pixel from the transformation of the second image,

B. the first pass further including (i) storing the first image and the transformation of the second image in respective textures on the GPU. (ii) utilizing the GPU to draw the polygon by the steps of computing with any of a pixel shader and a vertex shader of the GPU a location of a sample point in each of the first image and the transformation of the second image corresponding to each pixel in the two-channel first render target, and writing to that pixel in the two-channel first render target intensities of those sample points,

C. a second pass that includes utilizing the GPU to render a second render target at vertices corresponding to values in pixels of the two-channel first render target,

D. the second pass further including (i) interpreting the two-channel first render target as a buffer of vertices, (ii) setting up the second render target with N×

N pixels and initializing them with zeros, where N represents a maximum binning value for the histogram, and (iii) rendering, for each sample point over which the probability and histogram are to be computed, rendering a single vertex in the second render target using coordinates from the buffer of vertices,

E. returning the second render target as representing the histogram.

16. In a digital data processing method of determining the mutual information of two images by executing software on a central processing unit (CPU), comprising

computing a histogram representing a probability that pixel values in a first image coincide with pixel values in a transformation of a second image by a two-pass approach that includes:

A. a first pass that includes utilizing the graphics processing unit (GPU) that is coupled to the CPU to draw a polygon covering a one-channel render first target,

B. the first pass further including (i) storing in a texture on the GPU locations of the samples in the first image and the transformation of the second image, (ii) storing the first image and the transformation of the second image in respective textures on the GPU;

(iii) utilizing the GPU to draw the polygon by the steps of identifying, for each pixel in the one-channel first render target, the location of a corresponding sample point in each of the first and second images, and writing to that pixel a value expressed as

A(x)*N+B(T(x))

where

A(x) denotes a pixel value at sample location x in the first image,

B(T(x)) denotes a pixel value at sample location x in the transformation of the second image,

N is a maximum binning value for the histogram,

C. a second pass that includes utilizing the GPU to render a further, one-dimensional second render target representing a histogram of the one-channel first render target,

D. interpreting the one-dimensional second render target as a two-dimensional histogram expressed as follows