System and method for GPU-based 3D nonrigid registration

• US 20050190189A1
• Filed: 02/22/2005
• Published: 09/01/2005
• Est. Priority Date: 02/25/2004
• Status: Active Grant

• Current Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Original Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Inventors: Chefd'hotel, Christophe, Saddi, Kinda Anna
• Granted Patent: US 7,483,034 B2
• Time in Patent Office: Days
• US Class Current: 345/501
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration

Abstract

A method of registering two images using a graphics processing unit includes providing a pair of images with a first and second image, calculating a gradient of the second image, initializing a displacement field on the grid point domain of the pair of images, generating textures for the first image, the second image, the gradient, and the displacement field, and loading said textures into the graphics processing unit. A pixel buffer is created and initialized with the texture containing the displacement field. The displacement field is updated from the first image, the second image, and the gradient for one or more iterations in one or more rendering passes performed by the graphics processing unit.

Claims

1. A method of registering two images using a graphics processing unit, said method comprising the steps of:

providing a pair of images with a first image and a second image, wherein said images comprise a plurality of intensities corresponding to a domain of grid points in a D-dimensional space;

calculating a gradient of the second image;

initializing a displacement field on the grid point domain of the pair of images, wherein said displacement field transforms said second image into said first image;

generating textures for the first image, the second image, the gradient, and the displacement field, and loading said textures into the graphics processing unit;

creating a pixel buffer and initializing it with the texture containing the displacement field;

smoothing the displacement field;

updating the displacement field from the first image, the second image, and the gradient for one or more iterations; and

reading the updated displacement field from the pixel buffer.

2. The method of claim 1, wherein creating a pixel buffer further comprises creating a rendering context, initializing the pixel buffer attributes, and generating a new texture where the pixel buffer is bound.

3. The method of claim 1, further comprising the steps of setting texture parameters, and setting uniform variables.

4. The method of claim 1, further comprising creating a rendering context for the image registration, and compiling and loading fragment programs for updating the displacement field into the graphics processing unit.

5. The method of claim 1, further comprising expanding the resolution of the displacement field after reading the displacement field from the pixel buffer.

6. The method of claim 5, wherein expanding the resolution includes linearly interpolating displacement field values for points between grid points in the displacement field domain.

7. The method of claim 1, wherein the updating of the displacement field is a rendering pass performed by the graphics processing unit according to the equation

u_k+1=u_k+δ

[(I₁−

I₂·

(Id+u_k))∇

I₂·

(Id+u_k)]+α

Δ

u_k,

wherein u_k and u_k+1 are the displacement field and updated displacement field, respectively, I₁ is the first image, I₂ is the second image, ∇

I₂ is the gradient, Id is the identity map, Δ

u_k is the Laplacian of the displacement field, α

is a regularization parameter, and δ

is the time step.

8. The method of claim 7, wherein the domain of grid points is a 3-dimensional space, and wherein the displacement field is converted from 3 dimensions to 2 dimensions for the rendering pass according to the transformations

9. The method of claim 8, wherein the diaplacement field is converted from a 2-dimensional grid to a 3-dimensional grid according to the transformations

10. The method of claim 7, wherein the Laplacian of the displacement field u_k is defined in 3-dimensions as

Δ

u_k(x,y,z)=u_k(x+1,y,z)+u_k(x−

1,y,z)+u_k(x,y+1,z)+u_k(x,y−

1,z)+u_k(x,y,z+1)+u_k(x,y,z−

1)−

6.0×

u_k(x,y,x),

and in 2-dimensions as

Δ

u_k(p,q)=u_k(p+1,q)+u_k(p−

1,q)+u_k(p,q+1)+u_k(p,q−

1)+u_k(r,s)+u_k(t,u)−

60.0×

u_k(p,q),

wherein (x, y, z) is a 3-dimensional grid point, (p, q) is a 2-dimensional grid point, R_x is the number of columns, R_y is the number of rows, R_z is the number of slices, and N is an integer defined as N={square root}{square root over (R_z)}, if the result is an integer, N={square root}{square root over (2×

R_z)}, otherwise,

11. The method of claim 7, wherein the boundary conditions of the 3-dimensional displacement field are clamp-to-edge, and further comprising controlling said boundary conditions of the 2-D displacement field by imposing a 2D mask, wherein said mask is a precomputed vector comprising 7 coefficients (a,b,c,d,e,f,g) that multiply each term of the displacement field according to the equation,

a×

u_k(x+1,y)+b×

u_k(x−

1,y)+c×

u_k(x,y+1)+d×

u_k(x,y−

1)+e×

u_k(r,s)+f×

u_k(t,u)+g×

u_k(x,y)

wherein u_k(x,y) is the 2D displacement field, (a,b,c,d,e,f) are either 0 or 1, wherein 0 indicates a displacement field value outside the boundary, and 1 a point inside the boundary, g can vary between −

6 and 0 depending on the number of nearest neighbor interior points about (x,y).

18. The computer readable program storage device of claim 1, wherein the method further comprises expanding the resolution of the displacement field after reading the displacement field from the pixel buffer.

19. The computer readable program storage device of claim 18, wherein expanding the resolution includes linearly interpolating displacement field values for points between grid points in the displacement field domain.

12. A method of registering two images, said method comprising the steps of:

providing a graphics processing unit;

providing a pair of images with a first image and a second image, wherein said images comprise a plurality of intensities corresponding to a domain of grid points in a 3-dimensional space;

calculating a gradient of the second image;

initializing a displacement field on the grid point domain of the pair of images, wherein said displacement field transforms said second image into said first image;

generating textures for the first image, the second image, the gradient, and the displacement field, and loading said textures into the graphics processing unit;

creating a pixel buffer, including creating a rendering context, initializing the pixel buffer attributes, generating a new texture where the pixel buffer is bound and initializing it with the texture containing the displacement field;

updating the displacement field from a rendering pass performed by the graphics processing unit according to the equation

u_k+1=u_k+δ

[(I₁−

I₂·

(Id+u_k))∇

I₂·

(Id+u_k)]+α

Δ

u_k,

wherein u_k and u_k+1 are the displacement field and updated displacement field, respectively, I₁ is the first image, I₂ is the second image, ∇

I₂ is the gradient, Id is an identity map, Δ

u_k is the Laplacian of the displacement field, α

is a regularization parameter, and δ

is the time step; and

expanding the resolution of the displacement field by linearly interpolating displacement field values for points between grid points in the displacement field domain.

13. The method of claim 12, further comprising smoothing the displacement field and reading the updated displacement field from the pixel buffer.

14. A program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform the method steps for registering two images using a graphics processing unit, said method comprising the steps of:

providing a pair of images with a first image and a second image, wherein said images comprise a plurality of intensities corresponding to a domain of grid points in a D-dimensional space;

calculating a gradient of the second image;

initializing a displacement field on the grid point domain of the pair of images, wherein said displacement field transforms said second image into said first image;

generating textures for the first image, the second image, the gradient, and the displacement field, and loading said textures into the graphics processing unit;

creating a pixel buffer and initializing it with the texture containing the displacement field;

smoothing the displacement field;

updating the displacement field from the first image, the second image, and the gradient for one or more iterations; and

reading the updated displacement field from the pixel buffer.

15. The computer readable program storage device of claim 14, wherein creating a pixel buffer further comprises creating a rendering context, initializing the pixel buffer attributes, and generating a new texture where the pixel buffer is bound.

16. The method of claim 14, further comprising the steps of setting texture parameters, and setting uniform variables.

17. The computer readable program storage device of claim 14, wherein the method further comprises creating a rendering context for the image registration, and compiling and loading fragment programs for updating the displacement field into the graphics processing unit.

20. The computer readable program storage device of claim 14, wherein the updating of the displacement field is a rendering pass performed by the graphics processing unit according to the equation

u_k+1=u_k+δ

[(I₁−

I₂·

(Id+u_k))∇

I₂·

(Id+u_k)]+α

Δ

u_k,

wherein u_k and u_k+1 are the displacement field and updated displacement field, respectively, I₁ is the first image, I₂ is the second image, ∇

I₂ is the gradient, Id is the identity map, Δ

u_k is the Laplacian of the displacement field, α

is a regularization parameter, and δ

is the time step.

21. The computer readable program storage device of claim 20, wherein the domain of grid points is a 3-dimensional space, and wherein the displacement field is converted from 3 dimensions to 2 dimensions for the rendering pass according to the transformations

22. The method of claim 21, wherein the diaplacement field is converted from a 2-dimensional grid to a 3-dimensional grid according to the transformations

23. The computer readable program storage device of claim 20, wherein the Laplacian of the displacement field u_k is defined in 3-dimensions as

Δ

u_k(x,y,z)=u_k(x+1,y,z)+u_k(x−

1,y,z)+u_k(x,y+1,z)+u_k(x,y−

1,z)+u_k(x,y,z+1)+u_k(x,y,z−

1)−

6.0×

u_k(x,y,x),

and in 2-dimensions as

Δ

u_k(p,q)=u_k(p+1,q)+u_k(p−

1,q)+u_k(p,q+1)+u_k(p,q−

1)+u_k(r,s)+u_k(t,u)−

60.0×

u_k(p,q),

wherein (x, y, z) is a 3-dimensional grid point, (p, q) is a 2-dimensional grid point, R_x is the number of columns, R_y is the number of rows, R_z is the number of slices, and N is an integer defined as N={square root}{square root over (R_z)}, if the result is an integer, N={square root}{square root over (2×

R_z)}, otherwise,

24. The method of claim 20, wherein the boundary conditions of the 3-dimensional displacement field are clamp-to-edge, and further comprising controlling said boundary conditions of the 2-D displacement field by imposing a 2D mask, wherein said mask is a precomputed vector comprising 7 coefficients (a,b,c,d,e,f,g) that multiply each term of the displacement field according to the equation,

a×

u_k(x+1,y)+b×

u_k(x−

1,y)+c×

u_k(x,y+1)+d×

u_k(x,y−

1)+e×

u_k(r,s)+f×

u_k(t,u)+g×

u_k(x,y)

wherein u_k(x,y) is the 2D displacement field, (a,b,c,d,e,f) are either 0 or 1, wherein 0 indicates a displacement field value outside the boundary, and 1 a point inside the boundary, g can vary between −

6 and 0 depending on the number of nearest neighbor interior points about (x,y).