Method and apparatus for 4-dimensional image warping

• US 20050031210A1
• Filed: 08/05/2004
• Published: 02/10/2005
• Est. Priority Date: 08/08/2003
• Status: Active Grant

• Current Assignee: Trustees Of The University Of Pennsylvania (University of Pennsylvania)
• Original Assignee: Trustees Of The University Of Pennsylvania (University of Pennsylvania)
• Inventors: Shen, Dinggang, Davatzikos, Christos
• Granted Patent: US 7,430,323 B2
• Time in Patent Office: Days
• US Class Current: 382/215
• CPC Class Codes: G06T 2207/10076 4D tomography; Time-sequential 3D tomography
• CPC Class Codes: G06T 2207/20128 Atlas-based segmentation
• CPC Class Codes: G06T 2207/30016 Brain
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/149 involving deformable models, e.g. active contour models
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

A method of atlas mapping and an apparatus for image mapping are provided. The method includes generating a four dimensional template, linearly transforming the four dimensional representation of a subject to the four dimensional template, generating attribute vectors for a plurality of points of the linearly transformed four dimensional representation of the subject and of the four dimensional template, the plurality of points consisting of a plurality of temporally successive voxels of the four dimensional representation of the subject and of the four dimensional template, evaluating the generated attribute vectors to identify points in the four dimensional template that correspond to the four dimensional representation of the subject, and deforming the four dimensional template to the four dimensional representation of the subject according to relative positions of the corresponding points of the linearly transformed four dimensional representation of the subject and of the four dimensional template.

Claims

1. A method of atlas mapping rigidly aligned temporally successive N-1 dimensional images of a subject, which form a N dimensional representation of the subject, the method comprising:

generating a N dimensional template;

generating attribute vectors for a plurality of points of the N dimensional representation of the subject and of the N dimensional template, the plurality of points corresponding to a plurality of temporally successive voxels of the N dimensional representation of the subject and of the N dimensional template;

evaluating the generated attribute vectors to identify points in the N dimensional template that correspond to respective points in the N dimensional representation of the subject; and

deforming the N dimensional template to the N dimensional representation of the subject according to relative positions of the respective corresponding points of the N dimensional representation of the subject and of the N dimensional template.

2. The method according to claim 1, further comprising:

selecting points in the four dimensional template according to measures of distinctiveness among the evaluated attribute vectors of the four dimensional template, wherein the deforming of the four dimensional is based on the selected points.

3. The method according to claim 1, wherein the four dimensional template is a four dimensional representation of a reference and corresponds to the four dimensional representation of the subject.

4. The method according to claim 1, wherein the four dimensional template is an atlas of a brain.

5. The method according to claim 1, wherein N is either 3 or 4.

6. The method according to claim 1, further comprising:

linearly transforming the N dimensional representation of the subject to the N dimensional template such that the linearly transformed N dimensional representation of the subject is used to generate the attribute vectors.

7. A method of atlas mapping of rigidly aligned temporally successive three dimensional images of a subject, which form a four dimensional representation of the subject, the method comprising:

generating a four dimensional template;

linearly transforming the four dimensional representation of the subject to the four dimensional template;

determining attribute vectors for a plurality of points of the linearly transformed four dimensional representation of the subject and of the four dimensional template, the plurality of points corresponding to a plurality of temporally successive voxels of the four dimensional representation of the subject and of the four dimensional template;

establishing correspondences between the four dimensional representation of the subject and the four dimensional template according to similarities between the determined attribute vectors the plurality of points of the linearly transformed four dimensional representation of the subject and of the four dimensional template;

deforming a plurality of four dimensional subvolumes located around respective points of the four dimensional template according to relative positions of the corresponding points of the linearly transformed four dimensional representation of the subject and of the four dimensional template to provide a deformed four dimensional template; and

smoothing the deformed four dimensional template.

8. The method according to claim 7, wherein the determining of the attribute vectors comprises:

determining respective edge types, image intensities, wavelet expansion coefficients, and geometric moment invariants of the plurality of points of the linearly transformed four dimensional representation of the subject and of the four dimensional template.

9. The method according to claim 7, wherein the determining of the geometric moment invariants for the plurality of points comprises:

determining a plurality of four dimensional subvolumes located around respective ones of the plurality of points of the linearly transformed four dimensional representation of the subject and of the four dimensional template;

calculating predetermined parameters that are invariant to rotation for the respective ones of the plurality of points of the linearly transformed four dimensional representation of the subject and of the four dimensional template according to the determining of the four dimensional subvolumes; and

repeating the determining of the plurality of four dimensional subvolumes and the calculating of the predetermined parameters operations while adjusting the four dimensional subvolumes in size.

10. The method according to claim 7, wherein the determining of the attribute vectors for the plurality of points of the linearly transformed four dimensional representation of the subject and of the four dimensional template comprises:

selecting the plurality of points according to at least one of user input, of measures of dissimilarity between the selected points and neighboring points or of boundary types of the selected points;

determining a four dimensional subvolume located around each selected point;

calculating for each selected point, predetermined parameters corresponding to the selected point that are invariant to spatial rotation of the selected point according to the determined four dimensional subvolume; and

repeating the determining of the four dimensional subvolume and the calculating of the predetermined parameters operations while adjusting the four dimensional subvolume in size.

11. The method according to claim 9, wherein the establishing of the correspondences between the linear transformed four dimensional representation of the subject and the four dimensional template comprises:

evaluating one or more measures of similarity of attribute vectors for each selected point of the four dimensional template to attribute vectors of selected points of the linearly transformed four dimensional representation of the subject; and

matching a respective, selected point of the four dimensional template to a respective, selected point of the linearly transformed four dimensional representation of the subject, when the respective one or more measures of similarity of the respective, selected point of the four dimensional template to the respective, selected point of the linearly transformed four dimensional representation of the subject are largest among the evaluated one or more measures of similarity, and each of the respective one or more measures of similarity is respectively larger than one or more predetermined threshold values.

12. The method according to claim 11, wherein the establishing of the correspondences between the linear transformed four dimensional representation of the subject and the four dimensional template further comprises:

minimizing the following cost function;

E_F+E_B+α

·

E_S^Spatial+β

·

E_S^Temporal

where E_F is one measure of dissimilarity of a respective attribute vector of the four dimensional subvolume of the four dimensional template to a respective attribute vector of the four dimensional subvolume of the four dimensional representation of the subject, and includes at least one of the one or more measures of similarity, E_B is another measure of dissimilarity of a respective attribute vector of the four dimensional subvolume of the four dimensional representation of the subject to a respective attribute vector of the four dimensional subvolume of the four dimensional template, and includes at least another one of the one or more measures of similarity, α and

β

are two weighting parameters, E_S^Spatial is a spatial smoothing component, and E_S^Temporal is a temporal smoothing component.

13. The method according to claim 12, wherein E_F which is the one measure of dissimilarity comprises:

$E_F=\underset{t=1}{\overset{N}{\sum <br><br>}}<br><br>\underset{x\in <br><br>V_T}{\sum <br><br>}<br><br>{\mathrm{\omega <br><br>}}_T<br><br>\left(x,t\right)<br><br>\left(\frac{\underset{\left(z,\mathrm{\tau <br><br>}\right)\in <br><br>n<br><br>\left(x,t\right)}{\sum <br><br>}<br><br>{\mathrm{\varepsilon <br><br>}}_T<br><br>\left(z,\mathrm{\tau <br><br>}\right)<br><br>\left(1-m<br><br>\left(a_T<br><br>\left(z,\mathrm{\tau <br><br>}\right),a_S<br><br>\left(h<br><br>\left(z,\mathrm{\tau <br><br>}\right)\right)\right)\right)}{\underset{\left(z,\mathrm{\tau <br><br>}\right)\in <br><br>n<br><br>\left(x,t\right)}{\sum <br><br>}<br><br>{\mathrm{\varepsilon <br><br>}}_T<br><br>\left(z,\mathrm{\tau <br><br>}\right)}\right)$

where a_T(z,τ

) is the attribute vector of the template point (z,τ

) a_S(h(z,τ

)) is the attribute vector of the corresponding subject point h(z,τ

), 1−

m(a_T(z,τ

), a_S(h(z,τ

))) is a difference measure of the corresponding attribute vectors of the four dimensional template and of the four dimensional representation of the subject, ε

_T(z,τ

) is a weighting parameter for the points of the four dimensional subvolume of the four dimensional template, and ω

_T(x,t) is an importance parameter of a respective template point (x,t).

14. The method according to claim 12, wherein E_B which is the other one of the measures of dissimilarity comprises:

$E_B=\underset{t=1}{\overset{N}{\sum <br><br>}}<br><br>\underset{x\in <br><br>V_S}{\sum <br><br>}<br><br>{\mathrm{\omega <br><br>}}_S<br><br>\left(x,t\right)<br><br>\left(\frac{\underset{\left(z,\mathrm{\tau <br><br>}\right)\in <br><br>n<br><br>\left(x,t\right)}{\sum <br><br>}<br><br>{\mathrm{\varepsilon <br><br>}}_S<br><br>\left(z,\mathrm{\tau <br><br>}\right)<br><br>\left(1-m<br><br>\left(a_T<br><br>\left(h^{-1}<br><br>\left(z,\mathrm{\tau <br><br>}\right)\right),a_S<br><br>\left(z,\mathrm{\tau <br><br>}\right)\right)\right)}{\underset{\left(z,\mathrm{\tau <br><br>}\right)\in <br><br>n<br><br>\left(x,t\right)}{\sum <br><br>}<br><br>{\mathrm{\varepsilon <br><br>}}_S<br><br>\left(z,\mathrm{\tau <br><br>}\right)}\right)$

where a_S(h(z,τ

)) is the attribute vector of the subject point h(z,τ

), a_T(h<sup>−

1</sup>((z,τ

)) is the attribute vector of the corresponding template point (z,τ

) for the inverse transformation, 1−

m(a_γ

(h<sup>−

1</sup>(z,τ

)), a_S(z,τ

) is a difference measure of the corresponding attribute vectors of the four dimensional representation of the subject and of the four dimensional template in the inverse transformation, ε

_S(z,τ

) is a weighting parameter for the points of the four dimensional subvolume of the four dimensional representation of the subject, and ω

_S(x,t) is an importance parameter of a respective subject point (x,t).

15. The method according to claim 12, wherein E_S^Spatial comprises:

$E_S^{\mathrm{Spatial}}=\underset{t=1}{\overset{N}{\sum <br><br>}}<br><br>\underset{x\in <br><br>V_T}{\sum <br><br>}<br><br>{<br><br>u<br><br>\left(x,t\right)-\frac{\underset{z\in <br><br>n<br><br>\left(x\right)}{\sum <br><br>}<br><br>g_1<br><br>\left(<br><br>z-x<br><br>\right)<br><br>u<br><br>\left(z,t\right)}{\underset{z\in <br><br>n<br><br>\left(x\right)}{\sum <br><br>}<br><br>g_1<br><br>\left(<br><br>z-x<br><br>\right)}<br><br>}^2$ $\mathrm{and}<br><br>\text{\hspace{1em}<br><br>}<br><br>E_S^{\mathrm{Temporal}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\text{comprises;<br><br>}$ $E_S^{\mathrm{Temporal}}=\underset{x\in <br><br>V_T}{\sum <br><br>}<br><br>\underset{t=1}{\overset{N}{\sum <br><br>}}<br><br>{<br><br>u<br><br>\left(x,t\right)-\frac{\underset{\mathrm{\tau <br><br>}\in <br><br>n<br><br>\left(t\right)}{\sum <br><br>}<br><br>g_2<br><br>\left(\mathrm{\tau <br><br>}-t\right)<br><br>u<br><br>\left(x,\mathrm{\tau <br><br>}\right)}{\underset{\mathrm{\tau <br><br>}\in <br><br>n<br><br>\left(t\right)}{\sum <br><br>}<br><br>g_2<br><br>\left(\mathrm{\tau <br><br>}-t\right)}<br><br>}^2$

where u(x,t) is a displacement field, g₁ is a Gaussian filter to obtain an average displacement in a three dimension spatial subvolume, and g₂ is a second Gaussian filter to obtain an average displacement in a one dimensional temporal subvolume.

16. A method of atlas mapping of temporally successive three dimensional images of a subject, which form a four dimensional representation of the subject, the method comprising:

generating a four dimensional template;

establishing correspondences between neighboring three dimensional images of the subject according to determined attribute vectors for the plurality of voxels of plural temporally successive three dimensional images of the subject;

rigidly aligning the plural temporally successive three dimensional images of the subject according to the established correspondences between the neighboring three dimensional images of the subject, the rigidly aligned plural temporally successive three dimensional images being the four dimensional representation of the subject;

linearly transforming the four dimensional representation of the subject to the four dimensional template;

establishing respective correspondences between the four dimensional representation of the subject and the four dimensional template according to similarities of determined attribute vectors for a plurality of points of the linearly transformed four dimensional representation of the subject and of the four dimensional template, the plurality of points consisting of a plurality of temporally successive voxels of the four dimensional representation of the subject and of the four dimensional template;

deforming four dimensional subvolumes located around the points of the four dimensional template according to relative positions of points of the four dimensional representation of the subject and the respective points of the four dimensional template to provide a deformed four dimensional template; and

smoothing the deformed four dimensional template.

17. The method according to claim 16, further comprising:

selecting the four dimensional subvolumes located around respective points of the four dimensional template according to measures of distinctiveness between the evaluated attribute vectors of the four dimensional template, wherein the deforming of the four dimensional template is based on the selected four dimensional subvolumes.

18. The method according to claim 16, wherein the linear transformation of the four dimensional representation of the subject to the four dimensional template is according to an affine transformation.

19. The method according to claim 16, wherein the generation of the four dimensional template, the establishment of correspondences between neighboring three dimensional images of the subject, the rigid alignment of the plural temporally successive three dimensional images of the subject, the linear transformation of the four dimensional representation of the subject, the establishment of the correspondences between the four dimensional representation of the subject and the four dimensional template, the deformation of the four dimensional subvolumes and the smoothing of the deformed four dimensional template operations automatically occur without human intervention.

20. A method of atlas mapping of temporally successive three dimensional images of a subject, which form a four dimensional representation of subject, the method comprising the steps of:

a) generating a four dimensional template representing a reference and corresponding to the four dimensional representation of the subject;

b) determining spatial attribute vectors for a plurality of voxels for plural temporally successive three dimensional images of the subject;

c) selecting respective voxels based on the determined spatial attribute vectors for the plurality of voxels;

d) establishing correspondences between neighboring three dimensional images of the subject according to the selected, respective voxels for the plural temporally successive three dimensional images of the subject;

e) rigidly aligning the plural temporally successive three dimensional images of the subject according to the established correspondences between the neighboring three dimensional images of the subject to form the four dimensional representation of the subject;

f) linearly transforming the four dimensional representation of the subject to the four dimensional template;

g) determining spatiotemporal attribute vectors for a plurality of points of the linearly transformed four dimensional representation of the subject and the four dimensional template, the plurality of points consisting of a plurality of temporally successive voxels of the four dimensional representation of the subject and of the four dimensional template;

h) selecting a plurality of subject active points from among the points of the linearly transformed four dimensional representation of the subject, and a plurality of template active points from among the points of the four dimensional template according to the determined spatiotemporal attribute vectors;

i) establishing correspondences between the four dimensional representation of the subject and the four dimensional template according to similarities between the selected plurality of subject active points and template active points;

j) deforming a plurality of four dimensional subvolumes, located around respective, selected ones of the template active points according to relative positions of respective, selected ones of the subject active points to provide a deformed four dimensional template, when respective correspondences are established between the respective, selected active points of the linearly transformed four dimensional representation of the subject and the four dimensional template;

k) smoothing the deformed four dimensional template using a Gaussian filter; and

l) repeating the steps g through j, while increasing a number of the selected active points in the linearly transformed four dimensional representation of the subject and the deformed four dimensional template.

21. The method according to claim 20, wherein the repeating of the steps h through k comprises:

iteratively repeating steps g through j until all of the points of the deformed four dimensional template are selected as the active points thereof and all of the points of the linearly transformed four dimensional representation of the subject are selected as the active points thereof.

22. The method according to claim 20, wherein a size of the plurality of four dimensional subvolumes is decreased after each repetition of steps g through j until the plurality of the four dimensional subvolumes includes only immediately neighboring active points.

23. A computer readable carrier including program code for causing a computer to implement a method of atlas mapping rigidly aligned temporally successive N-1 dimensional images of a subject, which form a N dimensional representation of the subject, the method comprising:

generating a N dimensional template;

generating attribute vectors for a plurality of points of the N dimensional representation of the subject and of the N dimensional template, the plurality of points corresponding to a plurality of temporally successive voxels of the N dimensional representation of the subject and of the N dimensional template;

evaluating the generated attribute vectors to identify points in the N dimensional template that correspond to respective points in the N dimensional representation of the subject; and

deforming the N dimensional template to the N dimensional representation of the subject according to relative positions of the respective corresponding points of the N dimensional representation of the subject and of the N dimensional template.