Method and system for motion correction in a sequence of images

• US 20060045366A1
• Filed: 08/17/2005
• Published: 03/02/2006
• Est. Priority Date: 08/31/2004
• Status: Active Grant

• Current Assignee: Siemens Healthineers AG (Siemens AG)
• Original Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Inventors: Chefd'hotel, Christophe
• Granted Patent: US 7,440,628 B2
• Time in Patent Office: Days
• US Class Current: 382/236
• CPC Class Codes: G06T 7/207 for motion estimation over a hierarchy of resolutions multi-resolution motion estimation or hierarchical motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/53
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

A method for motion compensation between first and second images in a temporal sequence includes processing the first and second images in a reduction process for providing respective reduced resolution first and second images; deriving respective first and second feature maps from the respective reduced resolution first and second images, the feature maps including deriving the respective Laplacian of image data in the respective reduced resolution first and second images; deriving a displacement field by processing the first and second feature maps in accordance with a registration algorithm, the registration algorithm comprising solving, for each picture element or voxel, a local Gaussian weighted least mean square problem so as to derive respective vectors forming the displacement field; and warping the second image with the displacement field.

Claims

1. A method for motion compensation between first and second images in a temporal sequence, said method comprising:

deriving respective first and second feature maps from said first and second images;

deriving a displacement field by processing said first and second feature maps in accordance with a registration algorithm, said registration algorithm comprising solving, for each picture element or voxel, a local Gaussian weighted least mean square problem so as to derive respective vectors forming said displacement field; and

warping said second image with said displacement field.

2. A method for motion compensation in accordance with claim 1, wherein said step of deriving a displacement field utilizes a previously derived displacement field for deriving said displacement field.

3. A method for motion compensation in accordance with claim 2, wherein said step of deriving a displacement field utilizes a given initial displacement field for an initial derivation of said displacement field.

4. A method for motion compensation in accordance with claim 3, wherein:

a default condition for said given initial displacement field is a null set.

5. A method for motion compensation in accordance with claim 3, wherein:

said given initial displacement field takes into account prior knowledge of a patient'"'"'s motion.

6. A method for motion compensation in accordance with claim 2, comprising repeating the step of deriving a displacement field, wherein each repetition is performed on first and second feature maps corresponding to said first and second images having higher resolutions than for the previous repetition.

7. A method for motion compensation in accordance with claim 2, wherein each repetition utilizes a previously derived displacement field from the immediately preceding step.

8. A method for motion compensation in accordance with claim 7, wherein said step of deriving a displacement field utilizes a given displacement field for an initial derivation of said displacement field.

9. A method for motion compensation in accordance with claim 7, wherein:

said step of deriving a displacement field comprises a step of expanding said previously derived displacement field to the resolution level of said increasing resolution versions.

10. A method for motion compensation in accordance with claim 7, including a step of utilizing a displacement field derived at the highest resolution present for warping said second image to produce a motion corrected image.

11. A method for motion compensation in accordance with claim 7, wherein:

said step of deriving respective first and second feature maps comprises deriving the respective Laplacian of image data in said respective reduced resolution first and second images.

12. A method for motion compensation between first and second images in a temporal sequence comprises:

processing said first and second images in a reduction process for providing respective reduced resolution first and second images;

deriving respective first and second feature maps from said respective reduced resolution first and second images, said feature maps including deriving a respective Laplacian of image data in said respective reduced resolution first and second images;

deriving a displacement field by processing said first and second feature maps in accordance with a registration algorithm, said registration algorithm comprising solving, for each picture element or voxel, a local Gaussian weighted least mean square problem so as to derive respective vectors forming the displacement field; and

warping the second image with said displacement field.

13. A method for motion compensation between first and second images in a temporal sequence, said method comprising:

processing said first and second images in a reduction process for providing respective reduced resolution first and second images;

deriving respective first and second feature maps from said respective reduced resolution first and second images;

deriving a displacement field by processing said first and second feature maps in accordance with a registration algorithm, said registration algorithm comprising solving, for each picture element or voxel, a local Gaussian weighted least mean square problem so as to derive respective vectors forming said displacement field; and

warping said second image with said displacement field.

14. A method for motion compensation in accordance with claim 13, wherein:

said step of deriving respective first and second feature maps comprises deriving the respective Laplacian of image data in said respective reduced resolution first and second images.

15. A method for motion compensation in accordance with claim 13, wherein:

said step of deriving a displacement field utilizes an initial displacement field.

16. A method for motion compensation in accordance with claim 15, wherein:

a default condition for said initial displacement field is a null set.

17. A method for motion compensation in accordance with claim 15, wherein:

said initial displacement field takes into account prior knowledge of a patient'"'"'s motion.

18. A method for motion compensation in accordance with claim 13, wherein:

said step of processing said first and second feature maps in accordance with a registration algorithm comprises a step of expanding said displacement field to a resolution level compatible with that of said second image.

19. A method for motion compensation between a reference image and a floating image in a temporal sequence, said method comprising:

deriving a first set of modified images of progressively decreasing resolution from said reference image;

deriving a second set of modified images of progressively decreasing resolution from said floating image;

deriving a first set of feature maps from said first set of modified images;

deriving a second set of feature maps from said second set of modified images;

deriving a first displacement field from the lowest resolution members of each of the first and second sets of feature maps, respectively, and a given initial displacement field, in accordance with a registration algorithm, said registration algorithm comprising solving, for each picture element or voxel, a local Gaussian weighted least mean square problem so as to derive respective vectors forming said displacement field;

deriving a second displacement field from the next to the lowest resolution members of each of the first and second sets of feature maps, respectively, and said first displacement field obtained in the preceding step, in accordance said registration algorithm;

repeating the foregoing step for successively higher resolution members, if any are present, of each of the first and second sets of feature maps, respectively and using in each case the displacement field obtained in the step preceding the current step, until the resolution of said floating image is reached and, if no higher resolution members are present, then proceeding directly to the next step; and

warping said floating image with the last obtained displacement field.

20. A method for motion compensation in accordance with claim 19, wherein:

said steps of deriving feature maps comprises deriving the Laplacian of image data in respective images.

21. A method for motion compensation in accordance with claim 19, wherein:

said steps of deriving a displacement field comprise a step of expanding said displacement field to a resolution level compatible with that of the next higher resolution level, if expansion is required.

22. A method for motion compensation in accordance with claim 19, wherein:

a default condition for said given initial displacement field is a null set.

23. A method for motion compensation in accordance with claim 19, wherein:

said given initial displacement field takes into account prior knowledge of a patient'"'"'s motion.

24. A method for motion compensation between a reference image and a floating image in a temporal sequence, said method comprising:

(a) processing said reference and floating images in respective first and second pluralities of cascaded resolution reduction processes for providing respective pluralities of successively reduced resolution reference and floating images, herein referred to as Level 0 for the lowest resolution level and Level 1 for the next higher resolution level, Level 2 for the second next higher resolution level, and so forth for any existing higher resolution levels;

(b) deriving respective pluralities of reference and floating feature maps from said respective pluralities of successively reduced resolution reference and floating images, at resolution levels L 0, L 1, L 2 and so forth;

(c) deriving a first displacement field by processing a reference and a floating feature map, corresponding to level L 0, in accordance with a registration algorithm, said registration algorithm comprising solving, for each picture element or voxel, a local Gaussian weighted least mean square problem based on a given initial displacement field, so as to derive respective vectors forming said first displacement field;

(d) expanding said first displacement field to a resolution level compatible with that of resolution L 1, to provide an expanded first displacement field;

(e) deriving a second displacement field by processing a reference and a floating feature map, corresponding to resolution level L 1, in accordance with said registration algorithm based on said expanded first displacement field, so as to derive respective vectors forming said second displacement field;

(f) expanding said second displacement field to a resolution level compatible with that of resolution L 1, to provide an expanded second displacement field;

(g) deriving a third displacement field by processing a reference and a floating feature map, corresponding to resolution level L 2, in accordance with said registration algorithm based on said expanded second displacement field, so as to derive respective vectors forming said second displacement field; and

(h) if L 2 is the resolution level of said reference and floating images, then warping said floating image by utilizing said third displacement field and ending; and

if not, then

(i) expanding said third displacement field to a resolution level compatible with the next higher resolution level to provide an expanded third displacement field, and

(j) repeating the sequence of steps beginning with step (g) with appropriate modification of resolution levels for obtaining a displacement field corresponding to increasingly higher levels of resolution based on the last previously obtained expanded displacement field until the resolution level of said reference and floating images is reached and thereupon warping said floating image with the final displacement field obtained and ending.

25. A method for motion compensation in accordance with claim 24, wherein:

said step of providing respective pluralities of successively reduced resolution reference and floating images comprises deriving the respective Laplacian of image data in said respective pluralities of successively reduced resolution reference and floating images.

26. A method for motion compensation in accordance with claim 24, wherein:

said step of deriving a first displacement field comprises inputting a given initial displacement field.

27. A method for motion compensation between first and second images in a temporal sequence, said method comprising:

deriving respective first and second feature maps from respective reduced resolution first and second images;

deriving a first displacement field by processing said first and second feature maps in accordance with a registration algorithm, said registration algorithm comprising solving, for each picture element or voxel, a local Gaussian weighted least mean square problem so as to derive respective vectors forming said displacement field;

expanding said first displacement field to correspond with the resolution of said first and second images;

deriving a second displacement field by processing said first and second images in accordance with said registration algorithm based on said first displacement field; and

warping said second image with said second displacement field so as to obtain a motion corrected image.

28. A method for motion compensation in accordance with claim 27, wherein:

said step of deriving a first displacement field comprises a step of inputting a given initial displacement field.

29. A method for motion compensation in accordance with claim 28, wherein:

a default condition for said given initial displacement field is a null set.

30. A method for motion compensation in accordance with claim 28, wherein:

said given initial displacement field takes into account prior knowledge of a patient'"'"'s motion.

31. A method for motion compensation in accordance with claim 27, wherein:

said step of deriving respective first and second feature maps comprises deriving the respective Laplacian of image data in said respective reduced resolution first and second images.

32. A method for motion compensation between first and second images in a temporal sequence, said method comprising:

deriving from said first image a first succession of images having progressively reduced resolution;

deriving from said second image a second succession of images having progressively reduced resolution;

deriving from said first succession of images a first succession of feature maps;

deriving from said second succession of images a second succession of feature maps;

deriving a succession of displacement fields by processing feature maps of said first succession of feature maps with corresponding feature maps of said second succession of feature maps, pertaining to the same resolution, in accordance with a registration algorithm for providing a respective displacement field, said registration algorithm comprising solving, for each picture element or voxel, a local Gaussian weighted least mean square problem so as to derive respective vectors forming said displacement field, starting with a given displacement field being used for deriving a first displacement field at the lowest resolution level and thereafter utilizing the immediately previous displacement field for deriving the next displacement field corresponding to the next higher resolution in said succession; and

warping said second image utilizing a final displacement field derived at the resolution level of said first and second images.

33. A method for motion compensation in accordance with claim 32, comprising:

expanding said each resolution field, except for said given field and said final field to correspond with the next higher resolution.

34. A method for motion compensation between a reference image and a floating image in a temporal sequence, said method comprising:

(a) processing said reference and floating images in respective first and second pluralities of cascaded resolution reduction processes for providing respective pluralities of successively reduced resolution reference and floating images, herein referred to as Level 0 for the lowest resolution level and Level 1 for the next higher resolution level, Level 2 for the second next higher resolution level, and so forth for any existing higher resolution levels;

(b) deriving respective pluralities of reference and floating feature maps from said respective pluralities of successively reduced resolution reference and floating images, at resolution levels L 0, L 1, L 2, in order of increasing resolution with L2 being the resolution level of said reference and floating images;

(c) deriving a first displacement field by processing a reference and a floating feature map, corresponding to level L 0, in accordance with a registration algorithm, said registration algorithm comprising solving, for each picture element or voxel, a local Gaussian weighted least mean square problem based on a given initial displacement field, so as to derive respective vectors forming said first displacement field;

(d) expanding said first displacement field to a resolution level compatible with that of resolution L 1, to provide an expanded first displacement field;

(e) deriving a second displacement field by processing a reference and a floating feature map, corresponding to resolution level L 1, in accordance with said registration algorithm based on said expanded first displacement field, so as to derive respective vectors forming said second displacement field;

(f) expanding said second displacement field to a resolution level compatible with that of resolution L 2, to provide an expanded second displacement field;

(g) deriving a final displacement field by processing a reference and a floating feature map, corresponding to resolution level L 2, in accordance with said registration algorithm based on said expanded second displacement field, so as to derive respective vectors forming said final displacement field; and

(h) warping said floating image by utilizing said final displacement field.

35. A method for motion compensation in accordance with claim 34, wherein:

a default condition for said given initial displacement field is a null set.

36. A method for motion compensation in accordance with claim 34, wherein:

said given initial displacement field takes into account prior knowledge of a patient'"'"'s motion.

37. A method for motion compensation in accordance with claim 34, wherein:

said step of deriving feature maps comprises deriving the respective Laplacian of image data in said respective reduced resolution first and second images.

38. A method for motion compensation in accordance with claim 34, wherein step (c), deriving a first displacement field, includes the steps of:

labeling each voxel by an index j, whereof coordinates are given by a vector x_j ε

R³;

defining p_j^k ε

R³ as the displacement vector recovered at point x_j after the k^th iteration of said algorithm;

given a set of displacement vectors p_j^k, denoting a corrected version of said floating feature map at iteration k by J_2,k;

∀

j, J_2,k(x_j)=J₂(x_j+p_j^k);

defining at each voxel j, an update rule p_j^k+1=p_j^k+s_j^k with s_j^k as a solution of the non-linear least mean square problem;

$\underset{s_j^k}{\min }<br><br>\underset{i}{\sum <br><br>}<br><br>G_{\mathrm{\sigma <br><br>}}<br><br>\left(x_j-x_i\right)·<br><br>{\left(z_i^k<br><br>\left(s_j^k\right)\right)}^2$

where z_i^k(s)=J₁(x_i)−

J_2,k(x_i+s) and G_σ

(x) is an isotropic tri-dimensional Gaussian kernel;

$G_{\mathrm{\sigma <br><br>}}<br><br>\left(x\right)=\frac{1}{\sqrt{{\left(2<br><br>\mathrm{\pi <br><br>}\right)}^3<br><br>{\mathrm{\sigma <br><br>}}^6}}<br><br>{\mathrm{e<br><br>}}^{-\frac{{<br><br>x<br><br>}^2}{{\mathrm{\sigma <br><br>}}^2}}$

of standard deviation σ

;

solving a linearized version of said non-linear least mean square problem by minimizing the following criterion;

$E<br><br>\left(s_j^k\right)=\underset{i}{\sum <br><br>}<br><br>G_{\mathrm{\sigma <br><br>}}<br><br>\left(x_j-x_i\right)·<br><br>{\left(z_i^k<br><br>\left(0\right)+\nabla <br><br>{z_i^k<br><br>\left(0\right)}^T·<br><br>s_j^k\right)}^2$

where ∇

z_i^k(s) is defined as ∇

z_i^k(s)=−

∇

J_2,k(x_i+s);

computing the first variation of E(s_j^k), the necessary condition of optimality

∀

h, dE(s_j^k)·

h=0

to yield a closed-form solution for s_j^k;

$s_j^k=-{\left(\underset{i}{\sum <br><br>}<br><br>G_{\mathrm{\sigma <br><br>}}<br><br>\left(x_j-x_i\right)<br><br>\nabla <br><br>z_i^k<br><br>\left(0\right)<br><br>\nabla <br><br>{z_i^k<br><br>\left(0\right)}^T\right)}^{-1}<br><br>\left(\underset{i}{\sum <br><br>}<br><br>G_{\mathrm{\sigma <br><br>}}<br><br>\left(x_j-x_i\right)<br><br>\nabla <br><br>z_i^k<br><br>\left(0\right)<br><br>z_i^k<br><br>\left(0\right)\right).$

39. A system for performing image motion compensation, comprising:

a memory device for storing a program and other data; and

a processor in communication with said memory device, said processor being operative with said program to perform;

a method for motion compensation between first and second images in a temporal sequence, said method comprising;

processing said first and second images in a reduction process for providing respective reduced resolution first and second images;

deriving respective first and second feature maps from said respective reduced resolution first and second images;

deriving a displacement field by processing said first and second feature maps in accordance with a registration algorithm, said registration algorithm comprising solving, for each picture element or voxel, a local Gaussian weighted least mean square problem so as to derive respective vectors forming said displacement field; and

warping said second image with said displacement field.

40. A computer program product comprising a computer useable medium having computer program logic recorded thereon for program code for performing image motion compensation by:

a method for motion compensation between first and second images in a temporal sequence, said method comprising;

processing said first and second images in a reduction process for providing respective reduced resolution first and second images;

deriving respective first and second feature maps from said respective reduced resolution first and second images;

deriving a displacement field by processing said first and second feature maps in accordance with a registration algorithm, said registration algorithm comprising solving, for each picture element or voxel, a local Gaussian weighted least mean square problem so as to derive respective vectors forming said displacement field; and

warping said second image with said displacement field.

41. A method for motion compensation between first and second images in a temporal sequence, said method comprising:

deriving a displacement field by processing said first and second images in accordance with a process, said process including a registration algorithm, said registration algorithm comprising solving, for each picture element or voxel, a local Gaussian weighted least mean square problem so as to derive respective vectors forming said displacement field; and

warping said second image with said displacement field.

42. A method for motion compensation in accordance with claim 41, wherein said step of processing said first and second images in accordance with a process includes a step of deriving respective first and second feature maps from said first and second images and applying said registration algorithm to said feature maps, respectively.

43. A method for motion compensation in accordance with claim 41, wherein said step of deriving a displacement field utilizes a previously derived displacement field for deriving said displacement field.

44. A method for motion compensation in accordance with claim 41, wherein said step of deriving a displacement field utilizes a given initial displacement field for an initial derivation of said displacement field.