Method and system for motion compensation in a temporal sequence of images
First Claim
Patent Images
1. A method for motion compensation in first and second images in a temporal sequence by computing a deformation function, said method comprising:
- using a computer to performing a conjugate gradient maximization of a similarity measure between said first and second images; and
deriving said deformation function by utilizing said gradient maximization in conjunction with composition of a plurality of regularized gradients of said similarity measure such that said deformation function exhibits essentially no singularities,wherein said step of deriving said deformation function comprises applying an algorithm to update a displacement field between said images, comprising;
field between said images, comprising;
wherein U0 denotes a current estimation of said displacement field, id denotes an identity function, k denotes an index, Vk denotes a regularized gradient of said similarity measure, operator Gσ
* denotes convolution by a Gaussian kernel, S denotes the similarity measure, ∇
S denotes the gradient of S with respect to φ
, I1;
→
R denotes said first image, I2;
→
R denotes said second image, φ
;
Ω
→
R3 denotes said deformation such that said similarity measure is maximized, ∘
denotes composition of functions, and ε
k is the step size, which is made sufficiently small to ensure invertibility.
3 Assignments
0 Petitions
Accused Products
Abstract
A method and system for performing motion compensation in a temporal sequence of images include performing a conjugate gradient maximization of a similarity measure between two images, based on the local cross-correlation of corresponding regions to obtain a displacement field for warping one of the images. The non-singularity of the deformation is ensured by utilizing a composition of regularized gradients of the similarity measure when building the solution.
13 Citations
65 Claims
-
1. A method for motion compensation in first and second images in a temporal sequence by computing a deformation function, said method comprising:
-
using a computer to performing a conjugate gradient maximization of a similarity measure between said first and second images; and deriving said deformation function by utilizing said gradient maximization in conjunction with composition of a plurality of regularized gradients of said similarity measure such that said deformation function exhibits essentially no singularities, wherein said step of deriving said deformation function comprises applying an algorithm to update a displacement field between said images, comprising; field between said images, comprising; wherein U0 denotes a current estimation of said displacement field, id denotes an identity function, k denotes an index, Vk denotes a regularized gradient of said similarity measure, operator Gσ
* denotes convolution by a Gaussian kernel,S denotes the similarity measure, ∇
S denotes the gradient of S with respect to φ
,I1;
→
R denotes said first image,I2;
→
R denotes said second image,φ
;
Ω
→
R3 denotes said deformation such that said similarity measure is maximized,∘
denotes composition of functions, andε
k is the step size, which is made sufficiently small to ensure invertibility.- View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)
-
-
17. A method for performing image motion compensation between a reference image and an initial floating image by computing a deformation function from an initial deformation function, said method comprising:
-
(a) setting said initial deformation function as a current deformation function; (b) warping said initial floating image with said current deformation function so as to obtain a current warped floating image; (c) using a computer to compute a current similarity measure between said reference image and said current warped floating image; (d) computing using a computer to compute the current gradient of said current similarity measure with respect to said current deformation function; (e) regularizing said current gradient so as to ensure its invertibility to obtain a current regularized gradient; (f) (A) in the first performance of this step, setting said current regularized gradient as current conjugate gradient, and (B) in subsequent iterations of this step, conjugating said current regularized gradient with said current conjugate gradient so as to obtain a subsequent conjugate gradient; (g) composing said subsequent conjugate gradient with said current deformation function so as to obtain a subsequent deformation function; (h) setting said current deformation function as said subsequent deformation function; (i) setting said current conjugate gradient as said subsequent conjugate gradient; (j) going to step (b) and repeating until reaching a predetermined stop criterion; and (k) defining the current warped floating image as a final warped floating image. - View Dependent Claims (18, 19)
-
-
20. A method for performing image motion compensation between a reference image and an initial floating image by computing a deformation function from an initial deformation function, said method comprising:
-
(a) acquiring reference and initial floating images in a time sequence; (b) setting said initial deformation function as a current deformation function; (c) warping said initial floating image with said current deformation function so as to obtain a current warped floating image; (d) computing using a computer to compute a current similarity measure between said reference image and said current warped floating image; (e) using a computer to compute the current gradient of said current similarity measure with respect to said current deformation function; (f) regularizing said current gradient so as to ensure its invertibility to obtain a current regularized gradient; (g) setting said current regularized gradient as a current conjugate gradient and conjugating said current regularized gradient with said current conjugate gradient so as to obtain a subsequent conjugate gradient; (h) composing said subsequent conjugate gradient with said current deformation function so as to obtain a subsequent deformation function; (i) setting said current deformation function as said subsequent deformation function; (j) setting said current conjugate gradient as said subsequent conjugate gradient; (k) warping said initial floating image with said current deformation function so as to obtain a current warped floating image; (l) computing a current similarity measure between said reference image and said current warped floating image; (m) computing the current gradient of said current similarity measure with respect to said current deformation function; (n) regularizing said current gradient so as to ensure its invertibility to obtain a current regularized gradient; (o) conjugating said current regularized gradient with said current conjugate gradient so as to obtain a subsequent conjugate gradient; (p) composing said subsequent conjugate gradient with said current deformation function so as to obtain a subsequent deformation function; (q) setting said current deformation function as said subsequent deformation function; (r) setting said current conjugate gradient as said subsequent conjugate gradient; (s) going to step (k) and repeating until a predetermined stop criterion is reached; and (t) defining the current warped floating image as a motion-corrected final warped floating image. - View Dependent Claims (21, 22, 23, 24, 25, 26, 27, 28)
-
-
29. A method for performing image motion compensation, comprising:
-
acquiring a reference image and an initial floating image; warping said initial floating image with a given initial deformation function so as to obtain a current warped floating image; using a computer to compute local cross-correlation between respective corresponding regions in said reference image and said warped floating image; performing a conjugate gradient maximization of a similarity measure based on said local cross-correlation so as to derive a deformation function for which said similarity measure is maximal, by utilizing a combination of a conjugate gradient optimization with a composition of displacements which are small as compared with the size of said images whereby fast convergence is achieved while ensuring invertibility so that said deformation function stays non-singular; and warping said floating image in accordance with said deformation function to obtain a motion-corrected image; wherein said step of deriving a deformation function comprises applying an algorithm to update a displacement filed between said images, comprising; wherein U0 denotes a current estimation of said displacement field, id denotes an identity function, k denotes an index, Vk denotes a regularized gradient of said similarity measure, operator Gσ
* denotes convolution by a Gaussian kernel,S denotes the similarity measure, ∇
S denotes the gradient of S with respect to φ
,I1;
→
R denotes said first image,I2;
→
R denotes said second image,φ
;
Ω
→
R3 denotes said deformation such that said similarity measure is maximized, and ∘
denotes composition of functions, andε
k is the step size, which is made sufficiently small to ensure invertibility.- View Dependent Claims (30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44)
-
-
45. A computer readable medium having a computer program logic recorded thereon for program code for performing image motion compensation, comprising by:
-
inputting a given temporal sequence of images, I1, . . . I2, . . . In; selecting an image of said temporal sequence with index k as a reference image; starting with i=1, then if i is not equal to k, performing a multi-resolution motion correction between Ik and I by performing a conjugate gradient maximization of a similarity measure between first and second images in said temporal sequence; deriving a deformation function by utilizing said gradient maximization in conjunction with composition of a plurality of regularized gradients of said similarity measure such that said deformation function exhibits essentially no singularities; warping one of said first and second images by said deformation function; following such correction, defining J1 as an output of the foregoing step of performing a multi-resolution motion correction until; in the event that i is equal to k, defining Ji as Ik and incrementing i by 1, comparing the incremented result with n; in the event the incremented result is less than or equal to n, continuing to repeat the foregoing steps until the incremented result is greater than n, and thereupon terminating said steps, thereby resulting in a series of respective motion-corrected images, J1, J2 . . . Jn. - View Dependent Claims (46, 47, 48, 49, 50, 51)
where ν
1,2(X), ν
1(x) and ν
2(x) are respectively the covariance and variances of the intensities of I1 and I2 around x;
whereof the first order variation defines a gradient given bywherein function Lcc is estimated as and all required space dependent quantities including μ
1(x) are computed by convolution with a Gaussian kernel.
-
-
49. The computer readable medium in accordance with claim 46, wherein said step of determining said deformation function comprises:
-
assigning the initial value of k as zero; warping F with U to obtain F; computing the similarity measure between U and F; defining G as the gradient of the similarity measure with respect to Uk; if GU is not zero and k is equal to or greater than 1, then conjugating GUk with GUk−
1;determining, using the conjugation of GUk with GUk−
1, whether an optimal step has been found;if an optimal step has been found, then updating Uk with GUk; determining whether k≧
kmax;if k≧
kmax,the process is ended, and if not,then k is incremented by 1; using the incremented value of k, warping F with Uk to obtain FU following again with the foregoing steps for computing the similarity measure, and so forth; and if GU is equal to zero, then ending the process, whereby the output is an improved version Uk of the deformation function.
-
-
50. The computer readable medium in accordance with claim 46, wherein said step of constructing said multi-resolution pyramid comprises:
reducing only the size of said images in constructing said pyramid.
-
51. The computer readable medium in accordance with claim 50, wherein said step of reducing only the size of said images in constructing said pyramid comprises:
using a low-pass filter designed for reducing in half the sampling frequency with minimal loss of information.
-
52. A method for performing image motion compensation between images in a temporal sequence by computing a deformation function from an initial deformation function, said method comprising:
-
(a) acquiring a reference image and an initial floating image; (b) setting said initial deformation function as a current deformation function; (c) warping said initial floating image with said current deformation function so as to obtain a current warped floating image; (d) computing using a computer to compute a current similarity measure between said reference image and said current warped floating image; (e) using a computer to compute the current gradient of said current similarity measure with respect to said current deformation function; (f) regularizing said current gradient so as to ensure its invertibility to obtain a current regularized gradient; (g) (A) in the first performance of this step, setting said current regularized gradient as current conjugate gradient, and (B) in subsequent iterations of this step, conjugating said current regularized gradient with said current conjugate gradient so as to obtain a subsequent conjugate gradient; (h) composing said subsequent conjugate gradient with said current deformation function so as to obtain a subsequent deformation function; (i) setting said current deformation function as said subsequent deformation function;
(j) setting said current conjugate gradient as said subsequent conjugate gradient;(k) going to step (c) and repeating until a predetermined stop criterion is reached; and (I) defining the current warped floating image as a final warped floating image. - View Dependent Claims (53, 54, 55, 56, 57, 58, 59)
-
-
60. A system for performing image motion compensation, comprising:
-
a memory device for storing a program and other data; and a processor in communication with said memory device, said processor operative with said program to perform; (a) setting an initial deformation function as a current deformation function; (b) warping said initial floating image with said current deformation function so as to obtain a current warped floating image; (c) computing a current similarity measure between said reference image and said current warped floating image; (d) computing the current gradient of said current similarity measure with respect to said current deformation function; (e) regularizing said current gradient so as to ensure its invertibility to obtain a current regularized gradient; (f) (A) in the first performance of this step, setting said current regularized gradient as current conjugate gradient, and (B) in subsequent iterations of this step, conjugating said current regularized gradient with said current conjugate gradient so as to obtain a subsequent conjugate gradient; (g) composing said subsequent conjugate gradient with said current deformation function so as to obtain a subsequent deformation function; (h) setting said current deformation function as said subsequent deformation function; (i) setting said current conjugate gradient as said subsequent conjugate gradient; (j) going to step (b) and repeating until reaching a predetermined stop criterion; and (k) defining the current warped floating image as a final warped floating image. - View Dependent Claims (61, 62)
-
-
63. A computer readable medium having a computer program logic recorded thereon for program code for performing image motion compensation by:
-
(a) setting an initial deformation function as a current deformation function; (b) warping an initial floating image with said current deformation function so as to obtain a current warped floating image; (c) computing a current similarity measure between a reference image and said current warped floating image; (d) computing the current gradient of said current similarity measure with respect to said current deformation function; (e) regularizing said current gradient so as to ensure its invertibility to obtain a current regularized gradient; (f) (A) in the first performance of this step, setting said current regularized gradient as current conjugate gradient, and (B) in subsequent iterations of this step, conjugating said current regularized gradient with said current conjugate gradient so as to obtain a subsequent conjugate gradient; (g) composing said subsequent conjugate gradient with said current deformation function so as to obtain a subsequent deformation function; (h) setting said current deformation function as said subsequent deformation function; (i) setting said current conjugate gradient as said subsequent conjugate gradient; (j) going to step (b) and repeating until reaching a predetermined stop criterion; and (k) define the current warped floating image as a final warped floating image. - View Dependent Claims (64, 65)
-
Specification