SYSTEM AND METHOD FOR DETERMINING CONVERGENCE OF IMAGE SET REGISTRATION

• US 20060165267A1
• Filed: 05/16/2002
• Published: 07/27/2006
• Est. Priority Date: 10/15/2001
• Status: Active Grant

• Current Assignee: Cloud Software Group
• Original Assignee: Insightful Corporation (Cloud Software Group)
• Inventors: Ng, Lydia, Wyman, Bradley
• Granted Patent: US 7,106,891 B2
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06T 7/38 Registration of image sequences

Abstract

Computer-based methods and systems for automatically determining convergence when registering image sets are provided. Example embodiments provide an Enhanced Image Registration System (EIRS), which includes an Image Comparison Module, a Transformation Optimizer, and a Convergence Calculator. When the EIRS receives two image sets to align, the Image Comparison Module compares two image sets to determine or measure how closely the image sets are aligned. The Transformation Optimizer determines an appropriate transformation to apply to one of the image sets to align it with the reference image set. The Transformation Optimizer then applies the determined transformation. The Convergence Calculator examines one or more points within the transformed image set to determine when convergence is attained.

Claims

1. A method for registering a plurality of image sets, wherein each of the plurality of image sets comprises at least one image, and wherein each of the plurality of image sets contain spatially overlapping areas of an imaged subject with at least one of the remaining plurality of the image sets, the method comprising:

selecting a reference image set and an evaluation image set from said plurality of image sets, wherein the evaluation image set is to be aligned with the reference image set;

selecting a methodology for comparing of the registration between the reference image set and the evaluation image set;

selecting one or more point locations on the evaluation image set for tracking image movement;

selecting one or more fixed reference points for comparison with the one or more point locations on the evaluation image set;

selecting type of transformation to apply to the evaluation image set for aligning the evaluation image set with the reference image set;

a) calculating quality of alignment between the reference image set and the evaluation image set using a selected quality of alignment methodology;

b) calculating a location value (C) from one or more points on the evaluation image set with respect to the selected one or more fixed reference points and storing the calculation in a memory;

c) calculating a next transformation to apply to the evaluation image;

d) applying the transformation to at least a subset of the evaluation image set;

e) calculating a convergence value (V) for the current iteration (i), and storing the convergence value to the memory;

f) performing steps (a), (b) (c), (d) and (e) until at least a predetermined number N of correspondence calculation iterations have been performed; and

g) repeating steps (a), (b), (c), (d), and (e) if a total number of iterations (i) performed ≦

N and the convergence value (V) ≦

(t), wherein (t) is a threshold value that is dynamically determined using a high-pass estimate of (V) to determine the noise level C_h(i).

2. The method for registering a plurality of image sets according to claim 1, wherein calculating the quality of alignment further includes calculating Mutual Information between the reference image set and the evaluation image set.

3. The method for registering a plurality of images sets according to claim 2, wherein the Mutual Information calculation further includes Stochastic sampling to compute the Mutual Information.

4. The method for registering a plurality of image sets according to claim 2, wherein the image sets to be registered are capable of differing in at least one of temporally, modality, acquisition orientation, and dimensionality.

5. The method for registering a plurality of image sets according to claim 1, wherein the image sets to be registered are capable of differing in at least one of temporally, modality, acquisition orientation, and dimensionality.

6. The method for registering a plurality of image sets according to claim 1, wherein the selecting of point locations includes selecting corner points of the evaluation image set as the selected location points.

7. The method for registering a plurality of image sets according to claim 1, further including using selected corner points of the evaluation image set before the initial transformation as the fixed reference locations.

8. The method for registering a plurality of image sets according to claim 1, wherein the location value (C) is an average distance between the one or more selected point(s) at the current iteration and the one or more fixed reference locations.

9. The method for registering a plurality of image sets according to claim 1, wherein the convergence value (V) is calculated from the location values (C).

10. The method for registering a plurality of image sets according to claim 9, wherein location value (C) is low pass filtered to remove noise.

11. The method for registering a plurality of image sets according to claim 1, wherein location value (C) is low pass filtered to remove noise.

12. The method for registering a plurality of image sets according to claim 1, wherein the convergence value (V) for each iteration (i) is defined as a slope over the last L₂ iterations as V(i)≡

(C(i)−

C(i-L₂))/L₂ wherein L₂ is a number of iterations between two points used to calculate slope.

13. The method for registering a plurality of image sets according to claim 3, wherein the convergence value (V) for each iteration (i) is defined as a slope over the last L₂ iterations as V(i)≡

(C(i)-C(i-L₂))/L₂ wherein L₂ is a number of iterations between two points used to calculate slope.

14. The method for registering a plurality of image sets according to claim 8, wherein the convergence value (V) for each iteration (i) is defined as a slope over the last L₂ iterations as V(i)≡

(C(i)-C(i-L₂))/L₂ wherein L₂ is a number of iterations between two points used to calculate slope.

15. The method for registering a plurality of images sets according to claim 14, wherein convergence is obtained when absolute value of (V)≦

t for at least L₃ of the last L₄ iterations such that L₃ ≦

L₄ wherein L₃ is a number of iterations that must meet a convergence requirement before stopping the calculation and L₄ is a period in which L₃ iterations must occur to meet the convergence requirement.

16. The method for registering a plurality of images sets according to claim 1, wherein convergence is obtained when absolute value of (V)≦

t for at least L₃ of the last L₄ iterations such that L₃≦

L₄ wherein L₃ is a number of iterations that must meet a convergence requirement before stopping the calculation and L₃ is a period in which L₄ iterations must occur to meet the convergence requirement.

17. The method for registering a plurality of image sets according to claim 1, wherein the transformation of the evaluation image set is one of affine, rigid, deformable, and perspective.

18. The method for registering a plurality of image sets according to claim 4, wherein the transformation of the evaluation image set is one of affine, rigid, deformable and perspective.

19. The method for registering a plurality of image sets according to claim 1, wherein the transformation applied to the evaluation image set is determined by an optimization method.

20. The method for registering a plurality of image sets according to claim 19, wherein calculating correspondence further includes at least one of a Powell'"'"'s method optimizer, a steepest gradient descent optimizer, a LBFGS optimizer, a Levenberg-Marquardt optimizer, a conjugate gradient optimizer, and a quasi-Newton optimizer.

21. The method for registering a plurality of image sets according to claim 1, wherein an image set comprises one or more images.

22. The method for registering a plurality of image sets according to claim 21, wherein the image set comprises data relating to functional measurements.

23. The method for registering a plurality of image sets according to claim 21, wherein the image set comprises data relating to anatomical data.

24. The method for registering a plurality of image sets according to claim 1, wherein the image set comprises data relating to functional measurements.

25. The method for registering a plurality of image sets according to claim 1, wherein the image set comprises data relating to anatomical data

26. The method for registering a plurality of image sets according to claim 11, wherein the low-passed filter is defined as

$C_1$

( i ) = 1 L Ii Li ⁢

C ⁡

( i )

wherein L₁ is the length of the low-pass filter.

29. The method for registering a plurality of image sets according to claim 1, wherein the convergence threshold is defined as t=f₁+C_h(i)*f₂,

30. The method for registering a plurality of image sets according to claim 1, wherein the quality of alignment is voxel based.

31. The method for registering a plurality of image sets according to claim 30, wherein the voxel based quality of alignment is further based on intensity of pixels of the image.

32. The method for registering a plurality of image sets according to claim 31, wherein Mutual Information is utilized to determine quality of alignment.

33. The method for registering a plurality of image sets according to claim 32, wherein Mutual Information further includes stochastic approximation.

34. The method for registering a plurality of image sets according to claim 1, wherein the quality of alignment is feature based.

35. The method for registering a plurality of image sets according to claim 34, wherein the feature based quality of alignment is further based on edges extracted from the image.

27. (canceled)

28. (canceled)

36. A system for registering a plurality of image sets, wherein each of the plurality of image sets comprises at least one image, and wherein each of the plurality of image sets contain spatially overlapping areas of an imaged subject with at least one of the remaining plurality of the image sets, the system comprising:

means for selecting a reference image set and an evaluation image set from said plurality of image sets, wherein the evaluation image set is to be aligned with the reference image set;

means for selecting a methodology for comparing the registration between the reference image set and the evaluation image set;

means for selecting one or more point locations on the evaluation image set for tracking image alignment;

means for selecting one or more fixed reference points for comparison with the one or more point locations on the evaluation image set;

means for selecting type of transformation to apply to the evaluation image set for aligning the evaluation image set with the reference image set;

a) means for calculating quality of alignment between the reference image set and the evaluation image set using the selected feature set;

b) means for calculating a location value (C) from one or more points on the evaluation image set with respect to the selected one or more fixed reference points and storing the calculation in the memory;

c) means for calculating a next transformation to apply to the evaluation image,

d) applying the transformation to at least a subset of the evaluation image set;

e) means for calculating a convergence value (V) for the current iteration (i), and storing the convergence value to the memory;

f) means for performing steps (a), (b) (c), (d) and (e) until at least L₅ correspondence calculation iterations have been performed wherein L₅ is a minimum number of iterations before commencing the calculation of convergence;

g) repeating steps (a), (b), (c), (d), and (e) if a total number of iterations (i) performed ≦

N and the convergence value (V)≦

(t), wherein (t) is a threshold value that is dynamically determined using a high-pass estimate of (V) to determine the noise level C_h(i).

37. The system for registering a plurality of image sets according to claim 36, wherein calculating correspondence further includes calculating Mutual Information between the reference image set and the evaluation image set.

38. The system for registering a plurality of images sets according to claim 37, wherein the Mutual Information calculation further includes Stochastic sampling to compute the Mutual Information.

39. The system for registering a plurality of image sets according to claim 37, wherein the image sets to be registered are capable of differing in at least one of temporally, modality, acquisition orientation, and dimensionality.

40. The system for registering a plurality of image sets according to claim 36, wherein the image sets to be registered are capable of differing in at least one of temporally, modality, acquisition orientation, and dimensionality.

41. The system for registering a plurality of image sets according to claim 36, wherein the calculating location step further includes selecting corner points of the evaluation image set as the selected location points.

42. The system for registering a plurality of image sets according to claim 36, further including using selected corner points of the evaluation image set as the fixed reference locations before the initial transformation.

43. The system for registering a plurality of image sets according to claim 36, wherein the location value (C) is an average distance between the one or more selected point(s) at the current iteration and the one or more fixed reference locations.

44. The system for registering a plurality of image sets according to claim 36, wherein the convergence value (V) is calculated from the location value (C).

45. The system for registering a plurality of image sets according to claim 36, wherein location value (C) is low pass filtered to remove noise.

46. The system for registering a plurality of image sets according to claim 36, wherein the convergence value (V) for each iteration (i) is defined as a slope over the last M iterations as

V(i)≡

(C(i)-C(i-M))/M wherein M is a number of iterations between two points used to calculate slope.

47. The system for registering a plurality of images sets according to claim 46, wherein convergence is obtained when absolute value of (V)≦

t for at least Q of the last L iterations such that Q≦

L wherein Q is a number of iterations that must meet a convergence requirement before stopping the calculation and L is a period in which Q iterations must occur to meet the convergence requirement.

48. The system for registering a plurality of image sets according to claim 36, wherein the transformation of the evaluation image set is capable of being limited to one of affine, rigid and deformable.

49. The system for registering a plurality of image sets according to claim 39, wherein the transformation of the evaluation image sat is capable of being limited to one of affine, rigid and deformable.

50. The system for registering a plurality of image sets according to claim 36, wherein an image set comprises one or more images.

51. The system for registering a plurality of image sets according to claim 36, wherein the image set comprises data relating to functional measurements.

52. The system for registering a plurality of image sets according to claim 36, wherein the image set comprises data relating to anatomical data.

53. The system for registering a plurality of image sets according to claim 36, wherein the calculation of the convergence threshold (t) is dynamically determined based on the data.

54. (canceled)

55. A software product capable of directing a general purpose computer to register a plurality of image sets, wherein each of the plurality of image sets contain spatially overlapping areas of an imaged subject with at least one of the remaining plurality of the image sets, the software product comprising:

directing a general purpose computer to execute the steps of;

selecting a reference image set and an evaluation image set from said plurality of image sets, wherein the evaluation image set is to be aligned with the reference image set;

selecting a methodology for comparing the registration between the reference image set and the evaluation image set;

selecting one or more point locations on the evaluation image set for tracking image alignment;

selecting one or more fixed reference points for comparison with the one or more point locations on the evaluation image set;

selecting type of transformation to apply to the evaluation image set for aligning the evaluation image set with the reference image set;

a) calculating quality of alignment between the reference image set and the evaluation image set using the selected feature set;

b) calculating a location value (C) from one or more points on the evaluation image set with respect to the selected one or more fixed reference points and storing the calculation in the memory;

c) calculating a next transformation to apply to the evaluation image;

d) applying the transformation to at least a subset of the evaluation image set;

e) calculating a convergence value (V) for the current iteration (i), and storing the convergence value to the memory;

f) performing steps (a), (b) (c), (d) and (e) until at least L₅ correspondence calculation iterations have been performed;

g) repeating steps (a), (b), (c), (d), and (e) if a total number of iterations (i) performed ≦

N and the convergence value (V)≦

(t), wherein (t) is a threshold value that is defined as t=f₁+C_h(i)*f₂.

56. A system for registering image sets, the system comprising;

a data processor, the data processor having a memory, wherein the data processor is capable of accessing and directing peripheral devices;

a graphical user interface, wherein the graphical user interface is capable of interfacing with an navigating an imaging registration software product for registering image sets and wherein the software product is capable of instructing the data processor to perform instructions pursuant to the software product, the software product comprises;

an instruction for selecting a reference image set and an evaluation image set from said plurality of image sets, wherein the evaluation image set is to be aligned with the reference image set;

an instruction for selecting a methodology for comparing the registration between the reference image set and the evaluation image set;

an instruction for selecting one or more point locations on the evaluation image set for tracking image alignment;

an instruction for selecting one or more fixed reference points for comparison with the one or more point locations on the evaluation image set;

an instruction for selecting type of transformation to apply to the evaluation image set for aligning the evaluation image set with the reference image set;

a) an instruction for calculating quality of alignment between the reference image set and the evaluation image set using the selected feature set;

b) an instruction for calculating a location value (C) from one or more points on the evaluation image set with respect to the selected one or more fixed reference points and storing the calculation in the memory;

c) an instruction for calculating a next transformation to apply to the evaluation image;

d) an instruction for applying the transformation to at least a subset of the evaluation image set;

e) an instruction for calculating a convergence value (V) for the current iteration (i), and storing the convergence value to the memory;

f) an instruction for performing steps (a), (b) (c), (d) and (e) until at least predetermined number of correspondence calculation iterations have been performed;

g) an instruction for repeating steps (a), (b), (c), (d), and (e) if a total number of iterations (i) performed ≦

N and the convergence value (V)≦

(t), wherein (t) is a threshold value that is dynamically determined using a high-pass estimate of (V) to determine the noise level C_h(i).

57. (canceled)

58. (canceled)