Multimodal image registration using compound mutual information

• US 20060029291A1
• Filed: 08/09/2004
• Published: 02/09/2006
• Est. Priority Date: 08/09/2004
• Status: Active Grant

• Current Assignee: CareStream Health Incorporated
• Original Assignee: Eastman Kodak Company
• Inventors: Ray, Lawrence A., Sun, Zhaohui
• Granted Patent: US 7,639,896 B2
• Time in Patent Office: Days
• US Class Current: 382/294
• CPC Class Codes: G06V 10/758 Involving statistics of pixels or of feature values, e.g. histogram matching

Abstract

In a digital image registration method, a second digital image is transformed to be more like a first digital image using a modification function, resulting in a modified second digital image. A set of first features of each of the first and modified second digital images is determined. Zones are specified in each of the first and modified second digital images. A set of zonal features, of each of the zones is determined. Compound mutual information is computed from the first and zonal features. The modification function is altered. The transforming, determining, specifying, determining, computing, and altering are iterated until the compound mutual information is maximized.

Claims

1. A method for registering first and second digital images, said method comprising the steps of:

transforming said second digital image to be more like said first digital image using a modification function, to provide a modified second digital image;

determining a set of first features of each of said first and modified second digital images;

specifying zones in each of said first and modified second digital images;

determining a set of zonal features of each of said specified zones of each of said first and modified second digital images;

computing compound mutual information from said first features and said zonal features;

altering said modification function; and

iterating said transforming, determining, specifying, determining, computing, and altering steps until said compound mutual information is maximized.

2. The method of claim 1 wherein said zonal features are different than said first features.

3. The method of claim 1 further comprising, following said iterating, using said modified second digital image in place of said second digital image.

4. The method of claim 1 wherein said first features are global.

5. The method of claim 4 further comprising, prior to said computing, determining at least one other set of global features of each of said first and modified second digital images.

6. The method of claim 5 wherein said first features are intensity/color histograms.

7. The method of claim 1 wherein said zones are mutually exclusive, in each of said first and modified second digital images.

8. The method of claim 7 wherein said zones are each rotation invariant relative to the center of the respective zone.

9. The method of claim 1 wherein said zones define ring/shell shapes.

10. The method of claim 1 wherein said zones each include a center pixel and a plurality of neighboring pixels, said neighboring pixels being encoded based on respective locations relative to said center pixel.

11. The method of claim 1 wherein said zones are each rotation invariant relative to the center of the respective zone.

12. The method of claim 1 wherein said specifying and said determining of said set of zonal features are concurrent.

13. The method of claim 1 further comprising, prior to said computing, designating at least one set of other zones in each of said first and modified second digital images, and determining at least one other set of zonal features of each of said other zones of each of said first and modified second digital images.

14. The method of claim 13 wherein said specified zones and said other zones are mutually exclusive, in each of said first and modified second digital images.

15. The method of claim 13 wherein said zones of each of said sets of features are at different distances from respective center pixels.

16. The method of claim 13 wherein said zones define ring/shell shapes.

17. The method of claim 1 wherein at least one of said determining steps includes converting said first and modified second digital images from spatial domain to one of frequency domain and space-frequency domain.

18. The method of claim 1 wherein said modification function is a geometric transform.

19. The method of claim 1 further comprising maintaining said digital images in one of the frequency domain and the space-frequency domain during one or more of said transforming, determining, specifying, determining, computing, altering, and iterating steps.

20. The method of claim 1 further comprising continuously maintaining said digital images in one of frequency domain and space-frequency domain during all of said transforming, determining, specifying, determining, computing, altering, and iterating steps.

21. The method of claim 1 further comprising continuously maintaining said digital images in spatial domain during all of said transforming, determining, specifying, determining, computing, altering, and iterating steps.

22. The method of claim 1 wherein said zonal features are spatial neighborhood properties.

23. The method of claim 1 wherein said computing further comprises extracting a set of global marginal densities from said set of global features and a set of zonal marginal densities from said set of zonal features.

24. The method of claim 23 wherein said computing further comprises aggregating marginal mutual information from said sets of marginal densities.

25. The method of claim 23 wherein said computing further comprises applying weighting coefficients to said sets of marginal densities to provided weighted marginal mutual information, and aggregating said weighted marginal mutual information.

26. The method of claim 25 wherein said aggregating is in accordance with the formula:

27. The method of claim 1 wherein said sets of global features are intensity histograms.

28. The method of claim 1 further comprising optimizing said altering.

29. The method of claim 28 wherein said optimizing utilizes a multidimensional direction set.

30. The method of claim 28 wherein said optimizing utilizes windowing and gradient decent.

31. The method of claim 1 wherein said first and second digital images are products of different modalities.

32. The method of claim 1 wherein at least one of said modalities is selected from the group consisting of:

ultrasound images, infrared images, visible light image, X-ray images, MRI images, and PET images.

33. The method of claim 1 wherein said first and second images differ in at least one of:

image dimension, pixel bit depth, noise characteristics, and spatial sampling density.

34. A system for registering first and second digital images, said method comprising the steps of:

means for transforming said second digital image to be more like said first digital image using a modification function, to provide a modified second digital image;

means for determining a set of first features of each of said first and modified second digital images;

means for specifying zones in each of said first and modified second digital images;

means for determining a set of zonal features of each of said specified zones of each of said first and modified second digital images, said zonal features being different than said first features;

means for computing compound mutual information from said first features and said zonal features;

means for altering said modification function; and

means for iterating said transforming, determining, specifying, determining, computing, and altering steps until said compound mutual information is maximized.

35. The method of claim 34 further comprising, following said iterating, using said modified second digital image in place of said second digital image.

36. A digital image registering method comprising the steps of:

receiving a pair of digital images;

determining a set of first features of each of said digital images;

specifying zones in each of said digital images;

determining a set of zonal features of each of said specified zones of each of said digital images, said zonal features being different than said first features; and

computing compound mutual information from said first features and said zonal features.

37. The method of claim 36 wherein said first features are global.

38. The method of claim 37 further comprising, prior to said computing, determining at least one other set of global features of each of said first and modified second digital images.

39. The method of claim 36 wherein said zones are mutually exclusive, in each of said first and modified second digital images.

40. The method of claim 36 wherein said specifying and said determining of said set of zonal features are concurrent.

41. The method of claim 36 further comprising, prior to said computing, designating at least one set of other zones in each of said first and modified second digital images, and determining at least one other set of zonal features of each of said other zones of each of said first and modified second digital images.

42. The method of claim 41 wherein said specified zones and said other zones are mutually exclusive, in each of said first and modified second digital images.

43. The method of claim 36 wherein at least one of said determining steps includes converting said first and modified second digital images from spatial domain to one of frequency domain and space-frequency domain.

44. The method of claim 36 wherein said modification function is a geometric transform.

45. The method of claim 36 further comprising maintaining said digital images in one of the frequency domain and the space-frequency domain during one or more of said determining, specifying, determining, and computing steps.

46. The method of claim 36 further comprising continuously maintaining said digital images in one of frequency domain and space-frequency domain during all of said determining, specifying, determining, and computing steps.

47. The method of claim 36 further comprising continuously maintaining said digital images in spatial domain during all of said determining, specifying, determining, and computing steps.

48. The method of claim 36 wherein said zonal features are spatial neighborhood properties.

49. The method of claim 36 wherein said computing further comprises extracting a set of global marginal densities from said set of global features and a set of zonal marginal densities from said set of zonal features.

50. The method of claim 49 wherein said computing further comprises aggregating marginal mutual information from said sets of marginal densities.

51. The method of claim 49 wherein said computing further comprises applying weighting coefficients to said sets of marginal densities to provided weighted marginal mutual information, and aggregating said weighted marginal mutual information.

52. The method of claim 51 wherein said aggregating is in accordance with the formula:

53. The method of claim 36 wherein said sets of global features are intensity histograms.

54. A computer program product for registering first and second digital images, the computer program product comprising computer readable storage medium having a computer program stored thereon for performing the steps of:

transforming said second digital image to be more like said first digital image using a modification function, to provide a modified second digital image;

determining a set of first features of each of said first and modified second digital images;

specifying zones in each of said first and modified second digital images;

determining a set of zonal features of each of said specified zones of each of said first and modified second digital images;

computing compound mutual information from said first features and said zonal features;

altering said modification function; and

iterating said transforming, determining, specifying, determining, computing, and altering steps until said compound mutual information is maximized.