Simultaneous grayscale and geometric registration of images

• US 7,664,302 B2
• Filed: 12/30/2004
• Issued: 02/16/2010
• Est. Priority Date: 07/18/2003
• Status: Expired due to Fees

• Current Assignee: Hologic Incorporated
• Original Assignee: Hologic Incorporated
• Inventors: Karssemeijer, Nico, Snoeren, Peter R.
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): Rashid; David P
• Application Number:
  US11/026,558
• Publication Number:
  US 20050163360A1
• Time in Patent Office: 1,874 Days
• Field of Search: 382/131
• US Class Current: 382/131
• CPC Class Codes: G06T 2207/30068 Mammography; Breast
• CPC Class Codes: G06T 7/35 using statistical methods
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image

Abstract

Simultaneous grayscale and geometric registration of images, such as mammograms, facilitates temporal comparison and enhances the speed and reliability of computer aided diagnosis (CAD) detection of medical abnormalities. The method generally includes optimizing a merit function, e.g., sum of squared errors, containing parameters associated with a transformation function for simultaneous geometric and grayscale registering of the images, the optimizing of the merit function being performed by determining optimal values of the parameters using data in the images and registering one image to the other by applying the geometric and grayscale transformation function using the optimal values of the parameters. The optimizing may be performed iteratively from coarse to fine resolutions using a modified Levenberg-Marquardt method for optimizing nonlinear parameters with linear regression for optimizing linear parameters. A final iteration may be performed after removing pixel value pairs from the images that correspond to outliers of a joint pixel value histogram.

Claims

1. A method implemented by a computing device comprising at least one processor and memory for registration of a first image and a second image, the first and second images being medical images in digital form of one of a same body part and respective corresponding body parts, the method comprising:

acquiring, by the computing device, the first and second images from a computer readable image memory;

optimizing, by the computing device, a merit function containing a plurality of parameters associated with a transformation function for simultaneous geometric and grayscale registering of the first and second images relative to each other, the optimizing of the merit function being performed by determining optimal values of the parameters using data in the first and second images; and

registering, by the computing device, the first image to the second image by applying the geometric and grayscale transformation function using the optimal values of the parameters to generate a third image, the third image being transformed from the first image and registered to the second image; and

performing, by the computing device, at least one of the steps of;

(i) outputting said second and third images to a display device, and

(ii) processing said second and third images according to an automated anatomical abnormality detection algorithm and outputting at least one result thereof in digital form to a destination computer readable memory.

2. The method of claim 1, wherein said plurality of parameters includes (i) at least one grayscale mapping parameter corresponding to a predetermined model of medical image acquisition associated with said first and second images, and (ii) at least one geometric mapping parameter corresponding to a predetermined model of a geometrical relationship between said first and second images.

3. The method of claim 2, wherein said first and second images are mammograms, and wherein said predetermined model of medical image acquisition associated with said first and second images is selected from the group consisting of:

a first model in which the first image is a digitized analog mammogram and the second image is a digitized analog mammogram;

a second model in which the first image is a digitized analog mammogram and the second image is a digitally acquired mammogram;

a third model in which the first image is a digitally acquired mammogram and the second image is a digitized analog mammogram; and

a fourth model in which the first image is a digitally acquired mammogram and the second image is a digitally acquired mammogram.

4. The method of claim 2, wherein said predetermined model of the geometric relationship between said first and second images models is characterized by an isotropic scaling, a rotation about a point, and a lateral translation.

5. The method of claim 1, wherein the optimizing includes iteratively performing an optimization process from a coarse resolution to a fine resolution of the images.

6. The method of claim 5, wherein each iterative optimization process utilizes results from a previous iteration.

7. The method of claim 5, wherein the parameters include at least one linear parameter and at least one nonlinear parameter and wherein each iterative optimization process includes:

optimizing each linear parameter of the merit function while fixing the value of each nonlinear parameter; and

optimizing each nonlinear parameter of the merit function while fixing the value of each linear parameter.

8. The method of claim 7, wherein the optimizing each linear parameter of the merit function includes performing linear regression.

9. The method of claim 7, wherein the optimizing each nonlinear parameter of the merit function includes performing a Levenberg-Marquardt method.

10. The method of claim 1, further comprising forming a lookup table that implements said transformation function and said associated determined optimal parameter values, wherein the registering includes applying the first image to the lookup table to generate the third image.

11. The method of claim 1, wherein the optimizing the merit function further includes:

determining a preliminary set of optimal values of the parameters using data in the first and second images;

generating a joint histogram of pixel value pairs of the first and second images using the preliminary set of optimal values of the parameters, the joint histogram having its mass centered about a curve;

identifying outliers as points in the joint histogram that exceed a predetermined distance from the curve of the joint histogram;

determining pixel value pairs in the first and second images corresponding to the outliers; and

repeating the optimizing the merit function using data in the first and second images without the pixel value pairs corresponding to the outliers.

12. The method of claim 1, wherein each of the first and second images includes a target portion and a remainder portion, the method further comprising:

cropping each of the first and second images to contain only the respective target portion prior to the optimizing.

13. The method of claim 1, wherein each of the first and second images is a mammogram.

14. The method of claim 1, wherein each of the first and second images is selected from a digitized analog image and a digitally acquired image.

15. A system for registering a first image to a second image, the first and second images being medical images in digital form of one of a same body part and respective corresponding body parts, the system comprising:

a processor optimizing a merit function containing a plurality of parameters associated with a transformation function for simultaneous geometric and grayscale registering of the first and second images relative to each other, the processor optimizing the merit function by determining optimal values of the parameters using the data in the first and second images, the processor further registering the first image to the second image by applying the geometric and grayscale transformation function using the optimal values of the parameters to generate a third image, the third image being transformed from the first image and registered to the second image.

16. The system of claim 15, wherein the processor further performs computer aided detection processing including comparing the second and third images, the system further comprising:

a display device displaying the second and third images with results of the computer aided detection processing.

17. The system of claim 15, wherein said plurality of parameters includes (i) at least one grayscale mapping parameter corresponding to a predetermined model of medical image acquisition associated with said first and second images, and (ii) at least one geometric mapping parameter corresponding to a predetermined model of a geometrical relationship between said first and second images.

18. The system of claim 17, wherein said first and second images are mammograms, and wherein said predetermined model of medical image acquisition associated with said first and second images is selected from the group consisting of:

a first model in which the first image is a digitized analog mammogram and the second image is a digitized analog mammogram;

a second model in which the first image is a digitized analog mammogram and the second image is a digitally acquired mammogram;

a third model in which the first image is a digitally acquired mammogram and the second image is a digitized analog mammogram; and

a fourth model in which the first image is a digitally acquired mammogram and the second image is a digitally acquired mammogram.

19. The system of claim 18, wherein the parameters include at least one linear parameter and at least one nonlinear parameter and wherein, for each iterative optimization process, the processor optimizes each linear parameter of the merit function while fixing the value of each nonlinear parameter and optimizes each nonlinear parameter of the merit function while fixing the value of each linear parameter.

20. The system of claim 18, wherein the processor optimizes each linear parameter of the merit function by performing linear regression.

21. The system of claim 18, wherein the processor optimizes each nonlinear parameter of the merit function by performing a Levenberg-Marquardt method.

22. The system of claim 15, wherein the processor iteratively performs an optimization process from a coarse resolution to a fine resolution of the images, and wherein the processor further utilizes results from a previous iteration in each iterative optimization process.

23. The system of claim 15, wherein the processor registers the first image to the second image by generating a simultaneous geometric and grayscale transformation lookup table and utilizes the lookup table to generate the pixel values of the third image from the first image.

24. The system of claim 15, wherein the processor optimizes the merit function by:

determining a preliminary set of optimal values of the parameters using data in the first and second images;

generating a joint histogram of pixel value pairs of the first and second images using the preliminary set of optimal values of the parameters, the joint histogram having its mass centered about a curve;

identifying outliers as points in the joint histogram that exceed a predetermined distance from the curve of the joint histogram;

determining pixel value pairs in the first and second images corresponding to the outliers; and

repeating the optimizing the merit function using data in the first and second images without the pixel value pairs corresponding to the outliers.

25. A computer readable medium storing computer code for directing a computing apparatus to automatically register a first image and a second image, the first and second images being medical images in digital form of one of a same body part and respective corresponding body parts, comprising:

computer code for optimizing a merit function containing a plurality of parameters associated with a transformation function for simultaneous geometric and grayscale registering of the first and second images relative to each other, the optimizing of the merit function being performed by determining optimal values of the parameters using data in the first and second images; and

computer code for registering the first image to the second image by applying the geometric and grayscale transformation function using the optimal values of the parameters to generate a third image, the third image being transformed from the first image and registered to the second image.

26. The computer readable medium of claim 25, further comprising:

computer code for performing computer aided detection process for comparing the second and third images; and

computer code for displaying the second and third images with results of the computer aided detection process.

27. The computer readable medium of claim 26, wherein said first and second images are mammograms, and wherein said predetermined model of medical image acquisition associated with said first and second images is selected from the group consisting of:

a first model in which the first image is a digitized analog mammogram and the second image is a digitized analog mammogram;

a second model in which the first image is a digitized analog mammogram and the second image is a digitally acquired mammogram;

a third model in which the first image is a digitally acquired mammogram and the second image is a digitized analog mammogram; and

a fourth model in which the first image is a digitally acquired mammogram and the second image is a digitally acquired mammogram.

28. The computer readable medium of claim 27, wherein the parameters include at least one linear parameter and at least one nonlinear parameter and wherein the computer code for performing each iterative optimization process includes:

computer code for optimizing each linear parameter of the merit function while fixing the value of each nonlinear parameter; and

computer code for optimizing each nonlinear parameter of the merit function while fixing the value of each linear parameter.

29. The computer readable medium of claim 28, wherein the computer code for optimizing each linear parameter of the merit function includes computer code for performing linear regression.

30. The computer readable medium of claim 28, wherein the computer code for optimizing each nonlinear parameter of the merit function includes computer code for performing a Levenberg-Marquardt method.

31. The computer readable medium of claim 25, wherein said plurality of parameters includes (i) at least one grayscale mapping parameter corresponding to a predetermined model of medical image acquisition associated with said first and second images, and (ii) at least one geometric mapping parameter corresponding to a predetermined model of a geometrical relationship between said first and second images.

32. The computer readable medium of claim 25, wherein the computer code for optimizing includes computer code for iteratively performing an optimization process from a coarse resolution to a fine resolution of the images, and wherein each iterative optimization process utilizes results from a previous iteration.

33. The computer readable medium of claim 25, wherein the computer code for optimizing the merit function further includes:

computer code for determining a preliminary set of optimal values of the parameters using data in the first and second images;

computer code for generating a joint histogram of pixel value pairs of the first and second images using the preliminary set of optimal values of the parameters, the joint histogram having its mass centered about a curve;

computer code for identifying outliers as points in the joint histogram that exceed a predetermined distance from the curve of the joint histogram;

computer code for determining pixel value pairs in the first and second images corresponding to the outliers; and

computer code for repeating the optimizing the merit function using data in the first and second images without the pixel value pairs corresponding to the outliers.