Method for matching images using spatially-varying illumination change models

• US 6,621,929 B1
• Filed: 06/22/1999
• Issued: 09/16/2003
• Est. Priority Date: 06/22/1999
• Status: Expired due to Term

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Fang, Ming, Lai, Shang-Hong
• Primary Examiner(s): Mariam, Daniel G.
• Application Number:
  US09/338,298
• Time in Patent Office: 1,547 Days
• Field of Search: 382/103, 382/107, 382/151, 382/156, 382/159, 382/173, 382/174, 382/181, 382/209, 382/217, 382/220, 382/241, 382/254, 382/258, 382/274, 382/284, 382/294, 382/203, 382/218, 382/300, 348/154, 348/155, 348/169, 348/699, 348/441, 356/27, 706/16, 706/17, 706/19, 706/20, 706/21, 706/48, 708/290
• US Class Current: 382/217
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching

Abstract

A method for matching images is provided that includes the step of providing a template image and an input image. The images include pixels, each pixel having an intensity associated therewith. An energy function formed by weighting a sum of squared differences of data constraints corresponding to locations of both the template image and the input image is minimized to determine estimated geometric and spatially-varying illumination-parameters for the input image with respect to the template image. The estimated geometric and spatially-varying illumination parameters are outputted for further processing.

Claims

1. A method for matching images, comprising the steps of:

providing a template image and an input image, the images comprising pixels, each pixel having an intensity associated therewith;

minimizing an energy function formed by weighting a sum of squared differences of data constraints corresponding to locations of both the template image and the input image to determine estimated geometric and spatially-varying illumination parameters for the input image with respect to the template image, wherein said minimizing step comprises the step of modeling spatially-varying illumination multiplication and bias factors using low-order polynomials;

outputting the estimated geometric and spatially-varying illumination parameters for further processing wherein said minimizing step further comprises the steps of;

calculating a Hessian matrix and a gradient vector of the energy function based on an initial guess of geometric and illumination parameters;

updating the initial guess based on the calculating of the Hessian matrix and the gradient vector of the energy function; and

iteratively recalculating the Hessian matrix and the gradient vector of the energy function until an updated guess is within an acceptable increment from a previous updated guess.

2. The method as recited in claim 1, further comprising the steps of:

partitioning the template image into blocks of pixels;

determining a reliability measure for each pixel in each block; and

identifying pixel locations for each block having a largest reliability measure.

3. The method as recited in claim 1, further comprising the step of smoothing the template image to reduce noise effects.

4. The method as recited in claim 1, wherein said minimizing step further comprises the step of incorporating a spatially-varying illumination change factor into the data constraints to account for pixel intensity changes due to illumination effects.

5. The method as recited in claim 1, wherein said minimizing step comprises the step of alleviating errors due to nonlinear characteristics in an optical sensor using a consistency measure of image gradients and a nonlinear function of pixel intensities.

6. The method as recited in claim 1, wherein said minimizing step further comprises the step of dynamically assigning weights to the data constraints based on residues of the data constraints, a consistency measure of image gradients, and a non-linear function of the pixel intensities.

7. The method as recited in claim 1, further comprising the step of generating an initial guess corresponding to initial geometric and illumination parameters of the input image.

8. A method for matching images, comprising the steps of:

providing a template image comprising pixels, and selecting a portion of the pixels thereof;

providing an input image;

generating an initial guess for geometric and illumination parameter values of the input image;

minimizing an energy function formed by weighting a sum of data constraints corresponding to a region of interest in both the template image and the input image to determine estimated geometric and spatially-varying illumination parameter values between the template image and the input image, wherein said minimizing step comprises the step of modeling spatially-varying illumination multiplication and bias factors using low-order polynomials;

outputting the estimated geometric and spatially-varying illumination parameters for further processing wherein said minimizing step further comprises the steps of;

calculating a Hessian matrix and a gradient vector of the energy function based on an initial guess of geometric and illumination parameters;

updating the initial guess based on the calculating of the Hessian matrix and the gradient vector of the energy function; and

iteratively recalculating the Hessian matrix and the gradient vector of the energy function until an updated guess is within an acceptable increment from a previous updated guess.

9. The method as recited in claim 8, wherein said selecting step comprises the steps of:

partitioning the template image into blocks of pixels;

determining a reliability measure for each pixel in each block; and

identifying pixel locations for each block having the largest reliability measure.

10. The method as recited in claim 8, further comprising the step of smoothing the template image to reduce noise effects.

11. The method as recited in claim 8, wherein said minimizing step further comprises the step of incorporating a spatially-varying illumination change factor into the data constraints to account for pixel intensity changes due to illumination effects.

12. The method as recited in claim 8, wherein said minimizing step comprises the step of alleviating errors due to nonlinear characteristics in an optical sensor using a consistency measure of image gradients did a nonlinear function of pixel intensities.

13. The method as recited in claim 8, wherein said minimizing step further comprises the step of dynamically assigning weights to the data constraints based on residues of the data constraints, a consistency measure of image gradients, and a non-linear function of pixel intensity values.

14. The method as recited in claim 8, further comprising the step of generating an initial guess corresponding to initial geometric and illumination parameters of the input image.

15. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for matching images, said method steps comprising:

providing a template image and an input image, the images comprising pixels, each pixel having an intensity associated therewith;

minimizing an energy function formed by weighting a sum of squared differences of data constraints corresponding to locations of both the template image and the input image to determine estimated geometric and spatially-varying illumination parameters for the input image with respect to the template image, wherein said minimizing step comprises the step of modeling spatially-varying illumination multiplication and bias factors using low-order polynomials;

outputting the estimated geometric and illumination parameters for further processing wherein said minimizing step further comprises the steps of;

calculating a Hessian matrix and a gradient vector of the energy function based on an initial guess of geometric and illumination parameters;

updating the initial guess based on the calculating of the Hessian matrix and the gradient vector of the energy function; and

iteratively recalculating the Hessian matrix and the gradient vector of the energy function until an updated guess is within an acceptable increment from a previous updated guess.

16. The program storage device as recited in claim 15, further comprising the steps of:

partitioning the template image into blocks of pixels;

determining a reliability measure for each pixel in each block; and

identifying pixel locations for each block having a largest reliability measure.

17. The program storage device as recited in claim 15, further comprising the step of smoothing the template image to reduce noise effects.

18. The program storage device as recited in claim 15, wherein said minimizing step further comprises the step of incorporating a spatially-varying illumination change factor into the data constraints to account for pixel intensity changes due to illumination effects.

19. The program storage device as recited in claim 15, wherein said minimizing step comprises the step of alleviating errors due to nonlinear characteristics in an optical sensor using a consistency measure of image gradients and a nonlinear function of pixel intensities.

20. The program storage device as recited in claim 15, wherein said minimizing step further comprises the step of dynamically assigning weights to the data constraints based on residues of the data constraints, a consistency measure of image gradients, and a non-linear function of the pixel intensities.

21. The program storage device as recited in claim 15, further comprising the step of generating an initial guess corresponding to initial geometric and illumination parameters of the input image.