FEATURE BASED IMAGE REGISTRATION

• US 20130028519A1
• Filed: 07/29/2011
• Published: 01/31/2013
• Est. Priority Date: 07/29/2011
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Zuniga, Oscar, Pollard, Stephen, Voss, Shane D.
• Granted Patent: US 8,675,997 B2
• Time in Patent Office: Days
• US Class Current: 382/195
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

Example embodiments disclosed herein relate to feature based image registration. Feature based image registration determines correspondence between image features such as points, lines, and contours to align or register a reference or first image and a target or second image. The examples disclosed herein may be used in mobile devices such as cell phones, personal digital assistants, personal computers, cameras, and video recorders.

Claims

1. A method for registering a first image and a second image, comprising:

generating a blurred first image;

applying a coarse grid to the blurred first image;

computing a determinant of a Hessian matrix at predetermined points of the coarse grid on the blurred first image;

determining low resolution feature points in the blurred first image;

determining high resolution feature points in the blurred first image;

creating a first set of blurred first image key points;

extracting a first feature descriptor for each of the blurred first image key points;

generating a blurred second image;

applying the coarse grid to the blurred second image;

computing a determinant of a Hessian matrix at predetermined points of the coarse grid on the blurred second image;

determining low resolution feature points in the blurred second image;

determining high resolution feature points in the blurred second image;

creating a second set of blurred second image key points

extracting a second feature descriptor for each of the blurred second image key points;

selecting blurred first image key points and blurred second image key points for matching based on a measure of closeness of the first feature descriptors and the second feature descriptors; and

mapping the first image into the second image based on matching pairs of blurred first image key points and blurred second image key points.

2. The method of Claim 1, wherein generating the blurred first image includes convolving the first image with a two-dimensional box filter.

3. The method of Claim 2, wherein the first image includes a plurality of pixels and further comprising computing a predetermined sized box filter with four operations per pixel.

4. The method of Claim 1, wherein the predetermined locations on the coarse grid occur only once for every 4×

4 pixels of the blurred first image.

5. The method of Claim 1, wherein determining the low resolution feature points in the blurred first image includes selecting local maxima of the determinant of the Hessian matrix that exceed a pre-determined image dependent threshold.

6. The method of Claim 1, wherein determining the high resolution feature points in the blurred first image includes applying a fine grid around each of the low resolution feature points in the blurred first image and computing a determinant of a Hessian matrix at each point of the fine grid to determine a maximum value.

7. The method of Claim 6, wherein the fine grid measures 7×

7 pixels in the blurred first image.

8. The method of Claim 1, wherein creating a set of blurred first image key points include selecting a predetermined number of high resolution feature points that have strongest Hessian matrix determinant values.

9. The method of Claim 8, wherein the high resolution feature points are selected from a histogram of the quantized Hessian matrix determinant values.

10. The method of Claim 1, wherein extracting the first feature descriptor for each of the blurred first image key points includes utilizing luminance values in a window centered about each of the blurred first image key points.

11. The method of Claim 10, wherein extracting the first feature descriptor for each of the blurred first image key points further includes removing a mean of the luminance values in the window centered about each of the blurred first image key points to provide robustness against changes of illumination between the first image and the second image.

12. The method of Claim 1, wherein selecting blurred first image key points and blurred second image key points for matching based on a measure of closeness of the first feature descriptors and the second feature descriptors includes applying a spatial two-dimensional index to the first feature descriptors and the second feature descriptors to limit a number of first feature descriptors and second feature descriptors for matching based on a measure of closeness.

13. The method of Claim 12, wherein selecting blurred first image key points and blurred second image key points for matching based on the measure of closeness of the first feature descriptors and the second feature descriptors includes determining the measure of closeness with a difference operator.

14. The method of Claim 1, wherein mapping the first image into the second image based on matching pairs of blurred first image key points and blurred second image key points includes determining a global affine transformation.

15. The method of Claim 1, further comprising utilizing a mobile device to register the first image and the second image.

16. A non-transitory computer-readable storage medium storing instructions for registering a first image and a second image that, when executed by a processor, cause the processor to:

generate a blurred first image;

apply a coarse grid to the blurred first image;

compute a determinant of a Hessian matrix at predetermined points of the coarse grid on the blurred first image;

determine low resolution feature points in the blurred first image;

determine high resolution feature points in the blurred first image;

create a first set of blurred first image key points;

extract a first feature descriptor for each of the blurred first image key points;

generate a blurred second image;

apply the coarse grid to the blurred second image;

compute a determinant of a Hessian matrix at predetermined points of the coarse grid on the blurred second image;

determine low resolution feature points in the blurred second image;

determine high resolution feature points in the blurred second image;

create a second set of blurred second image key points

extract a second feature descriptor for each of the blurred second image key points;

select blurred first image key points and blurred second image key points for matching based on a measure of closeness of the first feature descriptors and the second feature descriptors; and

map the first image into the second image based on matching pairs of blurred first image key points and blurred second image key points.

17. The non-transitory computer-readable storage medium of Claim 16, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to convolve the first image with a two-dimensional box filter to generate the blurred first image.

18. The non-transitory computer-readable storage medium of Claim 17, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to compute a predetermined sized box filter with four operations for each pixel of the first image.

19. The non-transitory computer-readable storage medium of Claim 16, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to select local maxima of the determinant of the Hessian matrix that exceed a pre-determined image dependent threshold to determine the low resolution feature points in the blurred first image includes.

20. The non-transitory computer-readable storage medium of Claim 16, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to apply a fine grid around each of the low resolution feature points in the blurred first image, compute a determinant of a Hessian matrix at each point of the fine grid, and select a maximum value of the determinant to determine the high resolution feature points in the blurred first image.

21. The non-transitory computer-readable storage medium of Claim 16, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to select a predetermined number of high resolution feature points that have strongest Hessian matrix determinant values to create a set of blurred first image key points.

22. The non-transitory computer-readable storage medium of claim 16, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to utilize luminance values in a window centered about each of the blurred first image key points to extract the first feature descriptor for each of the blurred first image key points.

23. The non-transitory computer-readable storage medium of claim 16, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to apply a spatial two-dimensional index to the first feature descriptors and the second feature descriptors to limit a number of first feature descriptors and second feature descriptors for matching based on a measure of closeness.

24. The non-transitory computer-readable storage medium of claim 23, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to determine the measure of closeness with a difference operator.

25. The non-transitory computer-readable storage medium of claim 16, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to determining a global affine transformation to map the first image into the second image based on matching pairs of blurred first image key points and blurred second image key points.

26. The non-transitory computer-readable storage medium of claim 16, in a mobile device.