Feature based image registration

• US 8,675,997 B2
• Filed: 07/29/2011
• Issued: 03/18/2014
• Est. Priority Date: 07/29/2011
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Zuniga, Oscar, Pollard, Stephen, Vosa, Shane D
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US13/193,988
• Publication Number:
  US 20130028519A1
• Time in Patent Office: 963 Days
• Field of Search: 382168-172, 382/190, 382260-261, 382294-296
• US Class Current: 382/294
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

Example embodiments disclosed herein relate to feature based image registration. Feature based image registration determines correspondence between image features such as points, lines, and contours to align or register a reference or first image and a target or second image. The examples disclosed herein may be used in mobile devices such as cell phones, personal digital assistants, personal computers, cameras, and video recorders.

Claims

1. A method, comprising:

applying a first grid to a blurred first image;

computing a determinant of a Hessian matrix at predetermined points of the first grid on the blurred first image;

determining low resolution feature points in the blurred first image based on the first grid;

applying a second grid to the blurred first image, the second grid being finer than the first grid;

computing a determinant of a Hessian matrix at predetermined points of the second grid on the blurred first image;

determining high resolution feature points in the blurred first image based on the second grid;

creating a first set of blurred first image key points;

extracting a first feature descriptor for each of the blurred first image key points;

applying the first grid to a blurred second image;

computing a determinant of a Hessian matrix at predetermined points of the first grid on the blurred second image;

determining low resolution feature points in the blurred second image based on the first grid;

applying the second grid to the blurred second image;

computing a determinant of a Hessian matrix at predetermined points of the second grid on the blurred second image;

determining high resolution feature points in the blurred second image based on the second grid;

creating a second set of blurred second image key points

extracting a second feature descriptor for each of the blurred second image key points;

selecting blurred first image key points and blurred second image key points for matching based on a measure of closeness of the first feature descriptors and the second feature descriptors; and

mapping the first image into the second image based on matching pairs of blurred first image key points and blurred second image key points.

2. The method of claim 1, further comprising generating the blurred first image, wherein generating the blurred first image includes convolving the first image with a two-dimensional box filter.

3. The method of claim 2, wherein the first image includes a plurality of pixels and further comprising computing a predetermined sized box filter with four operations per pixel.

4. The method of claim 1, wherein the predetermined locations on the first grid occur only once for every 4×

4 pixels of the blurred first image.

5. The method of claim 1, wherein determining the low resolution feature points in the blurred first image includes selecting local maxima of the determinant of the Hessian matrix that exceed a pre-determined image dependent threshold.

6. The method of claim 1, wherein the second grid measures 7×

7 pixels in the blurred first image.

7. The method of claim 1, wherein creating a set of blurred first image key points include selecting a predetermined number of high resolution feature points that have strongest Hessian matrix determinant values.

8. The method of claim 7, wherein the high resolution feature points are selected from a histogram of the quantized Hessian matrix determinant values.

9. The method of claim 1, wherein extracting the first feature descriptor for each of the blurred first image key points includes utilizing luminance values in a window centered about each of the blurred first image key points.

10. The method of claim 9, wherein extracting the first feature descriptor for each of the blurred first image key points further includes removing a mean of the luminance values in the window centered about each of the blurred first image key points to provide robustness against changes of illumination between the first image and the second image.

11. The method of claim 1, wherein selecting blurred first image key points and blurred second image key points for matching based on a measure of closeness of the first feature descriptors and the second feature descriptors includes applying a spatial two-dimensional index to the first feature descriptors and the second feature descriptors to limit a number of first feature descriptors and second feature descriptors for matching based on a measure of closeness.

12. The method of claim 11, wherein selecting blurred first image key points and blurred second image key points for matching based on the measure of closeness of the first feature descriptors and the second feature descriptors includes determining the measure of closeness with a difference operator.

13. The method of claim 1, wherein mapping the first image into the second image based on matching pairs of blurred first image key points and blurred second image key points includes determining a global affine transformation.

14. The method of claim 1, further comprising utilizing a mobile device to register the first image and the second image.

15. A non-transitory computer-readable storage medium storing instructions, when executed by a processor, cause the processor to:

apply a first grid to a blurred first image;

compute a determinant of a Hessian matrix at predetermined points of the first grid on the blurred first image;

determine low resolution feature points in the blurred first image based on the first grid;

apply a second grid to at least portions of the blurred first image, the second grid being finer than the first grid;

compute a determinant of a Hessian matrix at predetermined points of the second grid on the blurred first image;

determine high resolution feature points in the blurred first image based on the second grid;

create a first set of blurred first image key points;

extract a first feature descriptor for each of the blurred first image key points;

apply the first grid to a blurred second image;

compute a determinant of a Hessian matrix at predetermined points of the first grid on the blurred second image;

determine low resolution feature points in the blurred second image based on the first grid;

apply the second grid to at least portions of the blurred second image;

compute a determinant of a Hessian matrix at predetermined points of the second grid on the blurred second image;

determine high resolution feature points in the blurred second image based on the second grid;

create a second set of blurred second image key points

extract a second feature descriptor for each of the blurred second image key points;

select blurred first image key points and blurred second image key points for matching based on a measure of closeness of the first feature descriptors and the second feature descriptors; and

map the first image into the second image based on matching pairs of blurred first image key points and blurred second image key points.

16. The non-transitory computer-readable storage medium of claim 15, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to convolve the first image with a two-dimensional box filter to generate the blurred first image.

17. The non-transitory computer-readable storage medium of claim 16, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to compute a predetermined sized box filter with four operations for each pixel of the first image.

18. The non-transitory computer-readable storage medium of claim 15, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to select local maxima of the determinant of the Hessian matrix that exceed a pre-determined image dependent threshold to determine the low resolution feature points in the blurred first image includes.

19. The non-transitory computer-readable storage medium of claim 15, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to apply the second grid around each of the low resolution feature points in the blurred first image, compute a determinant of a Hessian matrix at each point of the second grid, and select a maximum value of the determinant to determine the high resolution feature points in the blurred first image.

20. The non-transitory computer-readable storage medium of claim 15, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to select a predetermined number of high resolution feature points that have strongest Hessian matrix determinant values to create a set of blurred first image key points.

21. The non-transitory computer-readable storage medium of claim 15, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to utilize luminance values in a window centered about each of the blurred first image key points to extract the first feature descriptor for each of the blurred first image key points.

22. The non-transitory computer-readable storage medium of claim 15, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to apply a spatial two-dimensional index to the first feature descriptors and the second feature descriptors to limit a number of first feature descriptors and second feature descriptors for matching based on a measure of closeness.

23. The non-transitory computer-readable storage medium of claim 22, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to determine the measure of closeness with a difference operator.

24. The non-transitory computer-readable storage medium of claim 15, further comprising stored instructions for registering a first image and a second image that, when executed by a processor, cause the processor to determining a global affine transformation to map the first image into the second image based on matching pairs of blurred first image key points and blurred second image key points.

25. The non-transitory computer-readable storage medium of claim 15, in a mobile device.