Scale space normalization technique for improved feature detection in uniform and non-uniform illumination changes

• US 8,582,889 B2
• Filed: 01/07/2011
• Issued: 11/12/2013
• Est. Priority Date: 01/08/2010
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Vaddadi, Sundeep, Hong, John H., Hamsici, Onur C., Lee, Chong U.
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US12/986,607
• Publication Number:
  US 20110170780A1
• Time in Patent Office: 1,040 Days
• Field of Search: 382/159, 382/195, 382/201
• US Class Current: 382/195
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

A normalization process is implemented at a difference of scale space to completely or substantially reduce the effect that illumination changes has on feature/keypoint detection in an image. An image may be processed by progressively blurring the image using a smoothening function to generate a smoothened scale space for the image. A difference of scale space may be generated by taking the difference between two different smoothened versions of the image. A normalized difference of scale space image may be generated by dividing the difference of scale space image by a third smoothened version of the image, where the third smoothened version of the image that is as smooth or smoother than the smoothest of the two different smoothened versions of the image. The normalized difference of scale space image may then be used to detect one or more features/keypoints for the image.

Claims

1. A method operational in an image feature detection device, comprising:

generating a difference of scale space image by taking the difference between two different smoothened versions of an image;

generating a normalized difference of scale space image by dividing the difference of scale space image by a third smoothened version of the image, where the third smoothened version of the image is as smooth or smoother than the smoothest of the two different smoothened versions of the image;

modifying both the difference of scale space image and the normalized difference of scale space image by an illumination scaling function; and

using the normalized difference of scale space image to detect one or more features for the image.

2. The method of claim 1, further comprising:

convolving the image with a plurality of smoothening kernels of different scaling widths to obtain a plurality of smoothened versions of the image, the plurality of smoothened versions defining a scale space pyramid.

3. The method of claim 2, wherein the two different smoothened versions of the image are adjacent levels in the smoothened scale space pyramid.

4. The method of claim 2, further comprising:

generating a difference of scale space by taking the difference between pairs of different smoothened versions of the image, the difference of scale space image forming part of the difference of scale space.

5. The method of claim 4, wherein the pairs of different smoothened versions of the image are adjacent levels in the smoothened scale space pyramid.

6. The method of claim 2, wherein the third smoothened version of the image has an equal or wider scale than the greater of the scales for the two different smoothened versions of the image.

7. The method of claim 1, further comprising:

identifying the one or more features for the image using the normalized difference of scale space by detecting local extrema within the normalized difference of scale space image.

8. The method of claim 7, wherein each feature includes a keypoint and information about its surrounding points.

9. The method of claim 1, further comprising:

generating descriptors for the image based on the identified one or more features.

29. The method of claim 1, further comprising neutralizing the effect of illumination by a normalization process that factors out the illumination scaling function.

30. The method of claim 1, wherein modifying both the difference of scale space image and the normalized difference of scale space image by an illumination scaling function comprises multiplying both the numerator and the denominator of a normalizing function by the illumination scaling function.

31. The method of claim 30, wherein multiplying both the numerator and the denominator of a normalizing function by the illumination scaling function completely or substantially cancels the effect of the illumination scaling function.

32. The method of claim 1, wherein the illumination scaling function is spatially varying.

33. The method of claim 1, wherein the illumination scaling function is spatially constant.

10. An image feature detection device, comprising:

a difference of scale space generator adapted to generate a difference of scale space image by taking the difference between two different smoothened versions of an image;

a difference of scale space generator normalizer adapted to generate a normalized difference of scale space image by dividing the difference of scale space image by a third smoothened version of the image, where the third smoothened version of the image is as smooth or smoother than the smoothest of the two different smoothened versions of the image; and

a feature detector adapted to use the normalized difference of scale space image to detect one or more features for the image;

wherein both the difference of scale space image and the normalized difference of scale space image are modified by an illumination scaling function.

11. The image feature detection device of claim 10, further comprising:

a scale space generator adapted to convolve the image with a plurality of smoothening kernels of different scaling widths to obtain a plurality of smoothened versions of the image, the plurality of smoothened versions defining a scale space pyramid.

12. The image feature detection device of claim 11, wherein the two different smoothened versions of the image are adjacent levels in the smoothened scale space pyramid.

13. The image feature detection device of claim 11, wherein the difference of scale space generator is adapted to generate a difference of scale space by taking the difference between pairs of different smoothened versions of the image, the difference of scale space image forming part of the difference of scale space.

14. The image feature detection device of claim 13, wherein the pairs of different smoothened versions of the image are adjacent levels in the smoothened scale space pyramid.

15. The image feature detection device of claim 11, wherein the third smoothened version of the image has an equal or wider scale than the greater of the scales for the two different smoothened versions of the image.

16. The image feature detection device of claim 10, wherein the feature detector is adapted to identify features for the image using the normalized difference of scale space by detecting local extrema within the normalized difference of scale space image.

17. The image feature detection device of claim 10, further comprising:

a feature descriptor generator adapted to generate descriptors for the image based on the identified one or more features.

18. A image feature detection device, comprising:

means for generating a difference of scale space image by taking the difference between two different smoothened versions of an image;

means for generating a normalized difference of scale space image by dividing the difference of scale space image by a third smoothened version of the image, where the third smoothened version of the image is as smooth or smoother than the smoothest of the two different smoothened versions of the image;

means for using the normalized difference of scale space image to detect one or more features for the image; and

means for modifying both the difference of scale space image and the normalized difference of scale space image by an illumination scaling function.

19. The image feature detection device of claim 18, further comprising:

means for convolving the image with a plurality of smoothening kernels of different scaling widths to obtain a plurality of smoothened versions of the image, the plurality of smoothened versions defining a scale space pyramid.

20. The image feature detection device of claim 19, further comprising:

means for generating a difference of scale space by taking the difference between pairs of different smoothened versions of the image, the difference of scale space image forming part of the difference of scale space.

21. The image feature detection device of claim 20, wherein the pairs of different smoothened versions of the image are adjacent levels in a smoothened scale space pyramid.

22. The image feature detection device of claim 18, further comprising:

means for identifying the one or more features for the image using the normalized difference of scale space by detecting local extrema within the normalized difference of scale space image.

23. The image feature detection device of claim 18, further comprising:

means for generating descriptors for the image based on the identified one or more features.

24. A non-transitory processor-readable medium having one or more instructions operational on an image feature detection device, which when executed by a processor causes the processor to:

generate a difference of scale space image by taking the difference between two different smoothened versions of an image;

generate a normalized difference of scale space image by dividing the difference of scale space image by a third smoothened version of the image, where the third smoothened version of the image is as smooth or smoother than the smoothest of the two different smoothened versions of the image; and

use the normalized difference of scale space image to detect one or more features for the image;

wherein both the difference of scale space image and the normalized difference of scale space image are modified by an illumination scaling function.

25. The non-transitory processor-readable medium of claim 24, having one or more instructions which when executed by a processor causes the processor to further:

convolve the image with a plurality of smoothening kernels of different scaling widths to obtain a plurality of smoothened versions of the image, the plurality of smoothened versions defining a scale space pyramid.

26. The non-transitory processor-readable medium of claim 25, having one or more instructions which when executed by a processor causes the processor to further:

generate a difference of scale space by taking the difference between pairs of different smoothened versions of the image, the difference of scale space image forming part of the difference of scale space.

27. The non-transitory processor-readable medium of claim 24, having one or more instructions which when executed by a processor causes the processor to further:

identify the one or more features for the image using the normalized difference of scale space by detecting local extrema within the normalized difference of scale space image.

28. The non-transitory processor-readable medium of claim 24, having one or more instructions which when executed by a processor causes the processor to further:

generate descriptors for the image based on the identified one or more features.