EFFICIENT DESCRIPTOR EXTRACTION OVER MULTIPLE LEVELS OF AN IMAGE SCALE SPACE

• US 20110255781A1
• Filed: 04/19/2011
• Published: 10/20/2011
• Est. Priority Date: 04/20/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Lee, Chong Uk., Hong, John H., Reznik, Yuriy, Hamsici, Onur C., Vaddadi, Sundeep
• Granted Patent: US 9,530,073 B2
• Time in Patent Office: Days
• US Class Current: 382/170
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

A local feature descriptor for a point in an image is generated over multiple levels of an image scale space. The image is gradually smoothened to obtain a plurality of scale spaces. A point may be identified as the point of interest within a first scale space from the plurality of scale spaces. A plurality of image derivatives is obtained for each of the plurality of scale spaces. A plurality of orientation maps is obtained (from the plurality of image derivatives) for each scale space in the plurality of scale spaces. Each of the plurality of orientation maps is then smoothened (e.g., convolved) to obtain a corresponding plurality of smoothed orientation maps. Therefore, a local feature descriptor for the point may be generated by sparsely sampling a plurality of smoothed orientation maps corresponding to two or more scale spaces from the plurality of scale spaces.

Claims

1. A method for generating a local feature descriptor for an image, comprising:

identifying a point within a first scale space from a plurality of scale spaces for an image;

obtaining a plurality of orientation maps for each scale space in the plurality of scale spaces;

smoothing each of the plurality of orientation maps to obtain a corresponding plurality of smoothed orientation maps; and

sparsely sampling a plurality of smoothed orientation maps corresponding to two or more scale spaces from the plurality of scale spaces to generate a local feature descriptor for the point.

2. The method of claim 1, further comprising:

gradually smoothing the image to obtain the plurality of scale spaces, wherein the smoothing of an orientation map, within the plurality of smoothed orientation maps, is proportional to the scale space level of the orientation map.

3. The method of claim 1, further comprising:

obtaining a plurality of image derivatives for each of the plurality of scale spaces, where each of the plurality of orientation maps is obtained from non-negative values of a corresponding image derivative.

4. The method of claim 1, wherein the point is a sample point from a subset of locations within the plurality of scale spaces.

5. The method of claim 4, wherein the subset of locations is selected based on an expected pattern for an object.

6. The method of claim 4, wherein the subset of locations is selected based on identified keypoints within the image, wherein a keypoint is a point that has been identified as being robust to changes in imaging conditions.

7. The method of claim 1, wherein the two or more scale spaces include the first scale space and one or more additional scale spaces of lower resolution than the first scale space.

8. The method of claim 1, wherein the local feature descriptor has a kernel pooling configuration defined by spatial pooling of sample points distributed over a center of the point.

9. The method of claim 1, wherein sparsely sampling a plurality of smoothed orientation maps includes

sampling a first plurality of points on a first smoothed orientation map, the first plurality of points arranged in a first ring concentric with a location of the point;

and

sampling a second plurality of points on a second smoothed orientation map, the second plurality of points arranged in a second ring concentric with the location of the point, the second smoothed orientation map corresponding to a second scale space of lower resolution than the first scale space.

10. The method of claim 4, wherein sparsely sampling a plurality of smoothed orientation maps further includes

sampling a third plurality of points on a third smoothed orientation map, the third plurality of points arranged in a third ring concentric with the location of the point, the third smoothed orientation map corresponding to a third scale space of lower resolution than the first scale space.

11. The method of claim 10, wherein the second ring has a second radius greater than a first radius for the first ring, and the third ring has a third radius greater than the second radius for the second ring.

12. The method of claim 1, wherein the plurality of orientation maps for each scale space include orientation maps for a plurality of different orientations.

13. The method of claim 1, wherein each orientation map resolves to a single corresponding smoothed orientation map.

14. The method of claim 1, further comprising:

building a plurality of histograms of oriented gradients from the sparse sampling of the plurality of smoothed orientation maps, wherein the local feature descriptor comprises the plurality of histograms.

15. The method of claim 1, wherein the plurality of orientation maps are smoothed using the same smoothing coefficient, the first scale space is one of the two or more scale spaces, and a second scale space is selected to achieve a desired smoothing relative to the first scale space.

16. An image processing device, comprising:

a input interface adapted to obtain an image;

a storage device to store local feature descriptors for one or more images;

a processing circuit coupled to the input interface and the storage device, the processing device adapted to;

identify a point within a first scale space from a plurality of scale spaces for an image;

obtain a plurality of orientation maps for each scale space in the plurality of scale spaces;

smooth each of the plurality of orientation maps to obtain a corresponding plurality of smoothed orientation maps; and

sparsely sample a plurality of smoothed orientation maps corresponding to two or more scale spaces from the plurality of scale spaces to generate a local feature descriptor for the point.

17. The device of claim 16, wherein the two or more scales spaces include the first scale space and one or more additional scale spaces of lower resolution than the first scale space.

18. The device of claim 16, wherein the processing device is further adapted to:

obtain a plurality of image derivatives for each of the plurality of scale spaces, where each of the plurality of orientation maps is obtained from non-negative values of a corresponding image derivative.

19. The device of claim 16, wherein the point is a sample point from a subset of locations within the plurality of scale spaces.

20. The device of claim 16, wherein the local feature descriptor has a kernel pooling configuration defined by spatial pooling of sample points distributed over a center on the point.

21. The device of claim 16, wherein sparsely sampling a plurality of smoothed orientation maps includes

sampling a first plurality of points on a first smoothed orientation map, the first plurality of points arranged in a first ring concentric with a location of the point;

and

sampling a second plurality of points on a second smoothed orientation map, the second plurality of points arranged in a second ring concentric with the location of the point, the second smoothed orientation map corresponding to a second scale space of lower resolution than the first scale space.

22. The device of claim 21, wherein sparsely sampling a plurality of smoothed orientation maps further includes

sampling a third plurality of points on a third smoothed orientation map, the third plurality of points arranged in a third ring concentric with the location of the point, the third smoothed orientation map corresponding to a third scale space of lower resolution than the first scale space.

23. The device of claim 16, wherein the processing device is further adapted to:

build a plurality of histograms of oriented gradients from the sparse sampling of the plurality of smoothed orientation maps, wherein the local feature descriptor comprises the plurality of histograms.

24. The device of claim 16, wherein the plurality of orientation maps are smoothed using the same smoothing coefficient, the first scale space is one of the two or more scale spaces, and a second scale space is selected to achieve a desired smoothing relative to the first scale space.

25. An image processing device, comprising:

means for identifying a point within a first scale space from a plurality of scale spaces for an image;

means for obtaining a plurality of orientation maps for each scale space in the plurality of scale spaces;

means for smoothing each of the plurality of orientation maps to obtain a corresponding plurality of smoothed orientation maps; and

means for sparsely sampling a plurality of smoothed orientation maps corresponding to two or more scale spaces from the plurality of scale spaces to generate a local feature descriptor for the point.

26. The device of claim 25, further comprising:

means for obtaining a plurality of image derivatives for each of the plurality of scale spaces, where each of the plurality of orientation maps is obtained from non-negative values of a corresponding image derivative.

27. The device of claim 25, wherein the point is a sample point from a subset of locations within the plurality of scale spaces.

28. The device of claim 25, wherein the two or more scales spaces include the first scale space and one or more additional scale spaces of lower resolution than the first scale space.

29. The device of claim 25, wherein the local feature descriptor has a kernel pooling configuration defined by spatial pooling of sample points distributed over a center on the point.

30. The device of claim 25, wherein means for sparsely sampling a plurality of smoothed orientation maps includes

means for sampling a first plurality of points on a first smoothed orientation map, the first plurality of points arranged in a first ring concentric with a location of the point;

and

means for sampling a second plurality of points on a second smoothed orientation map, the second plurality of points arranged in a second ring concentric with the location of the point, the second smoothed orientation map corresponding to a second scale space of lower resolution than the first scale space.

31. The device of claim 30, wherein means for sparsely sampling a plurality of smoothed orientation maps further includes

means for sampling a third plurality of points on a third smoothed orientation map, the third plurality of points arranged in a third ring concentric with the location of the point, the third smoothed orientation map corresponding to a third scale space of lower resolution than the first scale space.

32. The device of claim 25, further comprising:

means for building a plurality of histograms of oriented gradients from the sparse sampling of the plurality of smoothed orientation maps, wherein the local feature descriptor comprises the plurality of histograms.

33. A processor-readable medium comprising one or more instructions operational in a device, which when executed by a processing circuit, causes the processing circuit to:

identify a point within a first scale space from a plurality of scale spaces for an image;

obtain a plurality of orientation maps for each scale space in the plurality of scale spaces;

smooth each of the plurality of orientation maps to obtain a corresponding plurality of smoothed orientation maps; and

sparsely sample a plurality of smoothed orientation maps corresponding to two or more scale spaces from the plurality of scale spaces to generate a local feature descriptor for the point.

34. The processor-readable medium of claim 33, wherein the two or more scales spaces include the first scale space and one or more additional scale spaces of lower resolution than the first scale space.