Descriptors adjustment when using steerable pyramid to extract features for content based search

• US 6,674,915 B1
• Filed: 11/15/1999
• Issued: 01/06/2004
• Est. Priority Date: 10/07/1999
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Wang, Lizhi
• Primary Examiner(s): Dastouri, Mehrdad
• Application Number:
  US09/440,491
• Time in Patent Office: 1,513 Days
• Field of Search: 382/190, 382/205, 382/219, 382/220, 382/221, 382/240, 382/260, 382/261, 382/262, 382/263, 382/264, 382/265, 382/274, 382/305, 382/130, 382/170, 382/181, 345/625, 345/853, 348/96, 348/97, 348/107, 707/3, 707/4, 707/530
• US Class Current: 382/263
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 10/52 Scale-space analysis, e.g. wavelet analysis multi-scale boundary representations G06V10/42

Abstract

A method of adjusting image descriptors when using steerable pyramid to extract features for content-based search is disclosed in this invention. Using steerable pyramid, the original image is filtered to produce Gaussian images and Laplacian images. The image descriptors are formed by filtering the Laplacian images. The filtering of the Laplacian images provides orientation data and energy data. From the orientation data, a first set of global information and local information is generated. From the energy data, a second set of global information and local information is generated. The descriptors are extracted from the respective sets of global information and local information associated with the orientation data and the energy data. The descriptors associated with the orientation data and the descriptors associated with the energy data are combined to form an image descriptor. The Laplacian image may be adjusted prior to the application of the filter. The adjustment is necessary since the value of the Laplacian image may not always be positive. When the adjustment is made to the Laplacian image, a spatial shift occurs to the corresponding energy data.

Claims

1. A method comprising:

getting a Laplacian image from corresponding Gaussian images in a steerable pyramid;

verifying the Laplacian image for negative value;

adjusting the Laplacian image to eliminate the negative value;

applying a steerable filter to the adjusted Laplacian image to generate orientation data and energy data;

removing resulting adjustment;

extracting a plurality of descriptors from the orientation data and the energy data; and

combining the plurality of descriptors into an image descriptor to be used for image processing.

2. The method of claim 1 wherein getting the Laplacian image comprises subtracting the two neighboring Gaussian images.

3. The method of claim 1 wherein verifying comprises measuring intensity value of the Laplacian image.

4. The method of claim 1 wherein adjusting comprises adding a constant factor to the Laplacian image such that when the steerable filter is applied to the adjusted Laplacian image, the resulting orientation map and energy map are not negative.

5. The method of claim 1 wherein removing resulting adjustment comprises compensating for a spatial shift to the energy data, the spatial shift resulting from the adjustment added to the Laplacian image.

6. A method of filtering images, comprising:

constructing a plurality of orientation maps and a plurality of energy maps from input images, the input images adjusted by a factor prior to applying filters, the adjustment factor being such that the plurality of orientation maps and the plurality of energy maps are non-negative;

removing resulting adjustment factors from the plurality of energy maps;

extracting, for each input image, a plurality of descriptors from corresponding non-negative orientation and energy maps; and

combining the plurality of descriptors into an image descriptor to be used for image processing.

7. The method of claim 6 wherein the input images are Laplacian images, the Laplacian images constructed by subtracting two corresponding Gaussian images in a steerable pyramid.

8. The method of claim 6 wherein the filters are steerable filters.

9. The method of claim 6 wherein the adjustment is made to the intensity value of the Laplacian images.

10. A method comprising:

filtering an input image at a first time to produce a corresponding orientation map and a corresponding energy map, each of the orientation map and the energy map being represented by a corresponding histogram and a corresponding set of co-occurrence matrices;

adjusting the input image by an adjustment factor if the orientation map and/or the energy map is negative, the adjustment factor being such that when the adjusted input image is filtered, the orientation map and/or the energy map is non-negative;

filtering the adjusted input image at a second time to produce non-negative orientation and energy maps;

extracting a plurality of descriptors from the non-negative orientation and energy maps; and

combining the plurality of descriptors into an image descriptor to be used for image processing.

11. The method of claim 10, wherein the filtering is done by applying steerable filters and wherein the input image is a Laplacian image in a steerable pyramid.

12. The method of claim 10, wherein adjusting the input image by an adjustment factor comprises determining the value of the Laplacian image, and adding the adjustment factor such that the value of the Laplacian image is non-negative.

13. The method of claim 10 wherein the intensity value is the value of the Laplacian image.

14. A computer readable medium containing executable instructions which, when executed in a processing system, causes the system to perform a method comprising:

getting a Laplacian image from corresponding Gaussian images in a steerable pyramid;

verifying the Laplacian image for negative value;

adjusting the Laplacian image to eliminate the negative value;

applying a steerable filter to the adjusted Laplacian image to generate orientation data and energy data;

removing resulting adjustments;

extracting a plurality of descriptors from the orientation data and the energy data; and

combining the plurality of descriptors into an image descriptor to be used for image processing.

15. The computer readable medium of claim 14, wherein getting the Laplacian image comprises subtracting the two neighboring Gaussian images.

16. The computer readable medium of claim 14, wherein verifying comprises measuring intensity value of the Laplacian image.

17. The computer readable medium of claim 14, wherein adjusting comprises adding a constant factor to the Laplacian image such that when the steerable filter is applied to the adjusted Laplacian image, the resulting orientation map and energy map are not negative.

18. The computer readable medium of claim 14, wherein removing resulting adjustment comprises compensating the resulting adjustment factor from the energy map, the resulting adjustment factor corresponding to the adjustment added to the Laplacian image.

19. An apparatus, comprising:

means for filtering input images to produce corresponding orientation maps and energy maps, each of the orientation maps and the energy maps being represented by a corresponding histogram and a corresponding set of co-occurrence matrices;

means for adjusting the input images such that the corresponding orientation maps and/or energy maps are non-negative;

means for extracting, for each input image, a plurality of descriptors from corresponding non-negative orientation and energy maps; and

combining the plurality of descriptors into an image descriptor to be used for image processing.

20. The apparatus of claim 19, wherein means for filtering input images comprises means for generating Gaussian images from an original image and means for generating Laplacian images from the Gaussian images, and wherein steerable filters are used in filtering the Laplacian images.

21. The apparatus of claim 19, wherein means for adjusting the input images comprises means for calculating intensity value of the Laplacian images and means for adding an adjustment factor to the Laplacian images such that the intensity value is non-negative.

22. A method comprising:

filtering input images to produce corresponding orientation maps and energy maps, each of the orientation maps and the energy maps being represented by a corresponding histogram and a corresponding set of co-occurrence matrices;

adjusting the input images such that the corresponding orientation maps and/or energy maps are non-negative;

extracting a plurality of descriptors from corresponding non-negative orientation and energy maps; and

combining the plurality of descriptors into an image descriptor to be used for image processing.

23. The method of claim 22, wherein filtering input images comprises:

generating Gaussian images from an original image;

generating Laplacian images from the Gaussian images; and

filtering the Laplacian images by using steerable filters.

24. The method of claim 22, wherein adjusting the input images comprises:

calculating intensity value of the Laplacian images; and

adding an adjustment factor to the Laplacian images such that the intensity value is non-negative.

25. A computer system comprising:

a processor; and

an image processing logic coupled to the processor, comprising

generation logic to form Gaussian images from an original image and to form a Laplacian image from the corresponding two Gaussian images,

adjustment logic to add a value to the Laplacian image such that the Laplacian image is non-negative,

logic to apply a steerable filter to the adjusted Laplacian image to produce an energy map and an orientation map, each of the orientation map and the energy map being represented by a corresponding histogram and a corresponding set of co-occurrence matrices; and

image descriptor logic to extract a plurality of descriptors from the orientation map and the energy map, and to combine the plurality of descriptors into an image descriptor to be used for image processing.

26. The computer system of claim 25 further comprising logic to compensate for partial shift to the energy map, the partial shift resulting from the corresponding adjustment made to the Laplacian image.

27. The computer system of claim 25 wherein the Gaussian images and the Laplacian images are in a steerable pyramid, and wherein the value of the Laplacian image is measured by its intensity.