Image similarity calculation system, image search system, image similarity calculation method, and image similarity calculation program

• US 7,991,232 B2
• Filed: 03/02/2005
• Issued: 08/02/2011
• Est. Priority Date: 03/03/2004
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Iwamoto, Kota
• Primary Examiner(s): Tucker; Wesley
• Assistant Examiner(s): WOLDEMARIAM, AKILILU K
• Application Number:
  US10/567,561
• Publication Number:
  US 20080152225A1
• Time in Patent Office: 2,344 Days
• Field of Search: 382/190, 382/305, 382/118, 707/6, 725/135, 348/207.99
• US Class Current: 382/190
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/758 Involving statistics of pixels or of feature values, e.g. histogram matching

Abstract

There is provided an image similarity calculation system which yields a large value for image similarity between an edited image and an original image. A local region weight calculation means 14 calculates a weight value for each local region in the image as a local region weight value, based on probability of editing each local region in the image. The local region weight calculation means 14 outputs the calculated local region weight value to an image similarity calculation means 122. A small region similarity calculation means 121 compares a feature quantity for each small region resulting from dividing an inquiry image with a feature quantity for each small region resulting from dividing a reference image. The small region similarity calculation means 121 calculates a similarity of feature quantities for the respective small regions. The image similarity calculation means 122 calculates an image similarity between the inquiry image and the reference image by applying a small-region-based weight value to the calculated similarity. This weight value is found from a local region weight value supplied from the local region weight calculation means 14.

Claims

1. An image similarity calculation system for calculating an image similarity between an inquiry image and a reference image, comprising one or more processors implementing the steps of:

an editing probability estimation unit configured to compute a value X corresponding to a probability for an editing process to be applied to an image for each local region, based on either a learning image or a device characteristic input to the editing probability estimation unit;

a local region weight calculation unit configured to calculate an image-region-based weight value based on the probability for the editing process output by the editing probability estimation unit;

a feature quantity extraction unit configured to extract a feature quantity for each divided small region of the inquiry image; and

an image similarity calculation unit configured to calculate the image similarity between the inquiry image and the reference image by comparing the feature quantity for each divided small region of the inquiry image with a feature quantity for each divided small region of the reference image, using the image-region-based weight value output by the local region weight calculation unit.

2. The image similarity calculation system according to claim 1, wherein the local region is a divided region so as to correspond to a small region in the inquiry image or the reference image.

3. The image similarity calculation system according to claim 1, wherein a small region in the inquiry image or the reference image is an equally sized rectangular region resulting from dividing an image.

4. The image similarity calculation system according to claim 1, wherein a small region in the inquiry image or the reference image is one of regions which are divided so as to be partially overlap with each other, and wherein each small region is weighted in proportion to overlapping areas by the local region weight calculation unit.

5. The image similarity calculation system according to claim 1, wherein a small region in the inquiry image or the reference image results from dividing only part of an image.

6. The image similarity calculation system according to claim 1, wherein the feature quantity is based on at least one of color information, edge information, texture information, shape information, and motion information.

7. The image similarity calculation system according to claim 1, wherein the feature quantity is at least one of an average value, a mode value, and a median value for color coordinates specified in color space systems such as RGB color space, HSV color space, YUV color space, YIQ color space, YCbCr color space, L*a*b* color space, and XYZ color space, and Dominant Color, Color Layout, Scalable Color, Color Structure, Edge Histogram, Homogeneous Texture, Texture Browsing, Contour Shape, and Shape 3D specified in international standard ISO/IEC15938-3.

8. The image similarity calculation system according to claim 1, wherein the editing process corresponds to at least one of superposing a ticker on an image, superposing a caption on an image, superposing a character on an image, superposing an object on an image, partially cutting an image, partially cropping an image, partially mosaicking an image, and partially blurring an image.

9. An image retrieval system to retrieve images using an image similarity calculated in the image similarity calculation system according to claim 1.

10. An image retrieval system to output an image similar to an inquiry image from a plurality of reference images based on a calculated image similarity using an image similarity calculation system according to claim 1.

13. The image similar calculation system according to claim 1, wherein the image-region-based weight value is set to be larger in proportion to a decrease in the value X.

14. The image similar calculation system according to claim 13, wherein the image-region-based weight value is computed by the equation 1−

X.

11. An image similarity calculation method for calculating an image similarity between an inquiry image and a reference image, comprising one or more processors implementing the following steps:

computing a value X corresponding to a probability for an editing process to be applied to an image for each local region, based on either a learning image or a device characteristic;

calculating an image-region-based weight value based on the probability for the editing process computed by the computing step;

extracting a feature quantity for each divided small region of the inquiry image; and

calculating the image similarity between the inquiry image and the reference image by comparing the feature quantity for each divided small region of the inquiry image with a feature quantity for each divided small region of the reference image, using the image-region-based weight value output by the calculating step.

12. A non-transitory computer readable medium storing an image similarity calculation program for calculating an image similarity between an inquiry image and a reference image, the image similarity calculation program allowing a computer to perform a process of:

computing a value X corresponding to a probability for an editing process to be applied to an image for each local region, based on either a learning image or a device characteristic;

calculating an image-region-based weight value based on the probability for the editing process computed by the computing step;

extracting a feature quantity for each divided small region of the inquiry image; and

calculating the image similarity between the inquiry image and the reference image by comparing the feature quantity for each divided small region of the inquiry image with a feature quantity for each divided small region of the reference image, using the image-region-based weight value output by the calculating step.