Image search system, image search method, and storage medium

• US 20070019864A1
• Filed: 05/25/2006
• Published: 01/25/2007
• Est. Priority Date: 07/21/2005
• Status: Active Grant

• Current Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Original Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Inventors: Kawabe, Shigehisa, Koyama, Takahiro
• Granted Patent: US 7,801,392 B2
• Time in Patent Office: Days
• US Class Current: 382/218
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables

Abstract

An image search system includes a first calculation section that calculates a first similarity score of each registered image with respect to an input image on the basis of image features of the registered and the input image, a second calculation section that calculates a second similarity score of each registered image with respect to the input image on the basis of text features of the registered and the input image, a candidate extraction section that extracts one or more candidate images on the basis of the first and the second similarity scores of each registered image, a third calculation section that calculates a third similarity score of each candidate image on the basis of projection waveforms of the input image and the candidate image, and a search section that determines one or more registered images similar to the input image on the basis of the third similarity score.

Claims

1. An image search system, comprising:

a first calculation section that calculates a first similarity score of each registered image with respect to an input image on the basis of image features of the registered image and the input image;

a second calculation section that calculates a second similarity score of each registered image with respect to the input image on the basis of text features of the registered image and the input image;

a candidate extraction section that extracts one or more candidate registered images on the basis of the first and the second similarity scores of each registered image;

a third calculation section that calculates a third similarity score of each candidate registered image on the basis of projection waveforms of the input image and the candidate registered image; and

a search section that determines one or more registered images similar to the input image on the basis of the third similarity score of each of registered image.

2. The image search system according to claim 1, wherein

the first calculation section divides the input image into a plurality of regions and obtains an image feature amount for each region, and calculates a first similarity score of each registered image with respect to the input image on the basis of a distribution of the image feature amount for each region of the input image which is obtained and a distribution of an image feature amount for each region of each registered image.

3. The image search system according to claim 1, wherein

the second calculation section obtains a text feature amount concerning a text character string obtained by character recognition processing applied to the input image, and calculates a second similarity score of each registered image with respect to the input image on the basis of the text feature amount which is obtained and a text feature amount of each registered image.

4. The image search system according to claim 1, wherein

the third calculation section obtains a projection waveform of the input image, and calculates a third similarity score of each candidate registered image with respect to the input image, on the basis of the projection waveforms of the input image and a projection waveform of each candidate registered image.

5. The image search system according to claim 1, wherein

the candidate extraction section applies statistical normalization processing to the first similarity score of each registered image to obtain a first normalized score of each registered image, applies statistical normalization processing to the second similarity score of each registered image to obtain a second normalized score of each registered image, and extracts a candidate registered image on the basis of the first normalized score and the second normalized score.

6. The image search system according to claim 5, wherein

the candidate extraction section calculates, as the first normalized score of a registered image, a deviation value of the first similarity score of the registered image with respect to first similarity scores of all the registered images and calculates, as the second normalized score of a registered image, a deviation value of the second similarity score of the registered image with respect to second similarity scores of all the registered images.

7. An image search system, comprising:

a first calculation section that calculates a first similarity score of each registered image with respect to an input image on the basis of image features of the registered image and the input image;

a second calculation section that calculates a second similarity score of each registered image with respect to the input image on the basis of text features of the registered image and the input image;

a candidate extraction section that supplies the input image to the first calculation section or the second score calculation section, to which of the first calculation section or the second score calculation section is determined based on the amount of text in the input image, and extracts one or more candidate registered images on the basis of the resulting first similarity score or the second similarity score of each registered image;

a third calculation section that calculates a third similarity score of each candidate registered image on the basis of projection waveforms of the input image and the candidate registered image; and

a search section that determines one or more registered images similar to the input image on the basis of the third similarity score of each of registered image.

8. The image search system according to claim 7, wherein

the candidate extraction section determines, on the basis of the amount of text in the input image, which of a continuous tone image and text is dominant in the input image, and supplies the input image to the first calculation section when it is determined that a continuous tone image is dominant and supplies the input image to the second score calculation section when it is determined that the text is dominant.

9. An image search system according to claim 8, wherein

the candidate extraction section, when determining that neither the text nor the continuous tone image is dominant in the input image, supplies the input document to both the first score calculation section and the second score calculation section, and extracts one or more candidate registered images based on the first similarity score and the second similarity score of each registered image.

10. An image search system, comprising:

a first candidate extraction section that calculates a first similarity score of each registered image with respect to the input image on the basis of image features of the registered image and the input image, and extracts a first candidate registered image on the basis of the first similarity score;

a second candidate extraction section that calculates a second similarity score of each first candidate registered image with respect to the input image on the basis of text features of the first candidate registered image and the input image, and extracts a second candidate registered image on the basis of the second similarity score; and

a search section that calculates a third similarity score of each second candidate registered image with respect to the input image on the basis of projection waveforms of the input image and the candidate registered image, and determines one or more registered images similar to the input document on the basis of the third similarity score of each of registered image.

11. An image search method, comprising:

calculating a first similarity score of each registered image with respect to an input image on the basis of image features of the registered image and the input image;

calculating a second similarity score of each registered image with respect to the input image on the basis of text features of the registered image and the input image;

extracting one or more candidate registered images on the basis of the first and the second similarity scores of each registered image;

calculating a third similarity score of each candidate registered image on the basis of projection waveforms of the input image and the candidate registered image; and

determining one or more registered images similar to the input image on the basis of the third similarity score of each of registered image.

12. An image search method, comprising:

calculating, by a first calculating section, a first similarity score of each registered image with respect to an input image on the basis of image features of the registered image and the input image;

calculating, by a second calculating section, a second similarity score of each registered image with respect to the input image on the basis of text features of the registered image and the input image;

determining, on the basis of the amount of text in the input image, to which of the first calculation section or the second calculation section the input image is to be supplied;

supplying the input image to the first calculation section or the second calculation section according to the determining;

extracting one or more candidate registered images on the basis of the resulting first similarity score or the second similarity score of each registered image;

calculating a third similarity score of each candidate registered image on the basis of projection waveforms of the input image and the candidate registered image; and

determining one or more registered images similar to the input image on the basis of the third similarity score of each of registered image.

13. An image search method, comprising:

calculating a first similarity score of each registered image with respect to the input image on the basis of image features of the registered image and the input image;

extracting a first candidate registered image on the basis of the first similarity score;

calculating a second similarity score of each first candidate registered image with respect to the input image on the basis of text features of the first candidate registered image and the input image;

extracting a second candidate registered image on the basis of the second similarity score;

calculating a third similarity score of each second candidate registered image with respect to the input image on the basis of projection waveforms of the input image and the candidate registered image; and

determining one or more registered images similar to the input document on the basis of the third similarity score of each of registered image.

14. A storage medium readable by a computer, the storage medium storing a program of instructions executed by the computer to perform a function for searching one or more registered images similar to an input image, the function comprising;

calculating a first similarity score of each registered image with respect to an input image on the basis of image features of the registered image and the input image;

calculating a second similarity score of each registered image with respect to the input image on the basis of text features of the registered image and the input image;

extracting one or more candidate registered images on the basis of the first and the second similarity scores of each registered image;

calculating a third similarity score of each candidate registered image on the basis of projection waveforms of the input image and the candidate registered image; and

determining one or more registered images similar to the input image on the basis of the third similarity score of each of registered image.

15. A storage medium readable by a computer, the storage medium storing a program of instructions executed by the computer to perform a function for searching one or more registered images similar to an input image, the function comprising:

calculating, by a first calculating section, a first similarity score of each registered image with respect to an input image on the basis of image features of the registered image and the input image;

calculating, by a second calculating section, a second similarity score of each registered image with respect to the input image on the basis of text features of the registered image and the input image;

determining, on the basis of the amount of text in the input image, to which of the first calculation section or the second calculation section the input image is to be supplied;

supplying the input image to the first calculation section or the second calculation section according to the determining;

extracting one or more candidate registered images on the basis of the resulting first similarity score or the second similarity score of each registered image;

calculating a third similarity score of each candidate registered image on the basis of projection waveforms of the input image and the candidate registered image; and

determining one or more registered images similar to the input image on the basis of the third similarity score of each of registered image.

16. A storage medium readable by a computer, the storage medium storing a program of instructions executed by the computer to perform a function for searching one or more registered images similar to an input image, the function comprising:

calculating a first similarity score of each registered image with respect to the input image on the basis of image features of the registered image and the input image;

extracting a first candidate registered image on the basis of the first similarity score;

calculating a second similarity score of each first candidate registered image with respect to the input image on the basis of text features of the first candidate registered image and the input image;

extracting a second candidate registered image on the basis of the second similarity score;

calculating a third similarity score of each second candidate registered image with respect to the input image on the basis of projection waveforms of the input image and the candidate registered image; and

determining one or more registered images similar to the input document on the basis of the third similarity score of each of registered image.