Image retrieval device, image retrieval method and storage medium storing similar-image retrieval program

• US 7,006,714 B2
• Filed: 12/11/2000
• Issued: 02/28/2006
• Est. Priority Date: 12/10/1999
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Kasutani, Eiji
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): KIM, CHONG R
• Application Number:
  US09/732,701
• Publication Number:
  US 20010041008A1
• Time in Patent Office: 1,905 Days
• Field of Search: 382/305, 382/209, 382/217, 382/218, 358/403, 707/6
• US Class Current: 382/305
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06V 10/431 Frequency domain transformation; Autocorrelation
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

An image retrieval device and method capable of high speed image retrieval. The image retrieval device has a first coefficient transforming section to transform a first group of image feature descriptors extracted from image data accumulated in an image database and to generate a second group of image feature descriptors to be used for calculating similarity. The image retrieval device has a second coefficient transforming section to transform the first group of image feature descriptors extracted from image data of an inquired image and to generate a second group of image feature descriptors to be used for calculating similarity. The image retrieval device has a similarity calculating section to compare the second group of image feature descriptors for each piece of image data produced by the first coefficient transforming section with the second group of image feature descriptors for the inquired image transformed by the second coefficient transforming section.

Claims

1. An image retrieval device for retrieving an image being similar to an inquired image from images stored in an image database comprising:

a first coefficient transforming means for transforming a first group of image feature descriptors extracted from image data accumulated in said image database and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said image data accumulated in said image database;

a second coefficient transforming means for transforming a first group of image feature descriptors extracted from image data of said inquired image and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said inquired image; and

a similarity calculating means for calculating similarity by comparing said second group of image feature descriptors for each piece of image data generated by said first coefficient transforming means with said second group of image feature descriptors transformed by said second coefficient transforming means, wherein said first coefficient transforming means and said second coefficient transforming means perform coefficient transforms using a transform table selected depending on a kind of transform coefficient to be used as said image feature descriptor and wherein said first coefficient transforming means and said second coefficient transforming means perform retrieval of a similar image on a trial basis using a plurality of said transform tables each having a differently segmented range of said transform coefficient and select a transform table which has showed a high rate of correctly solved retrieval in said trial basis retrieval.

2. The image retrieval device according to claim 1, further comprising an image feature descriptor storing means and wherein said similarity calculating means compares said second group of image feature descriptors of image data of said inquired image received from said second coefficient transforming means with said second group of image feature descriptors of image data contained in said image database read from said image feature descriptor storing means.

3. The image retrieval device according to claim 1, wherein said first coefficient transforming means and said second coefficient transforming means transform image feature descriptors so that visual similarity between images to be compared is approximated by a distance between an image expressed by said second group of image feature descriptors of image data contained in said image database and an image expressed by said second group of image feature descriptors of image data of said inquired image.

4. An image retrieval device for retrieving an image being similar to an inquired image from images stored in an image database comprising:

a first coefficient transforming means for transforming a first group of image feature descriptors extracted from image data accumulated in said image database and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said image data accumulated in said image database;

a second coefficient transforming means for transforming a first group of image feature descriptors extracted from image data of said inquired image and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said inquired image; and

a similarity calculating means for calculating similarity by comparing said second group of image feature descriptors for each piece of image data generated by said first coefficient transforming means with said second group of image feature descriptors transformed by said second coefficient transforming means, wherein said first coefficient transforming means and said second coefficient transforming means perform coefficient transforms in a manner that fine quantization is carried out on a range of a portion of each of said transform coefficients having a small amplitude and that coarse quantization is carried out on a range of a portion of each of said transform coefficients having a large amplitude.

5. The image retrieval device according to claim 4, wherein said first coefficient transforming means and said second coefficient transforming means perform coefficient transforms in a manner that fine quantization is carried out on said coefficient having a small power and coarse quantization is carried out on said coefficient having a large power.

6. An image retrieval device for retrieving an image being similar to an inquired image from images stored in an image database comprising:

a first coefficient transforming means for transforming a first group of image feature descriptors extracted from image data accumulated in said image database and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said image data accumulated in said image database;

a second coefficient transforming means for transforming a first group of image feature descriptors extracted from image data of said inquired image and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said inquired image; and

a similarity calculating means for calculating similarity by comparing said second group of image feature descriptors for each piece of image data generated by said first coefficient transforming means with said second group of image feature descriptors transformed by said second coefficient transforming means, wherein said first coefficient transforming means and said second coefficient transforming means perform coefficient transforms in a manner that fine quantization is carried out on said transform coefficients having a small power and coarse quantization is carried out on said transform coefficients having a large power.

7. An image retrieving method for retrieving an image being similar to an inquired image from images stored in an image database, said method comprising:

transforming a first group of image feature descriptors extracted from image data accumulated in said image database and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said image data accumulated in said image database;

transforming a first group of image feature descriptors extracted from image data of said inquired image and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said inquired image; and

comparing said second group of image feature descriptors of image data accumulated in said image database with said second group of image feature descriptors of image data of said inquired image to calculate similarity,

wherein said specified transform processing is performed using a transform table selected depending on a kind of transform coefficient to be used as said image feature descriptor and wherein retrieval of a similar image is performed on a trial basis using a plurality of said transform tables each having a differently segmented range of said transform coefficient and a transform table is selected which has showed a high rate of correctly solved retrieval in said trial basis retrieval.

8. An image retrieval device for retrieving an image being similar to an inquired image from images stored in an image database comprising:

a first coefficient transforming means for transforming a first group of image feature descriptors extracted from image data accumulated in said image database and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said image data accumulated in said image database;

a second coefficient transforming means for transforming a first group of image feature descriptors extracted from image data of said inquired image and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said inquired image;

a similarity calculating means for calculating similarity by comparing said second group of image feature descriptors for each piece of image data generated by said first coefficient transforming means with said second group of image feature descriptors transformed by said second coefficient transforming means, wherein said first coefficient transforming means and said second coefficient transforming means perform coefficient transforms in a manner that fine quantization is carried out on said transform coefficients having a small power and coarse quantization is carried out on said transform coefficients having a large power;

an image size resizing means for resizing image data accumulated in said image database and/or inquired image; and

an image feature descriptor producing means for performing an orthogonal transform on an image obtained by said image size resizing means and producing an orthogonal transform coefficient and using said orthogonal transform coefficient as a first group of image feature descriptors, wherein said image size resizing means comprising a block dividing means for partitioning said image data into blocks, a dominant color calculating means for calculating a dominant color of each of blocks obtained by said block dividing means and an image creating means for creating an image using said dominant color of each of said blocks as a pixel.

9. The image retrieval device according to claim 8, wherein said image creating means extracts a color average of entire pixels contained in each of said blocks as said calculated dominant color of each of said blocks.

10. The image retrieval device according to claim 8, wherein said block dividing means partitions said image into 64 blocks.

11. An image retrieval device for retrieving an image being similar to an inquired image from images stored in an image database comprising:

a first coefficient transforming means for transforming a first group of image feature descriptors extracted from image data accumulated in said image database and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said image data accumulated in said image database;

a second coefficient transforming means for transforming a first group of image feature descriptors extracted from image data of said inquired image and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said inquired image;

an image size resizing means for resizing image data accumulated in said image database and/or inquired image, wherein said image size resizing means comprises a block dividing means for partitioning said image data into blocks, a dominant color calculating means for calculating a dominant color of each of blocks obtained by said block dividing means and an image creating means for creating an image using said dominant color of each of said blocks as a pixel;

an image feature descriptor producing means for performing a discrete cosine transform on an image obtained by said image size resizing means to produce a discrete cosine transform coefficient for use as an extracted first group of image feature descriptors; and

a similarity calculating means for calculating similarity by comparing said second group of image feature descriptors generated by said first coefficient transforming means with said second group of image feature descriptors generated by said second coefficient transforming means, wherein said first coefficient transforming means and said second coefficient transforming means perform coefficient transforms in a manner that fine quantization is carried out on said transform coefficients having a small power and coarse quantization is carried out on said transform coefficients having a large power.

12. An image retrieval device for retrieving an image being similar to an inquired image from images stored in an image database comprising:

a coefficient transforming means for transforming first groups of image feature descriptors extracted from image data accumulated in said image database and from image data of said inquired image and then generating second groups of image feature descriptors used to calculate similarity, wherein said second groups of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said image data accumulated in said image database and on said inquired image;

a similarity calculating means for comparing said second groups of image feature descriptors generated by said coefficient transforming means, wherein said coefficient transforming means performs coefficient transforms in a manner that fine quantization is carried out on said transform coefficients having a small power and coarse quantization is carried out on said transform coefficients having a large power and then calculating similarity between an image accumulated in said image database and said inquired image;

an image size resizing means for resizing image data accumulated in said image database and/or inquired image; and

an image feature descriptor producing means for performing an orthogonal transform on an image obtained by said image size resizing means and producing an orthogonal transform coefficient and using said orthogonal transform coefficient as a first group of image feature descriptors, wherein said image size resizing means comprises a block dividing means for partitioning said image data into blocks, a dominant color calculating means for calculating a dominant color of each of blocks obtained by said block dividing means and an image creating means for creating an image using said dominant color of each of said blocks as a pixel.

13. The image retrieval device according to claim 12, wherein said image creating means extracts a color average of entire pixels contained in each of said blocks as said calculated dominant color of each of said blocks.

14. The image retrieval device according to claim 12, wherein said block dividing means partitions said image into 64 blocks.

15. An image retrieval device for retrieving an image being similar to an inquired image from images stored in an image database comprising:

a coefficient transforming means for transforming a first group of image feature descriptors extracted from image data accumulated in said image database and then generating a second group of image feature descriptors used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said image data accumulated in said image database, and for transforming a first group of image feature descriptors extracted from image data of said inquired image and then generating a second group of image feature descriptors used to calculate similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said inquired image,

wherein said coefficient transforming means comprises an image size resizing means for resizing image data accumulated in said image database and/or inquired image, wherein said image size resizing means comprises a block dividing means for partitioning said image data into blocks, a dominant color calculating means for calculating a dominant color of each of blocks obtained by said block dividing means and an image creating means for creating an image using said dominant color of each of said blocks as a pixel, and an image feature descriptor producing means for performing a discrete cosine transform on an image obtained by said image size resizing means to produce a discrete cosine transform coefficient for use as an extracted first group of image feature descriptors; and

a similarity calculating means for comparing said generated second group of image feature descriptors of said image data accumulated in said image database with said generated second group of image feature descriptors of said inquired image, wherein said coefficient transforming means performs coefficient transforms in a manner that fine quantization is carried out on said transform coefficients having a small power and coarse quantization is carried out on said transform coefficients having a large power.

16. An image retrieving method for retrieving an image being similar to an inquired image from images stored in an image database, said method comprising:

transforming a first group of image feature descriptors extracted from image data accumulated in said image database and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said image data accumulated in said image database;

transforming a first group of image feature descriptors extracted from image data of said inquired image and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said inquired image;

comparing said second group of image feature descriptors of image data accumulated in said image database with said second group of image feature descriptors of image data of said inquired image to calculate similarity, wherein said specified coefficient transform processing is performed in a manner that fine quantization is carried out on said transform coefficients having a small power and coarse quantization is carried out on said transform coefficients having a large power;

resizing image data accumulated in said image database and/or inquired image in size; and

producing an image feature descriptor by performing an orthogonal transform on an image obtained by said image size resizing and producing an orthogonal transform coefficient and using said orthogonal transform coefficient as a first group of image feature descriptors, wherein said image size resizing comprises a block dividing process of partitioning said image data into blocks, a dominant color calculating process of calculating a dominant color of each of blocks obtained by said block dividing process and an image creating process of creating an image using said dominant color of each of said blocks as a pixel.

17. The image retrieval method according to claim 16, wherein said image creating process is to extract a color average of entire pixels contained in each of said blocks as said calculated dominant color of each of said blocks.

18. The image retrieval method according to claim 16, wherein said block dividing process is to partition said image data into 64 blocks.

19. An image retrieving method for retrieving an image being similar to an inquired image from images stored in an image database, said method comprising:

transforming a first group of image feature descriptors extracted from image data accumulated in said image database and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said image data accumulated in said image database;

transforming a first group of image feature descriptors extracted from image data of said inquired image and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said inquired image;

resizing image data accumulated in said image database and/or inquired image in size, wherein image size resizing comprises a block dividing process of partitioning said image data into blocks, a dominant color calculating process of calculating a dominant color of each of blocks obtained by said block dividing process and an image creating process of creating an image using said dominant color of each of said blocks as a pixel;

producing an image feature descriptor by performing a discrete cosine transform on an image obtained by said image size resizing, and using the produced discrete cosine transform coefficient as an extracted first group of image feature descriptors; and

comparing said second group of image feature descriptors of image data accumulated in said image database with said second group of image feature descriptors of image data of said inquired image to calculate similarity, wherein said specified coefficient transform processing is performed in a manner that fine quantization is carried out on said transform coefficients having a small power and coarse quantization is carried out on said transform coefficients having a large power.

20. A storage medium storing a similar-image retrieval program to cause a computer to carry out retrieval of an image being similar to an inquired image from images stored in an image database, wherein said similar-image retrieval program causes the computer to execute the steps of:

transforming a first group of image feature descriptors extracted from image data accumulated in said image database and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said image data accumulated in said image database,

transforming a first group of image feature descriptors extracted from image data of said inquired image and then generating a second group of image feature descriptors to be used for calculating similarity, wherein said second group of image feature descriptors comprise transform coefficients obtained by performing specified coefficient transform processing on said inquired image;

comparing said second group of image feature descriptors of image data accumulated in said image database with said second group of image feature descriptors of image data of said inquired image to calculate similarity, wherein said specified transform processing is performed in a manner that fine quantization is carried out on said transform coefficients having a small power and coarse quantization is carried out on said transform coefficients having a large power;

resizing image data accumulated in said image database and/or inquired image; and

producing an image feature descriptor by performing an orthogonal transform on an image obtained by said image size process and producing an orthogonal transform coefficient and using said orthogonal transform coefficient as a first group of image feature descriptors, wherein said image size resizing comprises a block dividing process of partitioning said image data into blocks, a dominant color calculating process of calculating a dominant color of each of blocks obtained by said block dividing process and an image creating process of creating an image using said dominant color of each of said blocks as a pixel.