Image search system and image search method thereof

• US 20020110276A1
• Filed: 12/05/2001
• Published: 08/15/2002
• Est. Priority Date: 12/12/2000
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Kasutani, Eiji
• Granted Patent: US 7,006,689 B2
• Time in Patent Office: Days
• US Class Current: 382/165
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

An image search system for determining a similarity of an image whose feature are represented by either one of image features amounts, a color distribution features or a frequency distribution features, to search for a similar image, including a to-be-searched image features storage unit for referring to data of an image features amount of each image to be searched, an inquiry image features input unit for receiving input of data of an image features amount of an inquiry image, a data processing unit for converting a kind of an image features amount of an image set to be an object whose kind of image features amount is to be changed to make a kind of image features amount of each image to be searched and a kind of image features amount of the inquiry image be coincident with each other, and a similarity calculation unit for comparing the image features amount of the inquiry image and the image features amount of each image to be searched based on the converted image features amount and determining a similarity of each image to search for a similar image.

Claims

1. An image search system for determining a similarity of an image whose feature are represented by either one of image features amounts, a color distribution features or a frequency distribution features, to search for a similar image, comprising:

means for converting, with respect to an image set to be a target whose kind of image features amount is to be changed among respective images to be searched and an inquiry image, a kind of image features amount of the image in question to make kinds of image features amounts of each said image to be searched and said inquiry image coincident with each other; and

means for comparing the image features amount of said inquiry image with the image features amount of each said image to be searched based on said converted image features amount and determining a similarity of each image to search for a similar image.

2. The image search system as set forth in claim 1, further comprising

means for referring to data of the image features amount of each said image to be searched, and

means for receiving input of data of the image features amount of said inquiry image.

3. The image search system as set forth in claim 1, further comprising

frequency distribution features conversion means for converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

color distribution similarity calculation means for comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion means renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features.

4. The image search system as set forth in claim 1, further comprising

means for referring to data of the image features amount of each said image to be searched,

means for receiving input of data of the image features amount of said inquiry image,

frequency distribution features conversion means for converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

color distribution similarity calculation means for comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion means renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features.

5. The image search system as set forth in claim 1, further comprising

frequency distribution features conversion means for converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

color distribution similarity calculation means for comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion means renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features, and

said frequency distribution features conversion means including

inverse-frequency transformation means for decoding an applied frequency distribution features to generate a decoded image, and

color distribution features extraction means for extracting each pixel value of said decoded image as a color constituent features to extract a color distribution features indicative of feature similar to image features represented by said applied frequency distribution features.

6. The image search system as set forth in claim 1, further comprising

means for referring to data of the image features amount of each said image to be searched,

means for receiving input of data of the image features amount of said inquiry image,

frequency distribution features conversion means for converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

color distribution similarity calculation means for comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion means renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features, and

said frequency distribution features conversion means including

inverse-frequency transformation means for decoding an applied frequency distribution features to generate a decoded image, and

color distribution features extraction means for extracting each pixel value of said decoded image as a color constituent features to extract a color distribution features indicative of feature similar to image features represented by said applied frequency distribution features.

7. The image search system as set forth in claim 1, further comprising

frequency distribution features conversion means for converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

color distribution similarity calculation means for comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion means renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features, and said frequency distribution features conversion means including

inverse-frequency transformation means for decoding an applied frequency distribution features to generate a decoded image,

image division means for dividing said decoded image into a predetermined plurality of blocks, and

color distribution features extraction means for calculating each color constituent features of each said block to extract a color distribution features indicative of feature similar to image features represented by said applied frequency distribution features.

8. The image search system as set forth in claim 1, further comprising

means for referring to data of the image features amount of each said image to be searched,

means for receiving input of data of the image features amount of said inquiry image,

frequency distribution features conversion means for converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

color distribution similarity calculation means for comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion means renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features, and

said frequency distribution features conversion means including

inverse-frequency transformation means for decoding an applied frequency distribution features to generate a decoded image,

image division means for dividing said decoded image into a predetermined plurality of blocks, and

color distribution features extraction means for calculating each color constituent features of each said block to extract a color distribution features indicative of feature similar to image features represented by said applied frequency distribution features.

9. The image search system as set forth in claim 1, further comprising

frequency distribution features conversion means for converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

color distribution similarity calculation means for comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion means renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features, and

said frequency distribution features conversion means including

inverse-frequency transformation means for decoding an applied frequency distribution features to generate a decoded image,

image division means for dividing said decoded image into a predetermined plurality of blocks, and

color distribution features extraction means for calculating each color constituent features of each said block to extract a color distribution features indicative of feature similar to image features represented by said applied frequency distribution features, and

said color distribution features extraction means

determines a representative color of each said block obtained by the division by said image division means to extract a set of said representative colors as a color distribution features.

10. The image search system as set forth in claim 9, wherein

said color distribution features extraction means calculates

a color mean of a pixel in each said block obtained by the division by said image division means to determine a color of said calculated color mean as said representative color.

11. The image search system as set forth in claim 1, further comprising

color distribution features conversion means for converting a color distribution features into a frequency distribution features indicative of feature similar to image features represented by the color distribution features in question, and

frequency distribution similarity calculation means for comparing the frequency distribution features of said inquiry image with the frequency distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said color distribution features conversion means renders all the kinds of image features amounts of each image to be searched and the inquiry image into the frequency distribution features.

12. The image search system as set forth in claim 1, further comprising

means for referring to data of the image features amount of each said image to be searched,

means for receiving input of data of the image features amount of said inquiry image,

color distribution features conversion means for converting a color distribution features into a frequency distribution features indicative of feature similar to image features represented by the color distribution features in question, and

frequency distribution similarity calculation means for comparing the frequency distribution features of said inquiry image with the frequency distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said color distribution features conversion means renders all the kinds of image features amounts of each image to be searched and the inquiry image into the frequency distribution features.

13. The image search system as set forth in claim 11, wherein

said color distribution features conversion means comprising

representative color determination means for determining a representative color of each bock in an applied color distribution features,

image generation means for generating an image which uses the representative color of each said block as a pixel,

image size change means for changing the size of the image generated by said image generation means to a predetermined size, and

frequency distribution features extraction means for frequency-converting the image changed by said image size change means to extract a frequency distribution features indicative of feature similar to the image features represented by said applied color distribution features.

14. The image search system as set forth in claim 1, wherein

each said image to be searched is set to be a target whose kind of said image features amount is to be converted, and

the kind of image features amount of each said image to be searched is converted to be coincident with the kind of image features amount of said inquiry image.

15. The image search system as set forth in claim 1, wherein

said inquiry image is set to be a target whose kind of said image features amount is to be converted, and

the kind of image features amount of said inquiry image is converted to be coincident with the kind of image features amount of each said image to be searched.

16. The image search system as set forth in claim 1, wherein

both the images, said inquiry image and said image to be searched, are set to be a target whose kind of said image features amount is to be converted, and

the kinds of image features of the respective images, said inquiry image and said each image to be searched, are converted.

17. The image search system as set forth in claim 16, wherein a circuit for converting a kind of image features amount of said inquiry image and a circuit for converting a kind of image features amount of each said image to be searched are provided independently.

18. The image search system as set forth in claim 1, wherein

said color distribution features is set to be an image features amount obtained by dividing an image as an object representing feature into a predetermined plurality of blocks and determining a representative color of each said block to generate data of said representative color corresponding to each said block.

19. The image search system as set forth in claim 1, wherein

said frequency distribution features is set to be an image features amount generated by converting an image as an object representing feature into a reduced image of a fixed size and subjecting said reduced image to frequency transformation.

20. The image search system as set forth in claim 1, wherein

said frequency distribution features is set to be an image features amount generated by converting an image as an object representing feature into a reduced image of a fixed size and subjecting said reduced image to frequency transformation and quantization.

21. An image search method of determining a similarity of an image whose feature are represented by either one of image features amounts, a color distribution features or a frequency distribution features, to search for a similar image, comprising the steps of:

with respect to an image set to be a target whose kind of image features amount is to be converted among respective images to be searched and an inquiry image, converting the kind of image features amount of the image in question to make kinds of image features amounts of each said image to be searched and said inquiry image coincident with each other; and

comparing the image features amount of said inquiry image with the image features amount of each said image to be searched based on said converted image features amount and determining a similarity of each image to search for a similar image.

22. The image search method as set forth in claim 21, further comprising the steps of

referring to data of the image features amount of each said image to be searched, and

receiving input of data of the image features amount of said inquiry image.

23. The image search method as set forth in claim 21, further comprising

a frequency distribution features conversion step of converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

a color distribution similarity calculation step of comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion step renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features.

24. The image search method as set forth in claim 21, further comprising

a step of referring to data of the image features amount of each said image to be searched,

a step of receiving input of data of the image features amount of said inquiry image,

a frequency distribution features conversion step of converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

a color distribution similarity calculation step of comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion step renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features.

25. The image search method as set forth in claim 21, further comprising

a frequency distribution features conversion step of converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

a color distribution similarity calculation step of comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion step renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features, and

said frequency distribution features conversion step including

an inverse-frequency transformation step of decoding an applied frequency distribution features to generate a decoded image, and

a color distribution features extraction step of extracting each pixel value of said decoded image as a color constituent features to extract a color distribution features indicative of feature similar to image features represented by said applied frequency distribution features.

26. The image search method as set forth in claim 21, further comprising

a step of referring to data of the image features amount of each said image to be searched,

a step of receiving input of data of the image features amount of said inquiry image,

a frequency distribution features conversion step of converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

a color distribution similarity calculation step of comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion step renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features, and

said frequency distribution features conversion step including

an inverse-frequency transformation step of decoding an applied frequency distribution features to generate a decoded image, and

a color distribution features extraction step of extracting each pixel value of said decoded image as a color constituent features to extract a color distribution features indicative of feature similar to image features represented by said applied frequency distribution features.

27. The image search method as set forth in claim 21, further comprising

a frequency distribution features conversion step of converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

a color distribution similarity calculation step of comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion step renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features, and

said frequency distribution features conversion step including

an inverse-frequency transformation step of decoding an applied frequency distribution features to generate a decoded image,

an image division step of dividing said decoded image into a predetermined plurality of blocks, and

a color distribution features extraction step of calculating each color constituent features of each said block to extract a color distribution features indicative of feature similar to image features represented by said applied frequency distribution features.

28. The image search method as set forth in claim 21, further comprising

a step of referring to data of the image features amount of each said image to be searched,

a step of receiving input of data of the image features amount of said inquiry image,

a frequency distribution features conversion step of converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

a color distribution similarity calculation step of comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion step renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features, and

said frequency distribution features conversion step including

an inverse-frequency transformation step of decoding an applied frequency distribution features to generate a decoded image,

an image division step of dividing said decoded image into a predetermined plurality of blocks, and

a color distribution features extraction step of calculating each color constituent features of each said block to extract a color distribution features indicative of feature similar to image features represented by said applied frequency distribution features.

29. The image search method as set forth in claim 28, wherein

said color distribution features extraction step determines

a representative color of each said block obtained by the division by said image division step to extract a set of said representative colors as a color distribution features.

30. The image search method as set forth in claim 29, wherein

said color distribution features extraction step

calculates a color mean of a pixel in each said block obtained by the division by said image division step to determine a color of said calculated color mean as said representative color.

31. The image search method as set forth in claim 21, further comprising

a color distribution features conversion step of converting a color distribution features into a frequency distribution features indicative of feature similar to image features represented by the color distribution features in question, and

a frequency distribution similarity calculation step of comparing the frequency distribution features of said inquiry image with the frequency distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said color distribution features conversion step renders all the kinds of image features amounts of each image to be searched and the inquiry image into the frequency distribution features.

32. The image search method as set forth in claim 21, further comprising

a step of referring to data of the image features amount of each said image to be searched,

a step of receiving input of data of the image features amount of said inquiry image,

a color distribution features conversion step of converting a color distribution features into a frequency distribution features indicative of feature similar to image features represented by the color distribution features in question, and

a frequency distribution similarity calculation step of comparing the frequency distribution features of said inquiry image with the frequency distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said color distribution features conversion step renders all the kinds of image features amounts of each image to be searched and the inquiry image into the frequency distribution features.

33. The image search method as set forth in claim 31, wherein

said color distribution features conversion step comprising

a representative color determination step of determining a representative color of each bock in an applied color distribution features,

an image generation step of generating an image which uses the representative color of each said block as a pixel,

an image size change step of changing the size of the image generated by said image generation step to a predetermined size, and

a frequency distribution features extraction step of frequency-converting the image changed by said image size change step to extract a frequency distribution features indicative of feature similar to the image features represented by said applied color distribution features.

35. The image search program as set forth in claim 34, further comprising the functions of

referring to data of the image features amount of each said image to be searched, and

receiving input of data of the image features amount of said inquiry image.

36. The image search program as set forth in claim 34, further comprising

a frequency distribution features conversion function of converting a frequency distribution features into a color distribution features indicative of feature similar to image features represented by the frequency distribution features in question, and

a color distribution similarity calculation function of comparing the color distribution features of said inquiry image with the color distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said frequency distribution features conversion function renders all the kinds of image features amounts of each image to be searched and the inquiry image into the color distribution features.

37. The image search program as set forth in claim 36, wherein

said frequency distribution features conversion function including

an inverse-frequency transformation function of decoding an applied frequency distribution features to generate a decoded image, and

a color distribution features extraction function of extracting each pixel value of said decoded image as a color constituent features to extract a color distribution features indicative of feature similar to image features represented by said applied frequency distribution features.

38. The image search program as set forth in claim 36, wherein

said frequency distribution features conversion function including

an inverse-frequency transformation function of decoding an applied frequency distribution features to generate a decoded image,

an image division function of dividing said decoded image into a predetermined plurality of blocks, and

a color distribution features extraction function of calculating each color constituent features of each said block to extract a color distribution features indicative of feature similar to image features represented by said applied frequency distribution features.

39. The image search program as set forth in claim 38, wherein

said color distribution features extraction function determines

a representative color of each said block obtained by the division by said image division function to extract a set of said representative colors as a color distribution features.

40. The image search program as set forth in claim 39, wherein

said color distribution features extraction function

calculates a color mean of a pixel in each said block obtained by the division by said image division function to determine a color of said calculated color mean as said representative color.

41. The image search program as set forth in claim 34, further comprising

a color distribution features conversion function of converting a color distribution features into a frequency distribution features indicative of feature similar to image features represented by the color distribution features in question, and

a frequency distribution similarity calculation function of comparing the frequency distribution features of said inquiry image with the frequency distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said color distribution features conversion function renders all the kinds of image features amounts of each image to be searched and the inquiry image into the frequency distribution features.

42. The image search program as set forth in claim 34, further comprising

a function of referring to data of the image features amount of each said image to be searched,

a function of receiving input of data of the image features amount of said inquiry image,

a color distribution features conversion function of converting a color distribution features into a frequency distribution features indicative of feature similar to image features represented by the color distribution features in question, and

a frequency distribution similarity calculation function of comparing the frequency distribution features of said inquiry image with the frequency distribution features of each said image to be searched and determining a similarity of each image to search for a similar image, wherein

said color distribution features conversion function renders all the kinds of image features amounts of each image to be searched and the inquiry image into the frequency distribution features.

43. The image search program as set forth in claim 41, wherein

said color distribution features conversion function comprising

a representative color determination function of determining a representative color of each block in an applied color distribution features,

an image generation function of generating an image which uses the representative color of each said block as a pixel,

an image size change function of changing the size of the image generated by said image generation function to a predetermined size, and

a frequency distribution features extraction function of frequency-converting the image changed by said image size change function to extract a frequency distribution features indicative of feature similar to the image features represented by said applied color distribution features.

34. An image search program for determining a similarity of an image whose feature are represented by either one of image features amounts, a color distribution features or a frequency distribution features, to search for a similar image by controlling a computer, comprising the functions of:

with respect to an image set to be a target whose kind of image features amount is to be converted among respective images to be searched and an inquiry image, converting the kind of image features amount of the image in question to make kinds of image features amounts of each said image to be searched and said inquiry image coincident with each other; and

comparing the image features amount of said inquiry image with the image features amount of each said image to be searched based on said converted image features amount and determining a similarity of each image to search for a similar image.