Method and apparatus for representing and searching for color images

• US 6,801,657 B1
• Filed: 04/25/2000
• Issued: 10/05/2004
• Est. Priority Date: 04/29/1999
• Status: Expired due to Term

• Current Assignee: Rakuten Group, Inc.
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Cieplinski, Leszek
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): Kassa, Yosef
• Application Number:
  US09/556,799
• Time in Patent Office: 1,624 Days
• Field of Search: 382/164, 382/165, 382/166, 382/167, 382/170, 382/171, 382/190, 382/191, 382/194, 382/195, 382/201, 382/209, 382/217, 382/218, 382/219, 382/220, 382/224, 382/278, 382/282, 382/287, 382/305, 358/512, 358/515, 358/518, 358/519, 358/527, 707/2, 707/3, 707/4, 707/5, 707/6
• US Class Current: 382/164
• CPC Class Codes: G01J 3/463 Colour matching
• CPC Class Codes: G01J 3/50 using electric radiation detectors
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5862 using texture
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A method of representing a colour image comprises selecting a region of the image, selecting one or more colours as representative colours for the region and, for a region having two or more representative colours, calculating for each representative colour at least two parameters related to the colour distribution in relation to the respective representative colour and using said parameters to derive descriptors for the image region.

Claims

1. A method of representing a color image, comprising:

selecting two or more colors as representative colors for the image or a region of said image;

calculating for each representative color a parameter related to the color variance in relation to the respective representative color for said image or the image region having two or more representative colors;

calculating for each representative color a weight parameter representing the relative significance of the representative color in the image or image region, based on a ratio of the number of pixels corresponding to the representative color relative to the number of pixels in the image or image region; and

using said parameters to derive a descriptor for said image or image region, said descriptor including the parameter related to the corresponding color variance and the weight parameters.

2. A method as claimed in claim 1 comprising storing said descriptor in data storage means.

3. A method as claimed in claim 1 wherein the step of selecting representative colors comprises deriving a color histogram for the image or image region.

4. A method as claimed in claim 3 wherein the step of selecting representative colors comprises identifying local peaks in the color histogram and selecting the corresponding colors as representative colors.

5. A method as claimed in claim 4 wherein the local peaks are treated as mean values and the variances are calculated with respect to said mean values.

6. A method as claimed in claim 1 wherein the image or the image region is independent of the image content.

7. A method as claimed in claim 6 wherein the image or the image region is a polygon.

8. A method as claimed in claim 1 wherein the image or the image region corresponds to an object.

9. A method for searching for color images in data storage means comprising:

inputting a query relating to color of an image,

comparing said query with descriptors for stored images derived in accordance with a method as claimed in claim 1 using a matching function, and

selecting and displaying at least one image for which the matching function indicates a close match between the query and at least part of the image.

10. A method as claimed in claim 9 wherein inputting a query comprises selecting a query image or a region of said image and obtaining the derived descriptors for said image or image region and wherein the matching function uses the descriptors for the query and for the stored images.

11. A method as claimed in claim 9 wherein the matching function is based on M=exp[(−

½

)(q−

m)^TC<sup>−

1</sup>(q−

m)]

where q is a color vector corresponding to a query and m and C are descriptor values representing first and second central moments of the color distribution for a representative color.

12. A method as claimed in claim 9 wherein the matching function is based on

13. A method as claimed in claim 9 wherein a query is selected from a plurality of images displayed on display means.

14. A method as claimed in claim 9 wherein inputting a query comprises selecting a single color value.

15. A method as claimed in claim 9 wherein inputting a query comprises specifying one or more component distributions.

16. A method as claimed in claim 9 wherein a query is input using only some of the components of the color space.

17. An apparatus for implementing a method according to claim 1.

18. A computer system programmed to operate according to a method as claimed in claim 1.

19. A computer program for implementing a method as claimed in claim 1.

20. A computer-readable medium storing computer-executable process steps for implementing a method as claimed in claim 1.

21. The method of claim 1, further comprising:

selecting the image or image region.

22. The method of claim 1 wherein each weight parameter specifies percentage of pixels in the image or image region that have associated representative color.

23. A method of representing a color image by processing signals corresponding to said image, the method comprising:

identifying a number of representative colors for said image or a region of said image,

deriving a function approximating the color variance corresponding to each representative color for said image or the image region having two or more representative colors,

calculating for each representative color a weight parameter representing the significance of the representative color in the image or image region, based on a ratio of the number of pixels associated with the representative color relative to the number of pixels in the image or image region, and

using said functions and said weight parameters to define a color description of said image or image region.

24. A method for representing an image, comprising:

determining at least two dominant colors for an image or a region of said image; and

determining a color variance and a weight parameter representing the significance of the dominant color in the image or image region based on the number of pixels corresponding to the dominant color in the image or image region for each dominant color to generate a color descriptor based on pixels within said image or the image region, said color descriptor including at least two parameters related to the corresponding color variances and at least two weight parameters.

25. The method of claim 24, further comprising:

storing said color descriptor as a description for said object in a memory.

26. The method of claim 24, further comprising:

selecting the image or image region.

27. A method for representing an image, comprising:

determining at least two dominant colors for an image or a region of said image; and

determining a color distribution with respect to each dominant color, using factors for each dominant color including a weight factor based on the ratio of the number of pixels corresponding to the dominant color to the total number of pixels in said image or image region, a dominant color factor, and a color variance factor, to generate a color descriptor for said image or image region.

28. The method of claim 27, further comprising:

storing said color descriptor as a description for said image or image region in a memory.

29. A method for searching for an image, comprising:

receiving an input including at least one image;

determining at least two dominant colors for the input image or a region of said input image;

determining at least two color variances with respect to the corresponding dominant color and weight parameters representing the significance of each dominant color within the image or image region based on a ratio of the number of pixels corresponding to the dominant color relative to the number of pixels in the image or image region to generate a color descriptor based on pixels within said image or the image region;

comparing said color descriptor for said input image to a color descriptor for an image stored in memory to find at least one substantially matching image; and

displaying said at least one substantially matching image.

30. A method for propagating a signal, comprising:

generating a signal including a plurality of images;

encoding said images onto said signal using a description language to define at least a color descriptor generated using a predetermined algorithm, including a plurality of fields, for at least one selected image wherein said fields include fields representing the color space description, color variance description, and dominant color weight description for said image.

31. A system for representing an image, comprising:

a detector for determining at least two dominant colors for an image or a region of said image;

a controller for determining a color variance and a weight parameter representing the significance of the dominant color relative to other colors within the image or image region, based on a ratio of the number of pixels corresponding to the dominant color relative to the number of pixels in the image or image region for each dominant color to generate a color descriptor based on pixels within said image or the image region; and

a memory for storing said color descriptor as a description for said image or image region in a memory.

32. A system for representing an image, comprising:

a detector for determining at least two dominant colors for an image or a region of said image;

a controller for determining a color distribution with respect to each dominant color, using factors for each dominant color including a weight factor based on the ratio of the number of pixels corresponding to the dominant color to the total number of pixels in said image or image region, a dominant color factor, and a color variance factor, to generate a color descriptor for said image or image region; and

a memory for storing said color descriptor as a description for said image or image region in a memory.

33. A system for searching for an image, comprising:

an input device for inputting at least one image;

a detector for determining at least two dominant colors for said input image or a region of said input image;

a controller for determining a color variance with respect to each dominant color and weight parameters representing the significance of each dominant color relative to other colors in the image or image region based on a ratio of the number of pixels corresponding to the dominant color relative to the number of pixels in the image or image region for generating a color descriptor based on pixels within said image or image region;

a comparator for comparing said color descriptor for said input image to a color descriptor for an image stored in memory to find at least one substantially matching image; and

a display for displaying said at least one substantially matching image.

34. A machine-readable medium having stored thereon a plurality of executable instructions, the plurality of instructions comprising instructions to:

determine at least two dominant colors for an image or a region of said image;

determine a color variance with respect to each dominant color and weight parameters representing the significance of each dominant color relative to other colors in the image or image region based on a ratio of the number of pixels corresponding to the dominant color relative to the number of pixels in the image or image region to generate a color descriptor based on pixels within said image or the image region; and

store said color descriptor as a description for said image or image region in a memory.

35. A machine-readable medium having stored thereon a plurality of executable instructions, the plurality of instructions comprising instructions to:

determine at least two dominant colors for an image or a region of said image;

determine a color distribution with respect to each dominant color, using factors for each dominant color including a weight factor based on the ratio of the number of pixels corresponding to the dominant color to the total number of pixels in said image or image region, a dominant color factor, and a color variance factor, to generate a color descriptor for said image or image region; and

store said color descriptor as a description for said image or image region in a memory.

36. A machine-readable medium having stored thereon a plurality of executable instructions, the plurality of instructions comprising instructions to:

receive an input including at least one image;

determine at least two dominant colors for said input image or a region of said input image; and

determine a color variance with respect to each dominant color and weight parameters representing the significance of each dominant color relative to other colors in the image or image region based on a ratio of the number of pixels corresponding to the dominant color relative to the number of pixels in the image or image region to generate a color descriptor based on pixels within said image or the image region;

compare said color descriptor for said input image to a color descriptor for an image stored in memory to find at least one substantially matching image; and

display said at least one substantially matching image.

37. A method of representing a color image, comprising:

selecting two or more colors as dominant colors for an image;

for each dominant color, calculating the variance of the color distribution of the image in color space relative to the dominant color and weight parameters representing the significance of each dominant color relative to other colors in the image or image region based on a ratio of the number of pixels corresponding to the dominant color relative to the number of pixels in the image or image region; and

representing the image in terms of the color space value of each dominant color and the value of the respective color variance.

38. A method of representing a color image, comprising:

deriving a color histogram for an image region to determine two or more dominant colors;

calculating a parameter relating to variance of the two or more representative colors within the image region to represent the image;

calculating for each representative color a weight parameter representing the relative significance of the representative color in the image or image region, based on a ratio of the number of pixels corresponding to the representative color relative to the number of pixels in the image or image region; and

using said parameters and histogram to derive a descriptor for said image or image region, said descriptor including the parameter related to the corresponding color variance and the weight parameters.