Image retrieval method and apparatus independent of illumination change

• US 20030126125A1
• Filed: 11/27/2002
• Published: 07/03/2003
• Est. Priority Date: 11/27/2001
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Chang-yeong, Park, Du-sik, Choi, Yang-lim, Kim, Sang-kyun
• Granted Patent: US 7,130,845 B2
• Time in Patent Office: Days
• US Class Current: 707/3
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: Y10S 707/915 Image
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

An image retrieval method and apparatus independent of an illumination change are provided. The image retrieval method involves: inputting a query image; detecting an illumination color from the query image and converting the illumination color into a standard illumination color; extracting color information of the query image by using color descriptors; and retrieving a similar image by comparing the extracted color information with color information of a database which converts a variety of images into images of standard illumination colors and extracts and stores the color information of the images in advance. According to the method, without changing the structures of color descriptors or a similarity comparison using color information, an image retrieval independent of illumination changes is enabled by adding only a preceding step of standard illumination conversion.

Claims

1. An image retrieval method independent of illumination changes comprising:

inputting a query image;

detecting an illumination color from the query image and converting the illumination color into a standard illumination color;

extracting color information of the query image by using color descriptors; and

retrieving a similar image by comparing the extracted color information with color information of a database which converts a variety of images into images of standard illumination colors and extracts and stores the color information of the images in advance.

2. The method of claim 1, wherein the converting step comprises:

removing low luminance parts and self luminance parts from the input query image;

detecting an illumination color of the query image from the remaining pixels after the removal of the low luminance parts and self luminance parts; and

converting the detected illumination color into a standard illumination color.

3. The method of claim 2, wherein the removing step comprises:

converting a red, green, and blue (RGB) value of each pixel in the input query image into a tri-stimulus value XYZ;

removing pixels having low luminance values from the pixels in the image, by using the converted tri-stimulus value XYZ; and

removing self luminance pixels from the pixels in the image, by using the converted tri-stimulus value XYZ.

4. The method of claim 3, wherein if the Y value of the tri-stimulus value of a pixel is included in the lowest 5% of the Y values of all of the pixels, it is determined that the pixel has a low luminance value.

5. The method of claim 3, wherein it is determined that pixels having values greater than a predetermined self luminance threshold are self luminance pixels, and then the self luminance pixels are removed.

6. The method of claim 2, wherein the detecting step comprises:

converting a tri-stimulus value XYZ of each pixel of the query image from which low luminance parts and self luminance parts have been removed, into chromaticity coordinates (x, y);

projecting the converted chromaticity coordinates on an x-y plane;

dividing the x-y plane uniformly into a grid of a predetermined size;

counting the number of projected pixels existing in each divided grid;

removing pixels in a grid if the counting result indicates that the counted value of the grid is less than a predetermined value; and

calculating an average value of central coordinate values of the remaining grids after the removal, and determining the average as a standard illumination color of the input image.

7. The method of claim 6, wherein the x value of the chromaticity coordinates is calculated by a formula X/(X+Y+Z) and the y value of the chromaticity coordinates is calculated by a formula Y/(X+Y+Z).

8. The method of claim 6, wherein the x-y plane is divided into a 60×

60 grid.

9. The method of claim 2, wherein the step for converting the detected illumination color into a standard illumination color comprises:

finding standard illumination chromaticity coordinate values on a daylight locus corresponding to a standard color temperature;

converting the tri-stimulus value XYZ of each pixel of the illumination color detected in the detection step, into a standard illumination tri-stimulus value X′

Y′

Z′

, by using the standard illumination chromaticity coordinate values and a Bradford color adaptation transform matrix; and

converting the converted standard illumination tri-stimulus value X′

Y′

Z′

into an RGB value.

10. The method of claim 9, wherein the standard color temperature is 6500K.

11. The method of claim 1, wherein in the extracting step, color information of the query image is extracted by using a dominant color descriptor when a predetermined color or a small number of colors in an image represent the entire image or a part of the image.

12. The method of claim 1, wherein in the extracting step, color information of the query image is extracted by using a color structure descriptor which uses local color structure information together with a histogram, and when characteristic information is extracted, extracts color information on each pixel not independently, but in consideration of color information on neighboring pixels.

13. The method of claim 1, wherein in the extracting step, color information of the query image is extracted by using a color layout descriptor which indicates the spatial distribution of a color.

14. The method of claim 1, wherein in the extracting step, color information of the query image is extracted by using a scalable color descriptor which shows a color histogram encoded by a Harr transform and uses a hue saturation value (HSV) color space.

15. The method of claim 1, wherein in the retrieval step, retrieval results are output in order of increasing hamming distance by using the results of calculating hamming distances between the extracted color descriptors.

25. A computer readable medium having embodied therein a computer program for performing the method of claim 1.

16. An image retrieval apparatus independent of illumination changes comprising:

a query image input unit which receives a query image;

a standard illumination color converting unit which detects an illumination color from the query image and converts the illumination color into a standard illumination color;

a query image color information extracting unit which extracts color information of the converted query image by using color descriptors;

a database which converts a variety of images desired to be retrieved into images of standard illumination colors and extracts and stores the color information of the images in advance; and

a similar image retrieval unit which finds a similar image by comparing the color information of the query image that is converted into an image of the standard illumination color, with color information on the variety of images stored in the database.

17. The apparatus of claim 16, wherein the standard illumination color converting unit comprises:

a noise removing unit which removes low luminance parts and self luminance parts from the input query image;

an illumination color detection unit which detects an illumination color of the image from the remaining pixels after the noise removing unit has removed noise; and

a converting unit which converts the detected illumination color into a standard illumination color.

18. The apparatus of claim 17, wherein the noise removing unit converts a red, green, and blue (RGB) value of each pixel in the input query image into a tri-stimulus value XYZ, and removes pixels having low luminance values and self luminance pixels from the pixels in the image, by using the converted tri-stimulus value XYZ.

19. The apparatus of claim 17, wherein the illumination color detection unit converts a tri-stimulus value XYZ of each pixel of the query image from which low luminance parts and self luminance parts have been removed, into chromaticity coordinates (x, y);

projects the converted chromaticity coordinates on an x-y plane;

divides the x-y plane uniformly into a grid of a predetermined size;

counts the number of projected pixels existing in each divided grid;

removes pixels in a grid if the counting result indicates that the counted value of the grid is less than a predetermined value; and

calculates an average of central points of the remaining grids after the removal, and determining the average as a standard illumination color of the input image.

20. The apparatus of claim 17, wherein the converting unit finds standard illumination chromaticity coordinate values on a daylight locus corresponding to a standard color temperature;

converts the tri-stimulus value XYZ of each pixel of the illumination color detected in the detection step into a standard illumination tri-stimulus value X′

Y′

Z′

by using the standard illumination chromaticity coordinate values and a Bradford color adaptation transform matrix; and

then converts the converted standard illumination tri-stimulus value X′

Y′

Z′

into an RGB value.

21. A standard illumination color converting method comprising:

removing low luminance parts and self luminance parts from an input query image;

detecting an illumination color of the image from the remaining pixels after the removal of the low luminance parts and self luminance parts; and

converting the detected illumination color into a standard illumination color.

22. The method of claim 21, wherein the removing step comprises:

converting a red, green, and blue (RGB) value of each pixel in the input query image into a tri-stimulus value XYZ;

removing pixels having low luminance values from the pixels in the image by using the converted tri-stimulus value XYZ; and

removing self luminance pixels from the pixels in the image by using the converted tri-stimulus value XYZ.

23. The method of claim 21, wherein the detecting step comprises:

converting a tri-stimulus value XYZ of each pixel of the query image from which low luminance parts and self luminance parts have been removed, into chromaticity coordinates (x, y);

projecting the converted chromaticity coordinates on an x-y plane;

dividing the x-y plane uniformly into a grid of a predetermined size;

counting the number of projected pixels existing in each divided grid;

removing pixels in a grid if the counting result indicates that the counted value of the grid is less than a predetermined value; and

calculating an average value of central coordinates of the remaining grids after the removal, and determining the average as a standard illumination color of the input image.

24. The method of claim 21, wherein the converting step comprises:

finding standard illumination chromaticity coordinate values on a daylight locus corresponding to a standard color temperature;

converting the tri-stimulus value XYZ of each pixel of the illumination color detected in the detection step into a standard illumination tri-stimulus value X′

Y′

Z′

by using the standard illumination chromaticity coordinate values and a Bradford color adaptation transform matrix; and

converting the converted standard illumination tri-stimulus value X′

Y′

Z′

into an RGB value.

26. A computer readable medium having embodied therein a computer program for performing the method of claim 21.