Content-based multimedia retrieval method

• US 6,782,125 B2
• Filed: 03/12/2001
• Issued: 08/24/2004
• Est. Priority Date: 03/13/2000
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Lee, Jin Soo, Kim, Hyeon Jun
• Primary Examiner(s): Tran, Phuoc
• Application Number:
  US09/802,979
• Publication Number:
  US 20010028739A1
• Time in Patent Office: 1,261 Days
• Field of Search: 382/162, 382/164, 382/165, 382/168, 382/170, 382/305, 707/104.1
• US Class Current: 382/162
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

Disclosed is a content-based multimedia data retrieval method. The method according to the present invention comprises the steps of extracting color distortion information from multimedia data, setting weight of color-based descriptors on the basis of the extracted color distortion information, and extracting a distortion sensitivity by differently applying the weight according to the color distortion information. The method according to the present invention can improve a multimedia data retrieval performance and solve a problem of color distortion due to a different illumination or characteristics of a device.

Claims

1. A content-based multimedia retrieval method, comprising the steps of:

extracting color distortion information from multimedia data;

setting weight of a descriptor used in the content-based multimedia retrieval using the extracted color distortion information; and

retrieving multimedia data applying the weight.

2. The method as claimed in claim 1, wherein the step of setting weight includes a further step of setting weight of elements of the descriptor.

3. The method as claimed in claim 1, wherein the descriptors are color histogram, region representative color information, texture histogram and region representative texture information.

4. The method as claimed in claim 2, wherein the elements of the descriptor are bins constituting the color histogram.

5. The method as claimed in claim 2, wherein the weight is set according to distortion sensitivity of the descriptor or the element of the descriptor.

6. The method as claimed in claim 5, wherein the weight is applied low to the descriptor or the element of the descriptor, as the distortion sensitivity is large.

7. The method as claimed in claim 6, wherein the distortion sensitivity is determined in an inverse proportion to an average similarity, wherein the average similarity is obtained by calculating and averaging similarities of data containing the color distortion information and data not containing the color distortion information based on the corresponding descriptor or element of the descriptor.

8. The method as claimed in claim 6, wherein the distortion sensitivity is in proportion to the extent of reliance on color information out of the color-based descriptors.

9. The method as claimed in claim 2, wherein, of the color-based descriptors used in the retrieval, the weight of the descriptor having relatively large distortion sensitivity is set to be lower than the weight of the other descriptors.

10. The method as claimed in claim 2, wherein the descriptor or the element of the descriptor is divided into a color-based descriptor group and a non-color-based descriptor group, wherein the weight of the color-based descriptor group is set to be lower than the non-color-based descriptor group.

11. The method as claimed in claim 10, wherein the weight is set according to the distortion sensitivity of the color-based descriptor group and the non-color-based descriptor group.

12. The method as claimed in claim 11, wherein the color-based descriptor group is the color histogram and the region representative color information, and the non-color-based descriptor group is the texture histogram and the region representative texture information.

13. The method as claimed in claim 12, wherein, of the color-based descriptor group, if the distortion sensitivity of the color histogram is larger than that of the region representative color, the weight of the color histogram is set to be relatively lower than that of the region representative color.

14. The method as claimed in claim 1, wherein the color distortion information is an intensity value of the color distortion information, or a color value of the color distortion information, or a combination of the values.

15. The method as claimed in claim 14, wherein the color value of the color distortion information is hue.

16. The method as claimed in claim 1, wherein, of the color elements used in the color-based descriptor, the weight of the color element having relatively low chroma is set to be lower than that of the color element having high chroma.

17. The method as claimed in claim 1, wherein, of the color elements used in the color-based descriptor, the weight of color region having the hue different from that of the color distortion information is set to be low.

18. A color distortion information extracting method, comprising the steps of:

dividing an image into partial regions of N×

M;

calculating, with respect to all divided partial regions, average hue values of pixels that their chroma is lower than a predetermined threshold for each partial region, and variance of hue values of pixels applied to the average hue values; and

re-averaging the average hue values obtained in each partial region with respect to the partial regions in which the calculated variance is lower than the predetermined threshold.

19. The method as claimed in claim 18, wherein a ratio that the regions applied to the re-average of the hue value and of lower than the predetermined threshold occupy in whole image is designated as an intensity of the color distortion information.

20. The method as claimed in claim 18, wherein the re-averaged average hue values are designated as the hue value of the color distortion information.

21. A distortion sensitivity extracting method, comprising the steps of:

calculating a similarity based on a descriptor or an element of the descriptor with respect to the same image data in a data sample group that a color distortion does not occur and in a data sample group that the color distortion occurs;

obtaining an average value of the calculated similarities; and

determining a distortion sensitivity in inverse proportion to the average similarity.

22. In a content-based multimedia data retrieval using a descriptor such as a color histogram of image, a multimedia data information generating method, comprising the steps of:

generating a descriptor group having one or more descriptors used in the multimedia data retrieval; and

extracting color distortion information from the multimedia data, automatically setting weight of the descriptor group according to the extracted color distortion information, and generating color distortion information for retrieving the multimedia data.

23. The method as claimed in claim 22, wherein the color distortion information consists of an intensity of the color distortion information, hue information of the color distortion information, or the combination of the intensity and hue information.

24. The method as claimed in claim 22, wherein the color distortion information describes what color has affected the color distortion.

25. The method as claimed in claim 23, wherein the intensity of the color distortion information describes how much the distorted color has affected.