Content-based multimedia searching system using color distortion data

• US 7,065,248 B2
• Filed: 05/21/2001
• Issued: 06/20/2006
• Est. Priority Date: 05/19/2000
• Status: Expired due to Fees

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Kim, Hyun Jun, Lee, Jin Soo
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): Hung, Yubin
• Application Number:
  US09/860,464
• Publication Number:
  US 20010055419A1
• Time in Patent Office: 1,856 Days
• Field of Search: 382/162, 382/169, 382190-191, 382274-275, 382/305, 382/165, 382/167, 345589-590, 345600-604, 358/403, 358/461, 358518-523
• US Class Current: 382/165
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

Methods are provided for extracting color distortion data from multimedia data in a content-based multimedia search and for searching multimedia data based on the extracted color distortion data. A method is also provided for generating multimedia data to be used in the content-based multimedia search. The color distortion data includes hues of the distorted color identifying which color affected such a distortion and intensity data representing how much color distortion data affected such a distortion.

Claims

1. A content-based multimedia searching method using color distortion data, the method comprising:

extracting color distortion data from multimedia data;

restoring the distorted color of the multimedia data to the original color by using the extracted color distortion data;

measuring a similarity between the multimedia data restored to the original color; and

aligning the multimedia data on the basis of the measured similarity.

2. The content-based multimedia searching method of claim 1, wherein the extracted color distortion data are hue data.

3. A content-based multimedia searching method using color distortion data, the method comprising:

extracting color distortion data from multimedia data;

extracting features to be used for the search;

storing the extracted features and the color distortion data;

restoring a value of the features to values corresponding to the features of the original color by using the stored color distortion data;

measuring a similarity between the multimedia data by using the restored value of the features; and

aligning the multimedia data on the basis of the measured similarity.

4. The content-based multimedia searching method of claim 3, wherein the extracted color distortion data are hue data.

5. The content-based multimedia search method of claim 3, wherein the extracted color distortion data further include intensity data of the distorting color.

6. The content-based multimedia searching method of claim 3, wherein restoring the original color comprises:

converting a distorting color value to an R, G, B value satisfying the hue value of the color distortion data, assuming that max (RGB) and min (RGB) values are fixed to predetermined constants; and

calculating a relative variation of the distorting color value by multiplying the R, G, B value of the hue value by predetermined constants, and subtracting the distorting color value from original R, G, B value for each pixel.

7. A content-based multimedia searching method using color distortion data, the method comprising:

extracting a hue value of a distorting color as color distortion data from multimedia data,

dividing an image into N*M number of local areas and extracting representative color values of each area as features to be used for the search;

storing the extracted features and the color distortion data;

restoring the representative color values of each area to a value corresponding to the original color by using the hue value of the stored distorting color;

measuring a similarity between the multimedia data by using the restored value of the features; and

aligning the multimedia data on the basis of the measured similarity.

8. The content-based multimedia searching method of claim 7, wherein the extracted color distortion data further include intensity data of the distorting color.

9. The content-based multimedia searching method of claim 7, wherein restoring the original color comprises:

converting a distorting color value to an R, G, B value satisfying the hue value of the color distortion data, assuming that max (RGB) and min (RGB) values are fixed to predetermined constants; and

calculating a relative variation of the distorting color value by multiplying the R, G, B value of the hue value by predetermined constants, and subtracting the distorting color value from original R, G, B value for each pixel.

10. The content-based multimedia searching method of claim 9, wherein the extracted color distortion data further include intensity data, and the predetermined constants are intensity values or values proportional to the intensity values.

11. The content-based multimedia searching method of claim 9, wherein the max(RGB) value is set as 255 and the min(RGB) value is set as 0.

12. A method for extracting color distortion data, comprising:

calculating an average hue and a variation of the hue values for all pixels having a brightness value and a chromatic value, which are the same as or less than a predetermined critical value;

calculating an average hue and a variation of the hue values for all pixels having a brightness value the same as or greater than a predetermined critical value and a chromatic value the same as or less than a predetermined critical value; and

designating a re-averaged value of the two average hue values as a hue of the distorting color, if both of the two variation values calculated as above are the same as or less than a predetermined critical value and a difference between the two average hue values is the same as or less than a predetermined critical value.

13. The method of claim 12, further comprising, if only one of the two variation values is the same as or less than the predetermined critical value:

generating a hue histogram for all pixels of an image;

extracting a hue having a maximum variation value in the generated hue histogram; and

designating an average value between the extracted hue value and the average hue value having a variation the same as or less than the predetermined critical value as a hue of the distorting color, if the variation of the extracted hue is the same as or greater than a predetermined critical value.

14. A method for extracting color distortion data, comprising:

dividing an image into N*M number of areas;

calculating an average hue and a variation of the hue value for all pixels having a brightness value and a chromatic value the same as or less than a predetermined critical value for each divided area, and register the average hue as a candidate of distorting color for a dark color only when the variation is the same as or less than a predetermined critical value;

calculating an average hue and a variation of the hue value for all pixels having a brightness value the same as or greater than a predetermined critical value and a chromatic value the same as or less than a predetermined critical value for each divided area, and register the average hue as a candidate of distorting color for a bright color only when the variation is the same as or less than a predetermined critical value;

calculating an average of the hues registered as the candidate of distorting color for a dark color and an average of the hues registered as the candidate of distorting color for a bright color, and designating a re-averaged hue value of the two average hue values as a distorting hue if a difference between the two average hue values is the same as or less than a predetermined critical value;

calculating a ratio of the number of the registered candidate hues to N*M number of object areas divided from the entire image; and

designating the ratio as an intensity of the color distortion.

15. The method of claim 14, further comprising, if there are registered hues in only one of the candidate of distorting bright color or the candidate of distorting dark color:

designating an average hue value of the registered hues as a distorting hue only when the number of registered hues are the same as or greater than a predetermined critical value;

calculating a ratio of the number of registered hues to N*M number of object areas divided from the entire image; and

designating the ratio as an intensity of the color distortion.

16. A method for generating multimedia data in order to be used in a content-based multimedia search, comprising:

incorporating feature information; and

incorporating color distortion data, wherein the color distortion data comprises hues of a distorting color identifying which color affected such a color distortion and intensity data representing how much the color distortion data affected such a color distortion.

17. A content-based multimedia searching system, comprising:

a feature extracting unit including a feature extracting module configured to extract necessary features for the search, a color distortion data extracting module configured to extract color distortion data, and a storing module configured to store extracted features and color distortion data; and

a search unit including a color restoring module configured to receive stored features and color distortion data and restore distorted color value into original color value using the color distortion data, a similarity calculation module configured to calculate a similarity between object data, and a data alignment module configured to display the object data based on the calculated similarity.

18. The content-based multimedia searching system of claim 17, wherein the color distortion data are hues of a distorting color.

19. The content-based multimedia searching system of claim 17, wherein the color distortion data further include intensity of the distorting color.

20. The content-based multimedia searching system of claim 17, further comprising a user query interface configured to input a user inquiries and a display interface configured to display search results.