Method of searching or browsing multimedia data and data structure

• US 6,643,643 B1
• Filed: 01/28/2000
• Issued: 11/04/2003
• Est. Priority Date: 01/29/1999
• Status: Expired due to Term

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Lee, Jin Soo, Kim, Hyeon Jun
• Primary Examiner(s): Alam, Hosain T.
• Assistant Examiner(s): Liang, Gwen
• Application Number:
  US09/493,038
• Time in Patent Office: 1,376 Days
• Field of Search: 707/5, 707/104.1, 707/6, 707/522, 707/2, 707/3, 345/700, 345/156, 345/625, 382/219, 382/170, 382/141, 382/305, 382/217, 385/47, 701/50, 235/94.A, 220/15, 204/232, 131/84.4, 725/46
• US Class Current: 1/1
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99939 Privileged access

Abstract

A method of searching or browsing multimedia data which can effectively search or browse an image is disclosed. In the present method, the searching or browsing is performed in consideration of at least one of a reliability information on a weight of a feature of an input multimedia data and an authority code. According to the method, the system selects reference multimedia data, and searches the multimedia data using a weight of a feature and/or a feature element of the input multimedia data. A user gives feedback on the relevance of the searched or browsed multimedia data, and the system calculates a new weight using the relevance information. Thus, the system updates and maintains new weights of features and/or feature elements.

Claims

1. A method of searching or browsing multimedia data comprising:

(a) receiving a reference multimedia data with a data structure including features of said reference multimedia data and weight information of said features, wherein the data structure of the reference multimedia data further comprises authority information which limits an update of the weight information by a user feedback;

(b) searching for said reference multimedia data using the features and the weight information;

(c) receiving user feedback on a relevance of resultant multimedia data found in (b);

(d) measuring a similarity of the reference multimedia data to the resultant multimedia data and calculating a new weight information of said features using the measured value; and

(e) updating the weight information of said features in said data structure of the reference multimedia data using the new weight information.

2. A method of claim 1, wherein the authority information includes a plurality of authority levels, wherein each authority level has degree values affecting the degree of weight information update in (e).

3. A method of claim 2, wherein receiving a password from a user to determine an authority level of the user.

4. A method of searching or browsing multimedia data comprising:

(a) receiving a reference multimedia data with a data structure including features of said reference multimedia data and weight information of said features, wherein the data structure of the reference multimedia data further comprises reliability information indicating a reliability of the weight information, wherein the reliability of the weight information obtained based on authority of user feedback;

(b) searching for said reference multimedia data using the features and the weight information;

(c) receiving user feedback on a relevance of resultant multimedia data found in (b);

(d) measuring a similarity of the reference multimedia data to the resultant multimedia data and calculating a new weight information of said features using the measured value;

a measured value and

(e) updating the weight information of said features in said data structure of the reference multimedia data using the new weight information.

5. A method of claim 4, wherein a reliability of a weight assigned to a feature is proportional to the amount of training by user feedback.

6. A method of claim 4, wherein the data structure of the reference multimedia data further comprises authority information which limits an update of the weight information by a user feedback.

7. A method of claim 6, wherein the authority information includes a plurality of authority levels, wherein each authority level has degree values affecting the degree of weight information update in (e).

8. A method of claim 7, wherein a higher reliability is given to user feedback by a user with a high authority level.

9. A method of claim 6, wherein in (e), updating the weight information of said features in said data structure of the reference multimedia data depending upon the reliability information and the authority information.

10. The method of claim 4, wherein the reliability information is the number of weight updates made by a user.

11. A method of scarching of browsing multimedia data comprising:

searching for a reference multimedia data using variable information representing an importance of a feature of the reference multimedia data;

receiving user feedback on relevance of the searched multimedia data;

calculating a new variable information using user pattern information stored in a system;

updating the variable information using the calculated new variable information and/or reliability information; and

maintaining the updated variable information, wherein the user pattern information stored in a system is a number of weight updates made by a user.

12. A method of claim 11, wherein the variable information is one or a combination of:

a weight of features used for searching for the reference multimedia data;

a weight of a frame or segment used for searching or browsing a specific video;

a weight of user-dependent information representing a user preference or user habit and a portion of data used for searching or browsing a main region or main object of the multimedia data; and

information for multimedia grouping of a similarity list and a cluster model.

13. A method of claim 11, wherein if at least two identical multimedia data having different variable information for a search or browsing are provided, selecting a multimedia data which has a relatively high reliability with respect to the variable information to calculate the new variable information.

14. A method of claim 11, wherein if at least two identical multimedia data having different variable information for a search or browsing are provided, combining the provided multimedia data in proportion to the reliability of the variable information value to calculate the new variable information.

15. A method of searching or browsing multimedia data comprising:

searching for a reference multimedia data using variable information representing importance of features of the reference multimedia data;

receiving user feedback on relevance of the searched multimedia data;

updating the variable information based on the relevance information, which is used as training information and reliability information; and

maintaining the updated variable information, wherein the reliability information includes a number of weight updates made by a user.

16. A method performed by a computing device, comprising;

receiving an image;

extracting characteristic attributes from the image;

ranking the extracted characteristic attributes of the image;

determining reliability of extracted characteristic attributes of the image;

forcing a data structure including at least the extracted characteristic attributes and the reliability information; and

searching a database of images using the extracted characteristic attributes, the ranking of the extracted characteristic attributes, and the determined reliability of the extracted characteristic attributes in said data structure, wherein the determined reliability of the extracted characteristic attributes is obtained based on authority of user feedback.

17. The method of claim 16, comprising steps of:

outputting, to a user, images resulting from the searching of the database of images;

inputting information from a user about the images resulting from the searching of the database of images;

ranking characteristic attributes of the image based on the information input from a user about the images resulting from the searching of the database of images; and

determining reliability of extracted characteristic attributes of the image based on the information input from a user about the images resulting from the searching of the database of images.

18. The method of claim 16, wherein the method is a computer implemented method for searching or browsing multimedia data.

19. A data record stored in a database, tangibly embodiied on a computer readable medium, for a multimedia data searching or browsing system, comprising;

multimedia data;

variable information representing features of the multimedia data; and

reliability information representing a reliability of the variable information, wherein the reliability of the variable information is obtained based on authority of user feedback.

20. A data structure of claim 19, wherein the reliability information includes information on the number of variable information updates by a user.

21. A data structure of claim 19, further comprising an authority code.

22. A data structure of claim 21, wherein the reliability information is variable and includes a number of authority levels, a degree of variable information update for each authority level, and a number of variable information updates by a user of each authority level.

23. A data structure of claim 21, wherein the reliability information is fixed and includes a number of variable information updates by a user of fixed authority levels.

24. A method of searching or browsing a multimedia data having a data structure as in claim 21.

25. A method of searching or browsing a multimedia data having a data structure as in claim 19.

26. The method of claim 19, wherein the reliability information is the number of weight updates made by a user.