Multimedia structure and method for browsing multimedia with defined priority of multimedia segments and semantic elements

• US 6,678,689 B2
• Filed: 12/21/2000
• Issued: 01/13/2004
• Est. Priority Date: 12/30/1999
• Status: Expired due to Fees

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Lee, Jin Soo, Yoon, Kyoung Ro, Jun, Sung Bae
• Primary Examiner(s): Mizrahi, Diane D.
• Application Number:
  US09/740,852
• Publication Number:
  US 20010056427A1
• Time in Patent Office: 1,118 Days
• Field of Search: 707/1, 707/2, 707/3, 707/5, 707/6, 707/100, 707/101, 707/102, 707/104.1
• US Class Current: 1/1
• CPC Class Codes: G06F 16/743 a collection of video files or sequences
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A more efficient data structure and method for browsing a multimedia is disclosed. In the present invention, link information between segments and semantic elements of a multimedia data structure is generated with minimal use of storage. Also, priority or weight of segments is represented based on semantic elements and priority or weight of semantic elements is represented based on segments to allow a search of multimedia based on content. The priority or weight information is represented as an attribute of the link information to increase the efficiency of the storage and a method of automatically extracting the priority or weight information is disclosed.

Claims

1. A computer readable medium having stored thereon:

a structure information of a plurality of segments;

a semantic information of a plurality of semantic elements; and

a link information which connects at least one selected segment to at least one semantic element, wherein said link information includes, as an attribute, at least one of a priority or weight information between each connected at least one segment and at least one semantic element.

2. The medium of claim 1, wherein the information forms a link from a selected semantic element to each segment which describes the selected semantic element.

3. The medium of claim 2, wherein the priority or weight information is based on the semantic element and indicates the importance of the selected semantic element in each of the linked segments.

4. The medium of claim 1, wherein the link information forms a link from a segment to each semantic element which is described in the selected segment.

5. The medium of claim 4, wherein the priority or weight information is based on the segment and indicates the importance of each linked semantic element in the selected segment.

6. A multimedia data structure The medium of claim 1, wherein the link information includes semantic based importance of a segment and segment based importance of a semantic element between the connected at least one segment and at least one semantic element.

7. The medium of claim 1, wherein the link information is represented by one of connectors, relations, or locators.

8. The medium of claim 1, wherein the priority or weight information is a value ranging from 0 to 1, where the value of 1 has the most importance.

9. The medium of claim 8, further comprising:

setting one of a priority or a weight value for an image to a ratio which is an amount of a semantic element that occupies the image;

setting a corresponding one of a priority or weight value for a video clip by a temporal ratio which is the length of a section in which a semantic element appears in a segment; and

setting a corresponding one of a priority or weight value for an audio by a ratio which is a combination of the degree of sound due to a semantic element and the length of time in which the sound is heard during a segment.

10. A method for browsing multimedia, comprising:

receiving a predetermined selection of either a semantic element from semantic information or a segment from structure information;

obtaining link information corresponding to a selected semantic element or segment, wherein the link information includes at least one of a priority or weight information as an attribute of the link information;

displaying segments linked with a selected semantic element in sequence according to the priority or weight information using the link information, if a semantic element is selected for browsing based on semantic elements; and

displaying sections which describe semantic elements linked with a selected segment in sequence according to the priority or weight information using the link information, if a segment is selected for browsing based on segments.

11. The method of claim 10, wherein the link information connects a selected segment to at least one semantic element if a segment is selected, and wherein the link information connects a selected semantic element to at least one segment if a semantic element is selected.

12. The method of claim 10, wherein the link information forms a link from a semantic element to each segment which describes the selected semantic element.

13. The method of claim 12, wherein the priority or weight information is based on the first semantic element and indicates the importance of the selected semantic element in each of the linked segments.

14. The method of claim 10, wherein the link information forms a link from a segment to each semantic element which is described in the selected segment.

15. The method of claim 14, wherein the priority or weight information is based on the segment and indicates the importance of each linked semantic element in the selected segment.

16. The method of claim 10, wherein the link information is represented by one of connectors, relations, or locators.

17. The method of claim 10, wherein the priority or weight information is a value ranging from 0 to 1, where the value of 1 indicates the most importance.

18. A method of using a computer processor for displaying multimedia in a multimedia browsing system, comprising:

storing a structure information of a plurality of segments, and a semantic information of a plurality of semantic elements;

storing link information which links at least one segment to corresponding at least one semantic element, wherein said link information includes, as an attribute, at least one of a priority information or weight information between the connected at least one segment and at least one semantic element;

obtaining link information corresponding to a semantic element or a segment, selected by a user for browsing; and

producing and displaying segments linked with a selected semantic element in sequence according to the priority or weight information using the link information, if a semantic element is selected for browsing based on semantic elements; and

producing and displaying sections which describes semantic elements linked with a selected segment in sequence according to the priority or weight information using the link information, if a segment is selected for browsing based on segments.

19. The method of claim 18, wherein the priority or weight information of the link information includes semantic based importance of a segment and segment based importance of a semantic element between the connected at least one segment and at least one semantic element.

20. The method of claim 18, wherein the link information is represented by one of connectors, relations, or locators.

21. A method for generating multimedia data, comprising:

receiving a predetermined selection of either a semantic element from semantic information or a segment from structure information;

obtaining link information corresponding to a selected semantic element or segment, wherein the link information includes a strength of the link information indicating the strength of a relationship between the semantic element and the segment of the selected semantic element; and

identifying a plurality of corresponding multimedia data ranked according to said strength of the link information.

22. The method of claim 21, wherein the strength of the link information comprises semantic based importance of a segment and segment based importance of a semantic element between the connected at least one segment and at least one semantic element.

23. The method of claim 21, wherein the link information comprises one of connectors, relations, or locators.

24. A method performed by a computing device, comprising:

receiving a predetermined selection of either a semantic element from semantic information or a segment from structure information;

forming a computer readable medium including a data structure having link information corresponding to a selected semantic element or segment, wherein the link information includes a strength of the link information indicating the strength of a relationship between the semantic element and the segment of the selected semantic element; and

searching a database of images for the selected semantic element or segment using the link information including the strength of the link information.

25. The method of claim 24, wherein the strength of the link information comprises semantic based importance of a segment and segment based importance of a semantic element between the connected at least one segment and at least one semantic element.

26. The method of claim 24, wherein the link information comprises one of connectors, relations, or locators.