Video browser based on character relation

• US 7,509,581 B1
• Filed: 08/25/2000
• Issued: 03/24/2009
• Est. Priority Date: 08/26/1999
• Status: Expired due to Fees

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Lee, Jin Soo, Song, Jung Min
• Primary Examiner(s): KE, PENG
• Application Number:
  US09/645,613
• Time in Patent Office: 3,133 Days
• Field of Search: 345/704, 345/716, 345/717, 345/718, 345/723, 345/724, 345/719, 715/719, 715/720, 715/722, 715723-726, 715/716, 715/853, 715/854, 725/37, 725/44, 725/46, 725/47, 725/53
• US Class Current: 715/721
• CPC Class Codes: G06F 16/745 the internal structure of a single video sequence
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/326 used signal is a video-frame or a video-field (P.I.P.)
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

A video data structure and a video browser for video browsing based on content is disclosed. An embodiment of the present invention allows users to easily understand relationships between characters and changes in relationships between characters in a movie or drama by displaying significant events in connection to corresponding relationships and by displaying character relationships in a tree structure.

Claims

1. An apparatus for processing multimedia data, the apparatus comprising:

a memory; and

a processor configured to perform operations comprising;

receiving segment information about an audio-visual program, the segment information including segment location information identifying a plurality of audio-visual segments in the audio-visual program, wherein the segment location information defines each audio-visual segment by a temporal position in a multimedia stream of the audio-visual program and wherein each audio-visual segment represents a continuous temporal content portion in the audio-visual program;

generating a hierarchical data structure configured to facilitate browsing content in the audio-visual program, the hierarchical data structure including a first structural part which stores the segment information and a distinct second structural part which stores segment group information defining first and second segment groups for the audio-visual program, each of the first and second segment groups defining a respective set of non-contiguous audio-visual segments which are selected from the plurality of audio-visual segments and identified in the segment group information by references to the corresponding segments in the segment information of the audio-visual program, wherein said segment group information specifies a respective group type and a respective duration for each of said first and second segment groups, the respective group types indicating that the first and second segment groups represent respective first and second content summaries related to objects or events as depicted in the audio-visual program, and wherein the segment group information includes segment order information defining that (i) the audio-visual segments within the first segment group are ordered relative to each other according to a time sequence that is significant for the first content summary'"'"'s representation of the corresponding events or objects in the audio-visual program and (ii) the audio-visual segments within the second segment group are not ordered relative to each other according to any time sequence that is significant for the second content summary'"'"'s representation of the corresponding events or objects in the audio-visual program; and

storing the hierarchical data structure including the segment information and the segment group information in the memory.

2. The apparatus of claim 1, wherein said segment group information includes a level information.

3. The apparatus of claim 2, wherein said level information defines multiple levels.

4. The apparatus of claim 1, wherein said segment location information defines each audio-visual segment contained in the first and second segment groups by a respective start time and a respective segment duration.

5. The apparatus of claim 1, wherein the respective group types for the first and second segment groups specify that the first and second segment groups are related to at least two characters represented in the content of the multimedia stream.

6. The apparatus of claim 5, wherein the first segment group includes audio-visual segments representing time ordered relation changes between the at least two characters.

7. The apparatus of claim 5, wherein the audio-visual segments within the second segment group include audio-visual segments representing constant relations between the at least two characters.

8. The apparatus of claim 1, wherein the segment group information indicates that the first segment group includes audio-visual segments that represent highlights from the multimedia stream.

9. A method implemented by an apparatus for processing multimedia data, the method comprising:

generating segment information about an audio-visual program as a first structural part of a hierarchical data structure configured to facilitate browsing content in the audio-visual program, the segment information including segment location information identifying a plurality of audio-visual segments in the audio-visual program, wherein the segment location information defines each audio-visual segment by a temporal position in a multimedia stream of the audio-visual program and wherein each audio-visual segment represents a continuous temporal content portion in the audio-visual program;

generating segment group information as a distinct second structural part of the hierarchical data structure configured to facilitate browsing content in the audio-visual program; and

transmitting said segment information and said segment group information to a client,

wherein said segment group information;

defines first and second segment groups for the audio-visual program, each of which defines a respective set of non-contiguous audio-visual segments which are selected from the plurality of audio-visual segments and identified in the segment group information by references to the corresponding segments in the segment information of the audio-visual program;

specifies a respective group type and a respective duration for each of said first and second segment groups, the respective group types indicating that the first and second segment groups represent respective first and second content summaries related to objects or events as depicted in the audio-visual program; and

includes segment order information defining that (i) the audio-visual segments within the first segment group are ordered relative to each other according to a time sequence that is significant for the first content summary'"'"'s representation of the corresponding events or objects in the audio-visual program and (ii) the audio-visual segments within the second segment group are not ordered relative to each other according to any time sequence that is significant for the second content summary'"'"'s representation of the corresponding events or objects in the audio-visual program.

10. The method of claim 9, wherein said segment group information includes a level information.

11. The method of claim 10, wherein said level information defines multiple levels.

12. The method of claim 9, wherein the segment location information defines a respective start time and a respective segment duration for each audio-visual segment contained in the first and second segment groups.

13. An apparatus for processing multimedia data, the apparatus comprising:

a memory; and

a processor configured to perform operations comprising;

receiving segment information about an audio-visual program, the segment information including segment location information identifying a plurality of audio-visual segments in the audio-visual program, wherein segment location information defines each audio-visual segment by a temporal position in a multimedia stream of the audio-visual program and wherein each audio-visual segments represents a continuous temporal content portion in the audio-visual program;

receiving segment group information defining first and second segment groups for the audio-visual program;

generating a hierarchical data structure configured to facilitate browsing content in the audio-visual program, the hierarchical data structure including a first structural part which stores the received segment information and a distinct second structural part which stores the segment group information, wherein each of the first and second segment groups define a respective set of non-contiguous audio-visual segments which are selected from the plurality of audio-visual segments and identified in the segment group information by references to the corresponding segments in the segment information of the audio-visual program, wherein said segment group information specifies a respective group type and a respective duration for each of said first and second segment groups, the respective group types indicating that the first and second segment groups represent respective first and second content summaries related to objects or events as depicted in the audio-visual program, and wherein the segment group information includes segment order information defining that (i) the audio-visual segments within the first segment group are ordered relative to each other according to a time sequence that is significant for the first content summary'"'"'s representation of the corresponding events or objects in the audio-visual program and (ii) the audio-visual segments within the second segment group are not ordered relative to each other according to any time sequence that is significant for the second content summary'"'"'s representation of the corresponding events or objects in the audio-visual program; and

storing the hierarchical data structure including the segment information and the segment group information in the memory.

14. The apparatus of claim 13, wherein said segment group information includes a level information.

15. The apparatus of claim 14, wherein said level information defines multiple levels.

16. The apparatus of claim 13, wherein the segment location information defines each audio-visual segment contained in the first and second segment groups by a respective start time and a respective segment duration.

17. A method implemented by an apparatus for processing multimedia data, the method comprising:

receiving segment information about an audio-visual program, the segment information including segment location information identifying a plurality of audio-visual segments in the audio-visual program, wherein the segment location information defines each audio-visual segment by a temporal position in a multimedia stream of the audio-visual program and wherein each audio-visual segment represents a continuous temporal content portion in the audio-visual program;

receiving segment group information from a provider; and

storing said received segment information and said segment group information in distinct first and second structural parts of a hierarchical data structure configured to facilitate browsing content in the audio-visual program in a client, wherein said segment group information;

defines first and second segment groups for the audio-visual program, each of which defines a respective set of non-contiguous audio-visual segments that are selected from the plurality of audio-visual segments and identified in the segment group information by references to the corresponding segments in the segment information of the audio-visual program;

specifies a respective group type and a respective duration for each of said first and second segment groups, the respective group types indicating that the first and second segment groups represent respective first and second content summaries related to objects or events as depicted in the audio-visual program; and

includes segment order information defining that (i) the audio-visual segments within the first segment group are ordered relative to each other according to a time sequence that is significant for the first content summary'"'"'s representation of the corresponding events or objects in the audio-visual program and (ii) the audio-visual segments within the second segment group are not ordered relative to each other according to any time sequence that is significant for the second content summary'"'"'s representation of the corresponding events or objects in the audio-visual program.

18. The method of claim 17, wherein said segment group information includes a level information.

19. The method of claim 18, wherein said level information defines multiple levels.

20. The method of claim 17, wherein the segment location information defines each audio-visual segment contained in the first and second segment groups by a respective start time and a respective segment duration.

21. A storage medium storing a hierarchical data structure configured to be processed by multimedia data processing apparatus to facilitate browsing content in an audio-visual program, the stored hierarchical data structure comprising:

a first structural part which stores segment information about the audio-visual program, the segment information including segment location information identifying a plurality of audio-visual segments in the audio-visual program, wherein the segment location information defines each audio-visual segment by a temporal position in a multimedia stream of the audio-visual program and wherein each audio-visual segment represents a continuous temporal content portion in the audio-visual program;

a distinct second structural part which stores segment group information defining first and second segment groups for the audio-visual program, each of which defines a respective set of non-contiguous audio-visual segments that are selected from the plurality of audio-visual segments and identified in the segment group information by references to the corresponding segments in the segment information of the audio-visual program, wherein said segment group information specifies a respective group type and a respective duration for each of said first and second segment groups, the respective group types indicating that the first and second segment groups represent respective first and second content summaries related to objects or events as depicted in the audio-visual program, and wherein the segment group information includes segment order information defining that (i) the audio-visual segments within the first segment group are ordered relative to each other according to a time sequence that is significant for the first content summary'"'"'s representation of the corresponding events or objects in the audio-visual program and (ii) the audio-visual segments within the second segment group are not ordered relative to each other according to any time sequence that is significant for the second content summary'"'"'s representation of the corresponding events or objects in the audio-visual program.

22. The storage medium of claim 21, wherein said segment group information includes a level information.

23. The storage medium of claim 22, wherein said level information defines multiple levels.

24. The storage medium of claim 21, wherein the segment location information defines a respective start time and a respective segment duration for each audio-visual segment contained in the first and second segment groups.