Automatic content analysis and representation of multimedia presentations

• US 7,046,914 B2
• Filed: 05/01/2001
• Issued: 05/16/2006
• Est. Priority Date: 05/01/2001
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Dimitrova, Nevenka, Jasinschi, Radu S.
• Primary Examiner(s): NGUYEN, HUY THANH
• Application Number:
  US09/846,461
• Publication Number:
  US 20020164151A1
• Time in Patent Office: 1,841 Days
• Field of Search: 386/46, 386/83, 386/95, 386/121, 386/52, 386/96
• US Class Current: 386/241
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G11B 2220/20 Disc-shaped record carriers
• CPC Class Codes: G11B 2220/41 Flat as opposed to hierarchical combination, e.g. library of tapes or discs, CD changer, or groups of record carriers that together store one title
• CPC Class Codes: G11B 2220/65 wherein solid state memory is used for storing indexing information or metadata
• CPC Class Codes: G11B 2220/90 Tape-like record carriers
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

For use in a multimedia analysis system capable of analyzing the content of multimedia signals, there is disclosed an apparatus and method for creating a multimedia table of contents of videotaped material. In one advantageous embodiment, the apparatus of the present invention comprises a multimedia table of contents controller that is capable of receiving video signals, audio signals, and text signals of said videotaped material, and capable of combining portions of the video signals, audio signals, and text signals to create a table of contents of the videotaped material. The controller is capable of segmenting video signals with both a coarse and fine segmentation application. The controller is also capable of locating boundaries of elements of the videotaped material with both a coarse and fine boundary detection application. An index module of the controller links elements of the table of contents with combinations of audio, visual, and transcript cues. A retrieval module retrieves and displays a table of contents in response to a user request.

Claims

1. For use in a multimedia analysis system capable of analyzing content of multimedia signals, an apparatus for creating a multimedia table of contents of videotaped material, said apparatus comprising:

a multimedia table of contents controller capable of;

receiving video signals, audio signals, and text signals of said videotaped material;

combining portions of said video signals, audio signals, and text signals of said videotaped material to create a table of contents of said videotaped material;

creating said table of contents of said videotaped material by selecting a video segment that relates to an element of said videotaped material, and by adding said video segment to said table of contents of said videotaped material; and

executing computer software instructions contained within a memory unit coupled to said controller;

to create said table of contents of said videotaped material by segmenting a video signal of said videotaped material into elements using a coarse table of contents segmentation application, and by locating video boundaries of said elements of said videotaped material using a coarse boundary detection application; and

to create said table of contents of said videotaped material by segmenting said video signal of said videotaped material into said elements using a fine table of contents segmentation application, and by locating said video boundaries of said elements of said videotaped material using a fine boundary detection application.

2. The apparatus as claimed in claim 1, wherein said controller further comprises:

an index module capable of linking said elements of said videotaped material selected f or said table of contents, and capable of linking said elements with combinations of audio, visual, and transcript cues.

3. The apparatus as claimed in claim 2 wherein said controller further comprises:

a retrieval module capable of retrieving a table of contents stored in said memory unit and causing said table of contents to be displayed in response to a user request.

4. The apparatus as claimed in claim 1 wherein said multimedia table of contents controller is capable of combining portions of said video signals, audio signals, and text signals of said videotaped material to create a multimedia index of said videotaped material.

5. The apparatus as claimed in claim 4 wherein said multimedia index of said videotaped material comprises one of:

a specialized topical multimedia index, a multimedia bibliography, and a multimedia glossary.

6. A multimedia analysis system capable of analyzing content of multimedia signals, said multimedia analysis system comprising an apparatus for creating a multimedia table of contents of videotaped material, said apparatus comprising:

a multimedia table of contents controller capable of;

receiving video signals, audio signals, and text signals of said videotaped material;

combining portions of said video signals, audio signals, and text signals of said videotaped material to create a table of contents of said videotaped material;

creating said table of contents of said videotaped material by selecting a video segment that relates to an element of said videotaped material, and by adding said video segment to said table of contents of said videotaped material; and

executing computer software instructions contained within a memory unit coupled to said controller;

to create said table of contents of said videotaped material by segmenting a video signal of said videotaped material into elements using a coarse table of contents segmentation application, and by locating video boundaries of said elements of said videotaped material using a coarse boundary detection application; and

to create said table of contents of said videotaped material by segmenting said video signal of said videotaped material into said elements using a fine table of contents segmentation application, and by locating said video boundaries of said elements of said videotaped material using a fine boundary detection application.

7. The multimedia analysis system as claimed in claim 6, wherein said controller further comprises:

an index module capable of linking said elements of said videotaped material selected for said table of contents, and capable of linking said elements with combinations of audio, visual, and transcript cues.

8. The multimedia analysis system as claimed in claim 7 wherein said controller further comprises:

a retrieval module capable of retrieving a table of contents stored in said memory unit and causing said table of contents to be displayed in response to a user request.

9. The multimedia analysis system as claimed in claim 6 wherein said multimedia table of contents controller is capable of combining portions of said video signals, audio signals, and text signals of said videotaped material to create a multimedia index of said videotaped material.

10. The multimedia analysis system as claimed in claim 9 wherein said multimedia index of said videotaped material comprises one of:

a specialized topical multimedia index, a multimedia bibliography, and a multimedia glossary.

11. For use in a multimedia analysis system capable of analyzing content of multimedia signals, a method for creating a multimedia table of contents of videotaped material, said method comprising the steps of:

receiving in a multimedia table of contents controller video signals, audio signals, and text signals of said videotaped material;

combining portions of said video signals, audio signals, and text signals of said videotaped material in said multimedia table of contents controller to create said multimedia table of contents;

wherein the step of combining portions of said video signals, audio signals, and text signals of said videotaped material in said multimedia table of contents controller to create said multimedia table of contents comprises the steps of;

selecting a video segment that relates to an element of said videotaped material; and

adding said video segment to said table of contents of said videotaped material;

receiving in said multimedia table of contents controller instructions from computer software stored in a memory unit coupled to said multimedia table of contents controller;

executing said instructions in said multimedia table of contents controller to segment a video signal of said videotaped material into elements using a coarse table of contents segmentation application;

executing said instructions in said multimedia table of contents controller to locate video boundaries of said elements of said videotaped material using a coarse boundary detection application;

executing said instructions in said multimedia table of contents controller to segment said video signal of said videotaped material into said elements using a fine table of contents segmentation application; and

executing said instructions in said multimedia table of contents controller to locate said video boundaries of said elements of said videotaped material using a fine boundary detection application.

12. The method as claimed in claim 11, further comprising the steps of:

linking said elements of said videotaped material selected for said table of contents using an index module; and

linking said elements of said videotaped material with combinations of audio, visual, and transcript cues using said index module.

13. The method as claimed in claim 12 further comprising the steps of:

retrieving a table of contents stored in said memory unit in response to a user request using a retrieval module; and

causing said table of contents to be displayed.

14. The method as claimed in claim 11 further comprising the step of:

combining portions of said video signals, audio signals, and text signals of said videotaped material in said multimedia table of contents controller to create a multimedia index.

15. The method as claimed in claim 14 wherein said multimedia index comprises one of:

a specialized multimedia index, a multimedia bibliography, and a multimedia glossary.

16. For use in a multimedia analysis system capable of analyzing content of multimedia signals, computer-executable instructions stored on a computer-readable storage medium for creating a multimedia table of contents of videotaped material, the computer-executable instructions comprising the steps of:

receiving in a multimedia table of contents controller video signals, audio signals, and text signals of said videotaped material;

combining portions of said video signals, audio signals, and text signals of said videotaped material in said multimedia table of contents controller to create said multimedia table of contents;

wherein the step of combining portions of said video signals, audio signals, and text signals of said videotaped material in said multimedia table of contents controller to create said multimedia table of contents comprises the steps of;

selecting a video segment that relates to an element of said videotaped material; and

adding said video segment to said table of contents of said videotaped material;

receiving in said multimedia table of contents controller instructions from computer software stored in a memory unit coupled to said multimedia table of contents controller;

executing said instructions in said multimedia table of contents controller to segment a video signal of said videotaped material into elements using a coarse table of contents segmentation application; and

executing said instructions in said multimedia table of contents controller to locate video boundaries of said elements of said videotaped material using a coarse boundary detection application;

executing said instructions in said multimedia table of contents controller to segment said video signal of said videotaped material into said elements using a fine table of contents segmentation application; and

executing said instructions in said multimedia table of contents controller to locate said video boundaries of said elements of said videotaped material using a fine boundary detection application.

17. The computer-executable instructions stored on a computer-readable storage medium as claimed in claim 16, further comprising the steps of:

linking said elements of said videotaped material selected for said table of contents using an index module; and

linking said elements of said videotaped material with combinations of audio, visual, and transcript cues using said index module.

18. The computer-executable instructions stored on a computer-readable storage medium as claimed in claim 17 further comprising the steps of:

retrieving a table of contents stored in said memory unit in response to a user request using a retrieval module; and

causing said table of contents to be displayed.

19. The computer-executable instructions stored on a computer-readable storage medium as claimed in claim 16 further comprising the step of:

combining portions of said video signals, audio signals, and text signals of said videotaped material in said multimedia table of contents controller to create a multimedia index.

20. The computer-executable instructions stored on a computer-readable storage medium as claimed in claim 19 wherein said multimedia index comprises one of:

a specialized multimedia index, a multimedia bibliography, and a multimedia glossary.