Method and system for the automatic generation of multi-lingual synchronized sub-titles for audiovisual data

• US 20020087569A1
• Filed: 11/27/2001
• Published: 07/04/2002
• Est. Priority Date: 12/07/2000
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fischer, Uwe, Stenzel, Gerhard, Kriechbaum, Werner, Hoffmann, Stefan
• Granted Patent: US 7,117,231 B2
• Time in Patent Office: Days
• US Class Current: 707/100
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99954 Version management

Abstract

The present invention provides a method and system for computerized synchronization of an audio stream having a first synchronized textual representation usable for subtitling of the audio stream in the original language, with a second synchronized textual representation which can be used as an alternative subtitle including a transcription of the original language into another language. Time synchronous links can be built between the audio stream and, for instance, the textual representations of the words spoken in the audio stream. More particularly, the second representation can inherit the synchronization between the audio stream and the first representation using structure association information determined between the first and the second representation.

Claims

1. A computer-based method of synchronizing a realization of a media stream having a first representation synchronized with said realization, and at least one second representation, said method comprising:

determining structure information for said first representation and said at least one second representation;

determining structure association information between said first representation and said at least one second representation; and

synchronizing said at least one second representation with said first synchronized representation and said realization using said structure association information.

2. The method according to claim 1, said step of determining structure information further comprising:

analyzing said structure information of said first and said at least one second representation, and providing a stream of tree locators.

3. The method according to claim 2, further comprising:

aligning said determined structure information of said first representation and said at least one second representation.

4. The method according to claim 3, wherein said realization comprises at least one version of content, said method further comprising:

aligning said at least one version of content with said first representation to produce a web of relations between said at least one version of content and said first representation.

5. The method according to claim 4, wherein said aligning said at least one version of content with said first representation produces a web of relations between a structural view of said at least one version of content and said first representation.

6. The method according to claim 3, further comprising:

aligning an audio stream specified by said media stream with an audio structure corresponding to said audio stream.

7. The method according to claim 3, further comprising:

aligning a text stream specified by said media stream with a text structure corresponding to said text stream.

8. A system for synchronizing a realization of a media stream having a first representation synchronized with said realization, and at least one second representation, said method comprising:

a first structurer configured to determine structure information for said first representation;

at least a second structurer configured to determine structure information for said at least one second representation; and

a first aligner configured to align said structure information for said first representation and said at least one second representation.

9. The system according to claim 8, further comprising:

at least one renderer configured to render said at least one second representation, after being synchronized, in a form suitable for displaying as an overlayed subtitle.

10. The system according to claim 9, wherein said realization specifies a media stream, said system further comprising:

a tree aligner configured to determine a tree structure for said media stream.

11. The system according to claim 10, further comprising:

means for detecting speech and non-speech boundaries.

12. The system according to claim 10, further comprising:

means for detecting transitions and speaker changes.

14. The machine-readable storage according to claim 13, said step of determining structure information further comprising:

analyzing said structure information of said first and said at least one second representation, and providing a stream of tree locators.

15. The machine-readable storage according to claim 14, further comprising:

aligning said determined structure information of said first representation and said at least one second representation.

16. The machine-readable storage according to claim 15, wherein said realization comprises at least one version of content, said machine-readable storage further comprising:

aligning said at least one version of content with said first representation to produce a web of relations between said at least one version of content and said first representation.

17. The machine-readable storage according to claim 15, wherein said aligning said at least one version of content with said first representation produces a web of relations between a structural view of said at least one version of content and said first representation.

18. The machine-readable storage according to claim 15, further comprising:

aligning an audio stream specified by said media stream with an audio structure corresponding to said audio stream.

19. The machine-readable storage according to claim 15, further comprising:

aligning a text stream specified by said media stream with a text structure corresponding to said text stream.

13. A machine-readable storage, having stored thereon a computer program having a plurality of code sections executable by a machine for causing the machine to perform the steps of:

determining structure information for a first representation being synchronized to a corresponding media stream and at least one second representation;

determining structure association information between said first representation and said at least one second representation; and

synchronizing said at least one second representation with said first synchronized representation and said realization using said structure association information.