TRANSLATION AND INTEGRATION OF PRESENTATION MATERIALS WITH CROSS-LINGUAL MULTI-MEDIA SUPPORT

• US 20150120278A1
• Filed: 01/05/2015
• Published: 04/30/2015
• Est. Priority Date: 06/11/2013
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Waibel, Alexander
• Granted Patent: US 9,678,953 B2
• Time in Patent Office: Days
• US Class Current: 704/2
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/237 Lexical tools
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/005 Language recognition

Abstract

Computer-based systems and methods are disclosed for translation of a multi-media presentation (e.g., a lecture) along with the accompanying presentation materials. Translation and delivery of text-based presentation materials to a listener is annotated and aligned with audio, so that the listener can follow both the audio and the presentation material. The content of the presentation materials can be used to improve presentation translation, as it extracts supportive material from the presentation materials as they relate to the speech.

Claims

1. A method comprising:

recognizing speech by a speaker in a first language;

translating, by a translation system, the recognized speech into a second language;

receiving presentation materials presented with the speech in the first language;

extracting a portion of text in the first language from the presentation materials;

generating a third marker at a time associated with a portion of the presentation materials;

translating, by a translation system, the extracted text into the second language;

generating a translation of the presentation materials in the second language based on the text in the second language; and

aligning the speech in the second language and the translated presentation materials.

2. The method of claim 1, wherein the original presentation materials comprise a first plurality of slides.

3. The method of claim 1, further comprising:

generating a first marker at a time associated with a portion of the speech in the first language;

generating, based on the first marker, a second marker at a time associated with a portion of the translation in the second language synchronized with the first marker;

generating, based on the third marker, a fourth marker at a time associated with a portion of the translated presentation materials synchronized with the third marker; and

wherein aligning the speech in the second language and the translated presentation materials aligns the second marker with the fourth marker.

4. The method of claim 3, wherein the second marker comprises a time-stamp associated with a transition from a first slide to a second slide, and the fourth marker comprises a time-stamp associated with a portion of the speech in the second language, the portion of the speech in the second language corresponding to the transition from the first slide to the second slide.

5. The method of claim 3, wherein the generated translated presentation materials comprise a second plurality of slides.

6. The method of claim 5, wherein each of the second plurality of slides comprises the text in the first language and the text in the second language.

7. The method of claim 6 further comprising:

receiving an input from a user, the input associated with moving a cursor over a portion of the text in the first language on the second plurality of slides; and

responsive to the cursor moving over the portion of the text in the first language on the second plurality of slides, displaying a portion of the text in the second language corresponding to the portion of text in the first language.

8. The method of claim 7, wherein the portion of the text in the second language is displayed in a translation bubble adjacent to the portion of text in the first language.

9. The method of claim 6, wherein the portion of the original text comprises one line of the original text.

10. The method of claim 6, wherein the portion of the original text comprises one word of the original text.

11. The method of claim 3, wherein the first marker comprises a pause in the translated speech and the second marker comprises a break in the translated presentation materials corresponding to the pause in the translated speech.

12. The method of claim 3, wherein the first marker comprises a content item in the translated presentation materials, the content item selected from the group consisting of graphics, tables, and formulas, and the second marker comprises a passage in the translation, the passage corresponding to the identified content item.

13. The method of claim 1, wherein the speaker is a user of the translation system.

14. A computer program product for translating multimedia presentations, the computer program product comprising a non-transitory computer-readable storage medium containing computer program code for:

recognizing speech by a speaker in a first language;

translating, by a translation system, the recognized speech into a second language;

generating, based on the first marker, a second marker at a time associated with a portion of the translated speech in the second language synchronized with the first marker;

receiving presentation materials presented with the speech in the first language;

extracting a portion of text in the first language from the presentation materials;

translating, by a translation system, the extracted text into the second language;

generating, based on the third marker, a fourth marker at a time associated with a portion of the translated presentation materials synchronized with the third marker; and

aligning the speech in the second language and the translated presentation materials.

15. The computer program product of claim 14, wherein the original presentation materials comprise a first plurality of slides.

16. The computer program product of claim 14, further comprising computer program code for:

generating a first marker at a time associated with a portion of the speech in the first language;

generating, based on the first marker, a second marker at a time associated with a portion of the translated speech in the second language synchronized with the first marker;

generating, based on the third marker, a fourth marker at a time associated with a portion of the translated presentation materials synchronized with the third marker; and

wherein aligning the speech in the second language and the translated presentation materials aligns the second marker with the fourth marker.

17. The computer program product of claim 16, wherein the second marker comprises a time-stamp associated with a transition from a first slide to a second slide, and the fourth marker comprises a time-stamp associated with a portion of the speech in the second language, the portion of the speech in the second language corresponding to the transition from the first slide to the second slide.

18. The computer program product of claim 15, wherein the generated translated presentation materials comprise a second plurality of slides.

19. The computer program product of claim 18, wherein each of the second plurality of slides comprises the text in the first language and the text in the second language.

20. The computer program product of claim 16, wherein the first marker comprises a pause in the translated speech and the second marker comprises a break in the translated presentation materials corresponding to the pause in the translated speech.

21. The computer program product of claim 16, wherein the first marker comprises a content item in the translated presentation materials, the content item selected from the group consisting of graphics, tables, and formulas, and the second marker comprises a passage in the translated speech, the passage corresponding to the identified content item.