Subtitle generation and retrieval combining document processing with voice processing

• US 20070048715A1
• Filed: 01/23/2006
• Published: 03/01/2007
• Est. Priority Date: 12/21/2004
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Miyamoto, Kohtaroh, Arakawa, Kenichi, Negishi, Noriko
• Granted Patent: US 7,739,116 B2
• Time in Patent Office: Days
• US Class Current: 434/308
• CPC Class Codes: G06F 40/258 Heading extraction; Automatic titling; Numbering
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 5/44504 Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits

Abstract

Provides subtitle generation methods and apparatus which recognizes voice in a presentation to generate subtitles thereof, and retrieval apparatus for retrieving character strings by use of the subtitles. An apparatus of the present invention includes: a extraction unit for extracting text from presentation documents; an analysis unit for morphologically analyzing text to decompose it into words; a generation unit for generating common keywords by assigning weights to words; a registration unit for adding common keywords to a voice recognition dictionary; a recognition unit for recognizing voice in a presentation; a record unit for recording the correspondence between page and time by detecting page switching events; a regeneration unit for regenerating common keywords by further referring to the correspondence between page and time; a control unit for controlling the display of subtitles, common keywords, text and master subtitles; and a note generation unit for generating speaker notes from subtitles.

Claims

1. An apparatus for recognizing voice in a presentation to generate a subtitle corresponding to the voice, comprising:

extraction means for extracting a keyword from document data used in the presentation; and

processing means for one of generating the subtitle and assisting in generating the subtitle, by use of the keyword extracted by the extraction means.

2. The apparatus according to claim 1, wherein the extraction means assigns a weight to the keyword, and the processing means performs one of generating the subtitle and assisting in generating the subtitle, with the assigned weight taken into consideration.

3. The apparatus according to claim 2, wherein the extraction means assigns a weight to the keyword in the document data according to an attribute of the keyword.

4. The apparatus according to claim 2, wherein the extraction means assigns a weight to the keyword according to the number of times the keyword appeared in voice of the presentation.

5. The apparatus according to claim 1, wherein the processing means adds the keyword that has been extracted by the extraction means to a dictionary to be consulted at the time of recognizing the voice.

6. The apparatus according to claim 1, wherein the processing means performs at least one of:

setting a dictionary which belongs to a category suitable for the keyword, that has been extracted by the extraction means, as a dictionary to be consulted at the time of recognizing the voice; and

displaying the keyword that has been extracted by the extraction means together with the subtitle.

7. A computer program product comprising a computer usable medium having computer readable program code means embodied therein for causing recognition of voice in a presentation to generate a subtitle corresponding to the voice, the computer readable program code means in said computer program product comprising computer readable program code means for causing a computer to effect the functions of claim 1.

8. The apparatus according to claim 1, further comprising registration means for registering the subtitle that has been created by the processing means or an operation assisted by the processing means, so that the subtitle can be consulted at the presentation.

9. An apparatus for retrieving a character string, comprising:

storage means for storing first text data obtained by recognizing voice in a presentation, second text data extracted from document data used in the presentation, and associated information of the first text data and the second text data; and

retrieval means for retrieving, by use of the associated information, the character string from text data composed of the first text data and the second text data.

10. The apparatus according to claim 9, further comprising display means for displaying the result of retrieval conducted by the retrieval means, together with the associated information about the result of retrieval.

11. A method of causing a computer to combine a processing of a document having a plurality of pages with a processing of voice generated with reference to the document, comprising the steps of:

causing the computer to determine, among subtitles obtained by recognizing the voice, a specific subtitle obtained by recognizing voice generated with reference to a specific page of the document; and

causing the computer to store the correspondence between the specific subtitle and the specific page.

12. The method of claim 11, further comprising the step of causing the computer to display the specific subtitle together with information about the specific page.

13. The method of claim 12, wherein the specific information is text data contained in the specific page.

14. The method of claim 12, wherein the specific information concerns voice generated with reference to the specific page in the past.

15. The method of claim 11, further comprising the step of causing the computer to embed the specific subtitle in the specific page of the document.

16. The method according to claim 11, further comprising the step of causing the computer to retrieve character strings, with a retrieval target range extended from the specific subtitle to text data contained in the specific page.

17. A program product which allows a computer to realize:

a function of extracting a keyword from document data used in a presentation; and

a function of generating a subtitle corresponding to voice in the presentation or assisting in generating the subtitle, by use of the extracted keyword.

18. The program product according to claim 17, wherein in the function of generating the subtitle or assisting a process of generating the subtitle, the keyword that has been extracted is added to a dictionary to be consulted at the time of recognizing the voice.

19. The program product according to claim 17, wherein in the function of generating the subtitle or assisting in generating the subtitle, a dictionary which belongs to a category suitable for the keyword that has been extracted is set as a dictionary to be consulted at the time of recognizing the voice.

20. The program product according to claim 17, wherein in the function of generating the subtitle or assisting in generating the subtitle, the keyword that has been extracted is displayed together with the subtitle.

21. A program product which allows a computer to realize:

a function of determining, among subtitles obtained by recognizing voice generated with reference to a predetermined document, a specific subtitle obtained by recognizing voice generated with reference to a specific page of the document; and

a function of storing a correspondence between the specific subtitle and the specific page.

22. The program product according to claim 21, further allowing the computer to realize a function of displaying the specific subtitle together with specific information about the specific page.

23. The program product according to claim 21, further allowing the computer to realize a function of retrieving character strings, with the retrieval target range extended from the specific subtitle to text data contained in the specific page.