Hybrid-captioning system

• US 20070126926A1
• Filed: 12/04/2005
• Published: 06/07/2007
• Est. Priority Date: 12/04/2005
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Miyamoto, Kohtaroh, Arakawa, Kenichi, Negishi, Noriko
• Application Number:
  US11/294,234
• Publication Number:
  US 20070126926A1
• Time in Patent Office: Days
• US Class Current: 348/468
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04N 7/0885 for the transmission of subtitles

Abstract

A hybrid-captioning system for editing captions for spoken utterances within video includes an editor-type caption-editing subsystem, a line-based caption-editing subsystem, and a mechanism. The editor-type subsystem is that in which captions are edited for spoken utterances within the video on a groups-of-line basis without respect to particular lines of the captions and without respect to temporal positioning of the captions in relation to the spoken utterances. The line-based subsystem is that in which captions are edited for spoken utterances within the video on a line-by-line basis with respect to particular lines of the captions and with respect to temporal positioning of the captions in relation to the spoken utterances. For each section of spoken utterances within the video, the mechanism is to select the editor-type or the line-based subsystem to provide captions for the section of spoken utterances in accordance with a predetermined criteria.

Claims

1. A hybrid-captioning system to edit captions for spoken utterances within video comprising:

an editor-type caption-editing subsystem in which captions are edited for spoken utterances within the video on a groups-of-lines basis without respect to particular lines of the captions and without respect to temporal positioning of the captions in relation to the spoken utterances;

a line-based caption-editing subsystem in which captions are edited for spoken utterances within the video on a line-by-line basis with respect to particular lines of the captions and with respect to temporal positioning of the captions in relation to the spoken utterances; and

,

a mechanism to, for each section of spoken utterances within the video, select the editor-type caption-editing subsystem or the line-based caption-editing subsystem to provide captions for the section of spoken utterances in accordance with a predetermined criteria.

2. The hybrid-captioning system of claim 1, wherein the mechanism, for each section of spoken utterances within the video, selects the editor-type caption-editing subsystem or the line-based caption-editing subsystem to provide the captions for the section of spoken utterances based on a certainty level of voice recognition as to the section of spoken utterances.

3. The hybrid-captioning system of claim 2, wherein, for each section of spoken utterances within the video, where the certainty level of voice recognition as to the section of spoken utterances is greater than a predetermined threshold, the mechanism selects the line-based caption-editing subsystem to provide the captions for the section of spoken utterances, and otherwise selects the editor-type caption-editing subsystem to provide the captions for the section of spoken utterances.

4. The hybrid-captioning system of claim 1, wherein the mechanism, for each section of spoken utterances within the video, selects the editor-type caption-editing subsystem or the line-based caption-editing subsystem to provide the captions for the section of spoken utterances based on voice recognition results on a sample of the spoken utterances within the section of spoken utterances.

5. The hybrid-captioning system of claim 1, wherein the mechanism selects the line-based caption-editing subsystem for the sections of spoken utterances in which voice recognition results have a relatively high level of accuracy and selects the editor-type caption-editing subsystem for the sections of spoken utterances in which voice recognition results have a relatively low level of accuracy.

6. The hybrid-captioning system of claim 1, wherein the video is real-time video, such that the captions are edited for the real-time video.

7. The hybrid-captioning system of claim 1, wherein the video is recorded video, such that the captions are edited for the recorded video.

8. The hybrid-captioning system of claim 1, wherein generation of the captions is independent of an input path selected from the group of input paths essentially consisting of:

a microphone path, and a file path.

9. A method comprising:

in relation to video for which captions are to be edited, receiving user input as to a current section of the video for which captions have been generated, within an editor-type caption-editing subsystem in which captions are edited for spoken utterances within the video on a groups-of-lines basis without respect to particular lines of the captions and without respect to temporal positioning of the captions in relation to the spoken utterances;

where the user input corresponds to termination of the editor-type caption editing subsystem,

transmitting the captions generated for the current section to a general-matching subsystem;

the general-matching subsystem transmitting the captions generated for the current section to a line-based caption-editing subsystem in which captions are edited for spoken utterances within the video on a line-by-line basis with respect to particular lines of the captions and with respect to temporal positioning of the captions in relation to the spoken utterances;

otherwise,

transmitting the captions generated for the current section to a particular-matching subsystem; and

,

the particular-matching subsystem transmitting the captions generated for the current section back to the editor-type caption-editing subsystem.

10. The method of claim 9, further comprising, where temporal positioning of the captions has been determined as to the current section of the video, transmitting the captions generated for the current section to the line-based caption-editing subsystem from the editor-type caption-editing subsystem.

11. The method of claim 9, further comprising, where the captions generated for the current section comprise one or more predicted character strings, presenting the predicted character strings to the user.

12. The method of claim 9, wherein the particular-matching subsystem is to match spoken utterances of the current section of the video to the captions that have been generated for the current section, where the current section of the video corresponds to a group of lines of captions without demarcation of the lines.

13. The method of claim 9, wherein the general-matching subsystem is to match spoken utterances of the current section of the video to the captions that have been generated for the current section, where the current section of the video corresponds to one or more individually demarcated lines of captions.

14. The method of claim 9, further comprising, after the captions generated for the current section have been transmitted to the general-matching subsystem, the general-matching subsystem performing:

performing character-based line matching as to the captions that have been generated for the current section to demarcate one or more lines of the captions;

where not all of the captions have been matched via character-based line matching, performing phoneme-based character matching as to the captions that have been generated for the current section; and

,

receiving a previously determined temporal positioning of the captions in relation to spoken utterances of the current section of the video.

15. The method of claim 9, further comprising, after the captions generated for the current section have been transmitted to the particular-matching subsystem, the particular-matching subsystem performing:

determining a voice-recognition rate of the captions that have been generated for the current section of the video;

where the voice-recognition rate is greater than a threshold, returning the captions as one or more predicted character strings;

otherwise, where the voice-recognition rate is not greater than the threshold,

performing phoneme-based character matching as to the captions that have been generated for the current section;

where the phoneme-based character matching results in a match value greater than a predetermined threshold,

returning the captions as the one or more predicted character strings; and

,

otherwise, where the match value is not greater than the predetermined threshold, returning indication that no matching has occurred.

16. The method of claim 9, further comprising repeating the method for each of a plurality of other sections of the video.

17. The method of claim 9, wherein the video is real-time video, such that the captions are edited for the real-time video.

18. The method of claim 9, wherein the video is recorded video, such that the captions are edited for the recorded video.

19. An article of manufacture comprising:

a tangible recordable data storage medium; and

,

means in the medium for selecting an editor-type caption-editing subsystem or a line-based caption-editing subsystem to provide captions for each of a plurality of sections of spoken utterances of video, in accordance with a predetermined criteria,

wherein the editor-type caption-editing subsystem is that in which captions are edited for spoken utterances within the video on a groups-of-lines basis without respect to particular lines of the captions and without respect to temporal positioning of the captions in relation to the spoken utterances, and

where the line-based caption-editing subsystem is that in which captions are edited for spoken utterances within the video on a line-by-line basis with respect to particular lines of the captions and with respect to temporal positioning of the captions in relation to the spoken utterances.

20. The article of manufacture of claim 19, wherein the predetermined criteria comprises a certainty level of voice recognition as to each section of spoken utterances within the video being greater than a predetermined threshold.