SYNCHRONIZATION FOR DOCUMENT NARRATION

• US 20100324895A1
• Filed: 01/14/2010
• Published: 12/23/2010
• Est. Priority Date: 01/15/2009
• Status: Abandoned Application

• Current Assignee: EM Acquisition Corp., Inc.
• Original Assignee: K-NFB Reading Technology Inc.
• Inventors: Gibson, Lucy, Albrecht, Paul, Kurzweil, Raymond C., Chapman, Peter
• Application Number:
  US12/687,240
• Publication Number:
  US 20100324895A1
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems

Abstract

Disclosed are techniques and systems for synchronizing an audio file with a sequence of words displayed on a user interface.

Claims

1. A computer implemented method comprising:

applying speech recognition by one or more computer systems to an audio recording to generate a text version of recognized portions of text;

determining by the one or more computer systems an elapsed time period from a reference time in the audio recording to each recognized portion in the audio recording;

comparing by the one or more computer systems the recognized portions of text to expected portions of text; and

generating by the one or more computer systems a timing file that is stored on a computer-readable storage medium, the timing file comprising the elapsed time information for each expected portion of text by;

storing the elapsed time information for a recognized portion into the timing file if the recognized portion matches the corresponding expected portion of text; and

otherwise

computing the elapsed time information for the expected portion of text and storing the computed elapsed time information into the timing file if the recognized portion does not match the corresponding expected portion of text.

2. The method of claim 1, wherein the one or more recognized portions or expected portions of text comprise words.

3. The method of claim 1, further comprising,

during play back:

providing an audible output corresponding to the audio recording; and

displaying a sequence of words corresponding to at least a portion of the expected portion of text on a user interface rendered on a display device and providing visual indicia indicating a correspondence between the audio recording and the expected portion of text.

4. The method of claim 1 wherein one or more of the recognized portions or the expected portions of text are syllables.

5. The method of claim 1 wherein computing further comprises:

determining the number of syllables in the expected portion of text;

determining the elapsed time for the determined number of syllables in the recognized portion, and

outputting the determined elapsed time to the timing file.

6. The method of claim 1 wherein computing further comprises:

determining the elapsed time for an expected portion of text based on a metric associated with an expected length of time to verbalize the expected portion of text.

7. The method of claim 1 wherein computing comprises:

displaying on a user interface device, the recognized portions of text, the elapsed times, and the expected portions of text;

receiving from a user an indication of timings for the expected portions of text; and

storing elapsed time information in the timing file based on the received user indications.

8. A computer program product residing on a computer readable medium, the computer program product comprising instructions for causing a processor to:

apply speech recognition to an audio recording to generate a text version of recognized portions of text;

determine an elapsed time period from a reference time in the audio recording to each recognized portion in the audio recording;

generate a timing file that is stored on a computer-readable storage medium, the timing file comprising the elapsed time information for each expected portion of text by storing the elapsed time information for a recognized portion into the word timing file if the recognized portion matches the corresponding expected portion of text, and otherwise computing the elapsed time information for the expected portion of text and storing the computed elapsed time information into the timing file if the recognized portion does not match the expected portion of text.

9. The computer program product of claim 8, wherein the one or more recognized portions or portions of text comprise words.

10. The computer program product of claim 8 wherein the one or more of the recognized portions or portions of text comprise syllables.

11. The computer program product of claim 8, further comprising, during playback:

provide an audible output corresponding to the audio recording;

display a sequence of words corresponding to at least a portion of the expected portion of text on a user interface rendered on a display device; and

provide visual indicia indicating a correspondence between the portions in the audio recording and the expected portion of text.

12. The computer program product of claim 8 wherein the instructions to compute the elapsed time information further comprise instructions to:

determine the elapsed time for an expected portion of text based on a metric associated with an expected length of time to verbalize the expected portion of text.

13. The computer program product of claim 8 wherein the instructions to compute the elapsed time information comprise instructions to:

display on a user interface device, the recognized portions of text, the elapsed times, and the expected portions of text;

receive from a user an indication of timings for the expected portions of text; and

store elapsed time information in the timing file based on the received user indications.

14. A system comprising:

a memory; and

a computing device configured to;

apply speech recognition to an audio recording to generate a text version of recognized portions of text;

determine an elapsed time period from a reference time in the audio recording to each recognized portion in the audio recording version;

generate a timing file that is stored on a computer-readable storage medium, the timing file comprising the elapsed time information for each expected portion of text by storing the elapsed time information for a recognized portion into the timing file if the recognized portion matches the corresponding expected portion of text, and otherwise computing the elapsed time information for the expected portion of text and storing the computed elapsed time information into the timing file word if the recognized portion does not match the expected portion of text.

15. The system of claim 14, wherein the one or more recognized portions or portions of text comprise words.

16. The system of claim 14, wherein the one or more recognized portions or portions of text comprise syllables.

17. The system of claim 14, wherein the computing device is further configured to, during playback:

provide an audible output corresponding to the audio recording;

display a sequence of words corresponding to at least a portion of the expected portion of text on a user interface rendered on a display device; and

provide visual indicia indicating a correspondence between the portions in the audio recording and the expected portion of text.

18. The system of claim 14, wherein the computing device is further configured to:

determine the elapsed time for an expected portion of text based on a metric associated with an expected length of time to verbalize the expected portion of text.

19. The system of claim 14, wherein the computing device is further configured to:

display on a user interface device, the recognized portions of text, the elapsed times, and the expected portions of text;

receive from a user an indication of timings for the expected portions of text; and

store elapsed time information in the timing file based on the received user indications.