Method and apparatus for annotating a line-based document

• US 7,500,193 B2
• Filed: 08/18/2005
• Issued: 03/03/2009
• Est. Priority Date: 03/09/2001
• Status: Expired due to Fees

• Current Assignee: Copernicus Investments, LLC
• Original Assignee: Copernicus Investments, LLC
• Inventors: Spielberg, Steven, Gustman, Samuel
• Primary Examiner(s): Hutton; Doug
• Assistant Examiner(s): Tran; Quoc A
• Application Number:
  US11/207,598
• Publication Number:
  US 20060143559A1
• Time in Patent Office: 1,293 Days
• Field of Search: 715/512, 715/235
• US Class Current: 715/727
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

To facilitate the use of audio files for annotation purposes, an audio file format, which includes audio data for playback purposes, is augmented with a parallel data channel of line identifiers, or with a map associating time codes for the audio data with line numbers on the original document. The line number-time code information in the audio file is used to navigate within the audio file, and also to associate bookmark links and captured audio annotation files with line numbers of the original text document. An annotation device may provide an output document wherein links to audio and/or text annotation files are embedded at corresponding line numbers. Also, a navigation index may be generated, having links to annotation files and associated document line numbers, as well as bookmark links to selected document line numbers.

Claims

1. An apparatus for annotating a line-based document comprising lines of text, said lines of text being associated with corresponding line identifiers, said apparatus comprising:

an audio codec coupled to an audio output device;

a processing unit;

a computer readable medium having computer program code embodied therein for annotating said line-based document with at least one audible annotation received in connection with audio data generated using said line-based document, wherein said computer program code is configured to cause said processing unit to provide;

a voice recognition function, said voice recognition function configured to detect one or more audible document navigation commands and one or more audible annotation commands from audio input;

a navigation function responsive to a detected document navigation command received from said voice recognition function, said detected document navigation command comprising a first of said line identifiers associated with said lines of text of said line-based document, said navigation function configured to determine an audio time code for said audio data corresponding to said first line identifier and to direct said audio codec to play back said audio data from said determined audio time code;

an annotation function responsive to a detected annotation command received from said voice recognition function, said annotation function configured to capture an audible annotation via said audio input and to store said audible annotation as an audio annotation file; and

an index generator configured to add an annotation link entry to an index file to link said audible annotation to a second of said lines of text of said line-based document, said annotation link entry having a first reference to said audio annotation file and a second reference to said second line of text.

2. The apparatus of claim 1, wherein said navigation function is configured to track a current line identifier from a current time code of said audio data.

3. The apparatus of claim 2, wherein said index generator is configured to obtain said current line identifier to provide said second reference.

4. The apparatus of claim 2, further comprising:

a display unit; and

said computer program code further configured to cause said processing unit to provide a browser function configured to render hypertext data associated with said line-based document, said browser configured to render said hypertext data to said display.

5. The apparatus of claim 4, wherein said navigation function is configured to update said browser with said current line identifier as said audio codec plays back said audio data.

6. The apparatus of claim 2, wherein said computer readable medium further comprises:

computer program code configured to cause said processing unit to provide a bookmark function responsive to a detected bookmark command received from said voice recognition function, said bookmark function configured to store said current line identifier as a bookmark line identifier.

7. The apparatus of claim 6, wherein said index generator is further configured to add to said index file a bookmark link having a third reference to said bookmark line identifier.

8. The apparatus of claim 1, wherein said navigation function is configured to access a map associated with said line-based document, said map associating a plurality of line identifiers with a plurality of time codes.

9. The apparatus of claim 1, wherein said navigation function is configured to access a data channel associated with said audio data, said data channel comprising a plurality of line identifiers.

10. The apparatus of claim 9, wherein said data channel further comprises audio time codes.

11. A method for annotating a line-based document comprising lines of text, said lines of text being associated with corresponding line identifiers, said method comprising:

obtaining an audio data file comprising audio data representing said line-based document and a plurality of audio time codes associated with said audio data and with a plurality of said line identifiers;

detecting an audible document navigation command having a first line identifier, and navigating to a position of said audio data corresponding to said first line identifier;

detecting an audible annotation command at a second line identifier during playback of said audio data starting from said first line identifier, and capturing a corresponding audible annotation as an audio annotation file; and

adding an annotation entry to an index file, said annotation entry comprising a first reference to said audio annotation file and a second reference to said second line identifier.

12. The method of claim 11, further comprising:

detecting an audible bookmark command;

obtaining from said audio data file a current line identifier associated with the playback of said audio data when said audible bookmark command is detected; and

adding a bookmark entry in said index file, said bookmark entry comprising a third reference to said current line identifier.

13. The method of claim 11, further comprising:

detecting a speed command to change playback speed;

accessing a second audio data file corresponding to a target speed associated with said speed command, said second audio file being pre-generated by a text-to-speech converter at said target playback speed; and

playing back said second audio data file from a location corresponding to a most recent time code value.

14. The method of claim 11, further comprising using said index file to append one or more annotation links to one or more respective lines of said line-based document.

15. The method of claim 11, further comprising:

using voice recognition to generate one or more annotation text files corresponding to one or more respective audio annotation files.

16. The method of claim 15, further comprising:

providing a hypertext navigation document comprising links to said annotation text files and to said respective audio annotation files.

17. The method of claim 16, further comprising:

providing one or more bookmark links in said hypertext navigation document.

18. The method of claim 11, wherein obtaining said audio data file comprises:

obtaining a hypertext document comprising line identifiers;

using a text-to-speech engine to obtain said audio data file, wherein said text-to-speech engine records said line identifiers in association with time codes of said audio data file.

19. The method of claim 18, further comprising:

said text-to-speech engine rendering audio data for each line of said hypertext document based on one or more line-specific voice assignments.

20. The method of claim 19, wherein said line-specific voice assignments are embodied in one or more style settings of a word processing application.

21. A method for annotating a line-based document comprising lines of text, said lines of text being associated with corresponding line identifiers, said method comprising:

in a first computer system, pre-processing said line-based document to obtain an audio data file comprising audio data and a plurality of time codes associated with a plurality of said line identifiers;

providing said audio data file to a portable annotation device;

in said portable annotation device,

in response to line-based voice commands, navigating audio playback within said audio data file using said plurality of line identifiers in said audio data file;

in response to audible annotation commands, capturing an audio annotation file for each new user annotation, and providing an annotation entry in an index file, wherein said annotation entry comprises a first reference to a respective audio annotation file and a second reference to a line identifier in said audio data file;

providing a plurality of said annotation files and said index file to said first computer system;

in said first computer system,

using voice recognition to obtain a plurality of annotation text files corresponding to said plurality of annotation files;

for each annotation entry in said index file, appending to a corresponding line identifier in said line-based document one or more links to respective audio annotation files and corresponding annotation text files; and

providing a graphical navigation window comprising a plurality of links to said plurality of audio annotation files, a plurality of links to said annotation text files, and a plurality of links to line identifiers in said line-based document.

22. An apparatus for providing a line-based audio data file comprising:

a document server computer comprising a computer readable medium having computer program code embodied therein for providing line-based audio data, said computer program code comprising;

a text-to-speech convener configured to generate one or more audio data files from a line-based text document comprising lines of text, said lines of text being associated with corresponding line identifiers, each of said one or more audio data files comprising one or more channels of audio data and a parallel data channel comprising a plurality of audio time codes; and

a line identifier generator configured to obtain a plurality of said line identifiers from said line-based document and provide a plurality of respective line identifiers in said audio data file, said plurality of respective line identifiers being associated with said audio time codes.

23. The apparatus of claim 22, wherein said line identifier generator is configured to generate a map associating each of said plurality of respective line identifiers with a respective audio time code.

24. The apparatus of claim 22, wherein said line identifier generator is configured to add a line data channel to said audio data file in which said plurality of respective line identifiers are embedded.

25. The apparatus of claim 22, wherein said line identifier generator is configured to embed said plurality of respective line identifiers within said parallel data channel.

26. The apparatus of claim 22, wherein said computer program code further comprises a word processing application configured to associate one of a plurality of voices to individual lines within said line-based text document, wherein said text-to-speech converter is configured to use a respectively associated voice when converting each line of said line-based text document.

27. The apparatus of claim 26, wherein said word processing application comprises a plurality of styles by which said plurality of voices may be associated with said lines of said line-based text document.

28. The apparatus of claim 27, wherein said text-to-speech converter is configured to receive a voice preferences file associating one or more of said styles with one or more supported voices.

29. The apparatus of claim 22, wherein said text-to-speech converter is further configured to provide a plurality of sets of audio data corresponding respectively to said text-based document converted at a plurality of playback speeds.