Method and apparatus for annotating a line-based document

• US 20060143559A1
• Filed: 08/18/2005
• Published: 06/29/2006
• Est. Priority Date: 03/09/2001
• Status: Active Grant

• Current Assignee: Copernicus Investments, LLC
• Original Assignee: Copernicus Investments, LLC
• Inventors: Spielberg, Steven, Gustman, Samuel
• Granted Patent: US 7,500,193 B2
• Time in Patent Office: Days
• US Class Current: 715/201
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

To facilitate the use of audio files for annotation purposes, an audio file format, which includes audio data for playback purposes, is augmented with a parallel data channel of line identifiers, or with a map associating time codes for the audio data with line numbers on the original document. The line number-time code information in the audio file is used to navigate within the audio file, and also to associate bookmark links and captured audio annotation files with line numbers of the original text document. An annotation device may provide an output document wherein links to audio and/or text annotation files are embedded at corresponding line numbers. Also, a navigation index may be generated, having links to annotation files and associated document line numbers, as well as bookmark links to selected document line numbers.

Claims

1. An apparatus for annotating a line-based document, wherein said line-based document comprises audio data, said apparatus comprising:

an audio codec coupled to an audio output device;

a voice recognition function coupled to an audio input, said voice recognition function configured to detect one or more audible document navigation commands and one or more audible annotation commands;

a navigation function responsive to a detected document navigation command received from said voice recognition function, said detected document navigation command comprising a desired line identifier, said navigation function configured to determine a desired audio time code associated with said desired line identifier and to direct said audio codec to play back said audio data from said desired audio time code;

an annotation function responsive to a detected annotation command received from said voice recognition function, said annotation function configured to capture an audible annotation via said audio input and to store said audible annotation as an audio annotation file; and

an index generator configured to add to an index file an annotation link having a first reference to said audio annotation file and a second reference to an associated line of said line-based document.

2. The apparatus of claim 1, wherein said navigation function is configured to track a current line number from a current time code of said audio data.

3. The apparatus of claim 2, wherein said index generator is configured to obtain said current line number to provide said second reference.

4. The apparatus of claim 2, further comprising:

a display; and

a browser configured to render hypertext data associated with said line-based document, said browser configured to render said hypertext data to said display.

5. The apparatus of claim 4, wherein said navigation function is configured to update said browser with said current line number as said audio codec plays back said audio data.

6. The apparatus of claim 2, further comprising:

a bookmark function responsive to a detected bookmark command received from said voice recognition function by storing said current line number as a bookmark line number.

7. The apparatus of claim 6, wherein said index generator is further configured to add to said index file a bookmark link having a third reference to said bookmark line number.

8. The apparatus of claim 1, wherein said navigation function is configured to access a map associated with said line-based document, said map associating a plurality of line identifiers with a plurality of time codes.

9. The apparatus of claim 1, wherein said navigation function is configured to access a data channel associated with said audio data, said data channel comprising a plurality of line identifiers.

10. The apparatus of claim 9, wherein said data channel further comprises audio time codes.

11. A method for annotating a line-based document comprising:

obtaining an audio data file comprising audio data of an audio representation of said line-based document, a plurality of audio time codes and a plurality of line identifiers associated with said plurality of audio time codes;

detecting an audible document navigation command having a specified line identifier, and navigating to a desired position of said audio data based on said specified line identifier;

detecting an audible annotation command during playback of a first line number, and capturing a corresponding audible annotation as an audio annotation file; and

adding an annotation entry to an index file, said annotation entry comprising a first reference to said audio annotation file and a second reference to said first line number.

12. The method of claim 11, further comprising:

detecting an audible bookmark command;

obtaining from said audio data file a current line identifier associated with the playback of said audio data when said audible bookmark command is detected; and

adding a bookmark entry in said index file, said bookmark entry comprising a third reference to said current line identifier.

13. The method of claim 11, further comprising:

detecting a speed command to change playback speed;

accessing a second audio data file corresponding to a target speed associated with said speed command, said second audio file being pre-generated by a text-to-speech converter at said target playback speed; and

playing back said second audio data file from a location corresponding to a most recent time code value.

14. The method of claim 11, further comprising using said index file to append one or more annotation links to one or more respective lines of a text-based version of said line-based document.

15. The method of claim 11, further comprising:

using voice recognition to generate one or more annotation text files corresponding to one or more respective audio annotation files.

16. The method of claim 15, further comprising:

providing a hypertext navigation document comprising links to said annotation text files and to said respective audio annotation files.

17. The method of claim 16, further comprising:

providing one or more bookmark links in said hypertext navigation document.

18. The method of claim 11, wherein obtaining said audio data file comprises:

obtaining a hypertext document comprising line numbers;

using a text-to-speech engine to obtain said audio data file, wherein said text-to-speech engine records line identifiers in association with time codes of said audio data file.

19. The method of claim 18, further comprising:

said text-to-speech engine rendering audio data for each line of said hypertext document based on one or more line-specific voice assignments.

20. The method of claim 19, wherein said line-specific voice assignments are embodied in one or more style settings of a word processing application.

21. A method for annotating a line-based document comprising:

in a first computer system, pre-processing a line-based document to obtain an audio data file comprising audio data, a plurality of time codes and a plurality of line identifiers associated with said plurality of time codes;

providing said audio data file to a portable annotation device;

in said portable annotation device,

in response to line-based voice commands, navigating audio playback within said audio data file using said plurality of line identifiers in said audio data file;

in response to audible annotation commands, capturing an audio annotation file for each new user annotation, and providing an annotation entry in an index file, wherein said annotation entry comprises a first reference to a respective audio annotation file and a second reference to a line identifier in said audio data file;

providing a plurality of said annotation files and said index file to said first computer system;

in said first computer system,

using voice recognition to obtain a plurality of annotation text files corresponding to said plurality of annotation files;

for each annotation entry in said index file, appending to a corresponding line number in said line-based document one or more links to respective audio annotation files and corresponding annotation text files; and

providing a graphical navigation window comprising a plurality of links to said plurality of audio annotation files, a plurality of links to said annotation text files, and a plurality of links to line numbers in said line-based document.

22. An apparatus for providing a line-based audio data file comprising:

a text-to-speech converter configured to generate one or more audio data files from a line-based text document, said audio data files each comprising one or more channels of audio data and a parallel data channel comprising a plurality of audio time codes; and

a line identifier generator configured to obtain a plurality of line numbers from said line-based document and provide a plurality of respective line identifiers in said audio data file, said plurality of line identifiers being associated with said audio time codes.

23. The apparatus of claim 22, wherein said line identifier generator is configured to generate a map associating each of said plurality of line identifiers with a respective audio time code.

24. The apparatus of claim 22, wherein said line identifier generator is configured to add a line data channel to said audio data file in which said plurality of line identifiers are embedded.

25. The apparatus of claim 22, wherein said line identifier generator is configured to embed said plurality of line identifiers within said parallel data channel.

26. The apparatus of claim 22, further comprising a word processing application configured to associate one of a plurality of voices to individual lines within said line-based text document, wherein said text-to-speech converter is configured to use a respectively associated voice when converting each line of said line-based text document.

27. The apparatus of claim 26, wherein said word processing application comprises a plurality of styles by which said plurality of voices may be associated with said lines of said line-based text document.

28. The apparatus of claim 27, wherein said text-to-speech converter is configured to receive a voice preferences file associating one or more of said styles with one or more supported voices.

29. The apparatus of claim 22, wherein said text-to-speech converter is further configured to provide a plurality of sets of audio data corresponding respectively to said text-based document converted at a plurality of playback speeds.