System and method for linking an audio stream with accompanying text material
First Claim
1. A computer implemented system for automatically associating audio from at least one audio source with text from at least one text document related to the audio, the system including a query system comprising:
- means or accessing at least one text document;
means for accessing at least one audio stream generated contemporaneously with a display of at least a portion of the text document;
means for automatically associating at least portions of the audio with at least respective portions of the text document;
means for permitting a user to specify at least one query word or phrase using a computer input device;
means for presenting in summary form, on a computer output device, at least some occurrences if any of the word or phrase in the text document and the audio stream;
means for extracting at least one of;
text, and keywords, from the audio along with timing information representative of the temporal location of at least some of the text and keywords in the audio; and
means for extracting at least one of;
text, and keywords, from the text document along with position information representative of the position of at least some of the text and keywords in the text document.
2 Assignments
0 Petitions
Accused Products
Abstract
A system enables a user to query for key words and phrases a text document, such as a presentation slide file, and an associated audio stream, such as can be derived from an audio-video recording that is made of a presenter contemporaneously with the showing of the slides to an audience. A graphical user interface is presented in which query results for both the text document and the audio stream are displayed in a time-aligned format, to enable a user to easily and conveniently browse the text document and accompanying time-aligned audio stream based on the key words/phrases.
-
Citations
40 Claims
-
1. A computer implemented system for automatically associating audio from at least one audio source with text from at least one text document related to the audio, the system including a query system comprising:
-
means or accessing at least one text document;
means for accessing at least one audio stream generated contemporaneously with a display of at least a portion of the text document;
means for automatically associating at least portions of the audio with at least respective portions of the text document;
means for permitting a user to specify at least one query word or phrase using a computer input device;
means for presenting in summary form, on a computer output device, at least some occurrences if any of the word or phrase in the text document and the audio stream;
means for extracting at least one of;
text, and keywords, from the audio along with timing information representative of the temporal location of at least some of the text and keywords in the audio; and
means for extracting at least one of;
text, and keywords, from the text document along with position information representative of the position of at least some of the text and keywords in the text document.- View Dependent Claims (2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14)
means for electronically storing the audio or a transcript thereof in a database;
means for electronically storing the text document in a database; and
means for receiving a user query, wherein the automatically associating means is executed before or after the query.
-
-
8. The system of claim 7, wherein the means for automatically associating includes means for associating at least a first portion of the text document with at least a first portion of the audio when both first portions include at least one key word in the user query.
-
9. The system of claim 7, wherein the associating means includes means for associating at least a first portion of the text document with at least a first portion of the audio when both first portions contain identical time stamps.
-
10. The system of claim 9, wherein the time stamps include at least one of:
- discrete times, and discrete time periods.
-
11. The system of claim 1, wherein the text document includes at least one presentation slide.
-
12. The system of claim 1, wherein the means for presenting in summary form presents, based on relevance to the query word or phrase, at least summaries of the text document and the audio stream.
-
13. The system of claim 1, wherein the audio stream is from a recording having both audio and video, the recording being generated contemporaneously with a display of at least a portion of the text document.
-
14. The system of claim 13, wherein the means for presenting presents in summary form at least portions of the video.
-
5. A computer implemented system for automatically associating audio from at least one audio source with text from at least one text document related to the audio, the system including a query system comprising:
-
means for accessing at least one text document;
mean for accessing at least one audio stream generated contemporaneously with a display of at least a portion of the text document;
means for automatically associating at least portions of the audio with at least respective portions of the text document;
means for permitting a user to specify at least one query word or phrase using a computer input device;
means for presenting in summary form, on a computer output device, at least some occurrences if any of the word or phrase in the text document and the audio stream; and
means for determining, for at least portions of the text document, information representative of times when the portions were presented on a large screen display. - View Dependent Claims (6)
means for determining, for at least portions of the text document, information representative of times when the portions were removed from a large screen display.
-
-
15. A computer system, comprising:
-
a data store holding at least one audio stream and at least one text source, the audio stream being based on audio associated with the text source; and
a processor receiving a query for data in the audio stream or text source and in response enabling a user to access at least portions of tile audio stream and text source or symbols representative of one or more thereof simultaneously, wherein the processor extracts at least one of;
text, and keywords, from the audio stream along with timing information representative of temporal location of at least some of the text and keywords in the audio stream, the processor further extracting at least one of;
text, and keywords, from the text source along with position information representative of the position of at least some of the text and keywords in the text source.- View Dependent Claims (16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27, 28, 29)
for at least portions of the text document, determining information representative of times when the portions were removed from a large screen display.
-
-
26. The computer program device of claim 25, wherein the method steps further comprise:
-
electronically storing the audio signal or a transcript thereof in a database;
electronically storing the text document in a database; and
receiving a user query, wherein the presenting step is undertaken before or after the query.
-
-
27. The computer program device of claim 26, wherein the presenting step is accomplished by associating at least a first portion of the text document with at least a first portion of the audio signal when both first portions include at least one key word in the user query.
-
28. The computer program device of claim 26, wherein the presenting step is accomplished by associating at least a first portion of the text document with at least a first portion of the audio signal when both first portions contain substantially identical time stamps.
-
29. The computer program device of claim 28, wherein the time stamps include at least one of:
- discrete times, and discrete time periods.
-
24. A computer program product comprising:
-
a computer program storage device readable by a computer; and
a program on the program storage device and including program code elements embodying instructions executable by the computer for performing method steps for associating at least portions of a text document with at least portions of an audio signal or transcript of the audio signal, the audio signal having been generated contemporaneously with a large screen display of the text document, the method steps comprising;
simultaneously presenting, via a computer output device, at least symbols representative of the text document and audio signal to a user, such that the user can navigate between the text document and audio signal; and
for at least portions of the text document, determining information representative of times when the portions were presented on a large screen display;
extracting at least one of text and keywords, from the audio signal along with timing information representative of the temporal location of at least some of the text and keywords in the audio signal, extracting at least one of text and keywords, from the text document along with position information representative of the position of at least some of the text and keywords in the text document, and wherein linking the audio with the text document is accomplished by associating at least a first portion of the text document with at least a first portion of the audio when both first portions include at least one key word in a user query. - View Dependent Claims (30, 31, 32, 39)
-
-
33. A computer-implemented method for associating audio from at least one audio source with at least one text document relating to the audio, the text document having been presented contemporaneously with the generation of the audio, comprising:
-
linking the audio with the text document;
associating at least portions of the audio with at least respective portions of the text document such that associated portions can be presented simultaneously on a computer output device; and
for at least portions of the text document, determining information representative of times when the portions were presented on a large screen display;
extracting at least one of text and keywords, from the audio signal along with timing information representative of the temporal location of at least some of the text and keywords in the audio signal, extracting at least one of text and keywords, from the text document along with position information representative of the position of at least some of the text and keywords in the text document, and wherein linking the audio with the text document is accomplished by associating at least a first portion of the text document with at least a first portion of the audio when both first portions include at least one key word in a user query. - View Dependent Claims (34, 35, 36, 37, 38, 40)
for at least portions of the text document, determining information representative of times when the portions were removed from a large screen display.
-
-
35. The method of claim 33, further comprising:
-
electronically storing the audio or a transcript thereof in a database;
electronically storing the text document in a database; and
receiving a user query, wherein the linking step is undertaken before or after the query.
-
-
36. The method of claim 35, wherein the linking step is accomplished by associating at least a first portion of the text document with at least a first portion of the audio when both first portions contain identical time stamps.
-
37. The method of claim 36, wherein the time stamps include at least one of:
- discrete times, and discrete time periods.
-
38. The method of claim 35, wherein the text document includes at least one presentation slide.
-
40. The computer-implemented method of claim 33, wherein the audio is from a recording having both audio and video, the recording being generated contemporaneously with a display of at least a portion of the text document, and further wherein the method steps include presenting, in summary form, at least portions of the video.
Specification