Time ordered indexing of audio data

• US 7,292,979 B2
• Filed: 01/29/2002
• Issued: 11/06/2007
• Est. Priority Date: 11/03/2001
• Status: Active Grant

• Current Assignee: Longsand Limited (Open Text Corporation)
• Original Assignee: Autonomy Systems Limited (Open Text Corporation)
• Inventors: Karas, D. Matthew, Muldrew, William J.
• Primary Examiner(s): Smits; Talivaldis Ivars
• Assistant Examiner(s): Shortledge; Thomas E.
• Application Number:
  US10/060,579
• Publication Number:
  US 20030088397A1
• Time in Patent Office: 2,107 Days
• Field of Search: 704/235, 704/243, 704/244, 704/246
• US Class Current: 704/244
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/489 using time information
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

Methods and apparatuses in which attributes including one or more types of accents and one or more types of human languages from an audio information stream are identified. Each identified attribute from the audio information stream is encoded into a time ordered index. Each of the identified attributes shares a common time reference. Different human language models are compared at approximately the same time to generate an integrated time ordered index.

Claims

1. A method, comprising:

identifying attributes including one or more types of accents and one or more types of human languages from a multi-party audio information stream;

encoding each identified attribute from the audio information stream into a time ordered index, each of the identified attributes sharing a common time reference;

comparing results from different human language models at approximately the same time to generate an integrated time ordered index of the identified attributes; and

triggering an event to occur upon an identification of unique voice characteristics of a speaker in less than five seconds.

2. The method of claim 1, further comprising:

comparing confidence ratings of the different human language models.

3. The method of claim 1, further comprising:

generating a transcript including each spoken word, wherein each spoken word shares the common time reference.

4. A machine-readable storage medium that stores instructions, which when executed by a machine, cause the machine to perform operations comprising:

identifying attributes including one or more types of accents and one or more types of human languages from a multi-party audio information stream;

encoding each identified attribute from the audio information stream into a time ordered index, each of the identified attributes sharing a common time reference;

comparing results from different human language models at approximately the same time to generate an integrated time ordered index of the identified attributes; and

correlating a first identified attribute of the information stream with a second identified attribute having a similar time code, wherein the similar time code comprises the first identified attribute possessing a start time approximately the same as the second identified attribute or an overlapping of the durations associated with the first identified and the second identified attribute.

5. The article of manufacture of claim 4, further comprising instructions which cause the machine to perform further operations comprising:

generating a query on one or more of the identified attributes in the time ordered indexed.

6. The article of manufacture of claim 5, further comprising instructions which cause the machine to perform further operations comprising:

correlating a first identified attribute of the information stream with a second identified attribute having a similar time code.

7. The article of manufacture of claim 4, wherein the audio information stream comes from an unstructured information source.

8. The article of manufacture of claim 4, wherein the audio information stream includes audio-visual data.

9. The article of manufacture of claim 4, wherein the audio information stream includes speech data.

10. The article of manufacture of claim 4, wherein at least one of the identified attributes further comprises a change of accent.

11. The article of manufacture of claim 4, wherein at least one of the identified attributes further comprises a change of human language.

12. The article of manufacture of claim 4, wherein at least one of the identified attributes further comprises a discrete spoken word.

13. The article of manufacture of claim 4, wherein the identified attributes are encoded via extensible markup language.

14. The article of manufacture of claim 4, wherein the time ordered index includes a start time and a duration in which each identified attribute was conveyed.

15. The article of manufacture of claim 4, wherein the common time reference comprises a time indication.

16. The article of manufacture of claim 4, wherein the common time reference comprises a frame count.

17. The article of manufacture of claim 4, wherein the integrated time ordered index includes data from the different human language models.

18. An apparatus, comprising:

means for identifying attributes including one or more types of accents and one or more types of human languages from a multi-party audio information stream;

means for encoding each identified attribute from the audio information stream into a time ordered index, each of the identified attributes sharing a common time reference;

means for comparing results from different human language models at approximately the same time to generate an integrated time ordered index of the identified attributes; and

means for correlating a first identified attribute of the information stream with a second identified attribute having a similar time code, wherein the similar time code comprises the first identified attribute possessing a start time approximately the same as the second identified attribute or an overlapping of the durations associated with the first identified and the second identified attribute.

19. The apparatus of claim 18, further comprising:

means for generating a query on the one or more identified attributes in the time ordered indexed.

20. A machine-readable storage medium that stores instructions, which when executed by a machine, cause the machine to perform operations comprising:

converting spoken words in an information stream to written text, the information stream containing audio information;

generating a separate encoded file for every word, wherein each encoded file shares a common time; and

generating a link to relevant material based upon the spoken words and synchronizing a display of the link in less than five seconds from analyzing the information stream.

21. An apparatus comprising:

a software engine having one or more attribute filters to detect attributes from a multi-party audio information stream, identify the attributes, and assign a time ordered indication with each of the identified attributes, the software engine having an index control module to facilitate an integrated time order indexing of the identified attributes

a computer readable storage medium to store the software engine; and

a manipulation module to perform operations on a first set of attributes in order to manipulate a second set of attributes.

22. The apparatus of claim 21, wherein the time ordered indication comprises a start time and a duration in which the attribute was conveyed.

23. The apparatus of claim 21, wherein the one or more attribute filters generate a time ordered index of the audio information stream in real time.

24. The apparatus of claim 21, wherein the audio information stream passes through the one or more attribute filters a single time.

25. The apparatus of claim 21, wherein the first set of attributes compromises a section of transcribed text and the second set of attributes comprises video images having approximately the same time ordered indications as the transcribed text.

26. The apparatus of claim 21, further comprising:

a triggering and synchronization module to dynamically trigger a link and synchronize the appearance of the link based upon a transcribed text from the information stream.

27. An apparatus comprising:

a software engine having one or more attribute filters to detect attributes from a multi-party audio information stream, identify the attributes, and assign a time ordered indication with each of the identified attributes, the software engine having an index control module to facilitate an integrated time order indexing of the identified attributes

a computer readable storage medium to store the software engine; and

a triggering and synchronization module to dynamically trigger a link and synchronize the appearance of the link based upon a transcribed text from the information stream.

28. The apparatus of claim 27, wherein the time ordered indication comprises a start time and a duration in which the attribute was conveyed.

29. The apparatus of claim 27, wherein the one or more attribute filters generate a time ordered index of the audio information stream in real time.

30. The apparatus of claim 27, wherein the audio information stream passes through the one or more attribute filters a single time.

31. The apparatus of claim 27, further comprising:

a manipulation module to perform operations on a first set of attributes in order to manipulate a second set of attributes.

32. The apparatus of claim 31, wherein the first set of attributes compromises a section of transcribed text and the second set of attributes comprises video images having approximately the same time ordered indications as the transcribed text.