Time ordered indexing of audio data

• US 20030088397A1
• Filed: 01/29/2002
• Published: 05/08/2003
• Est. Priority Date: 11/03/2001
• Status: Active Grant

• Current Assignee: Longsand Limited (Open Text Corporation)
• Original Assignee: Automony Systems, Limited
• Inventors: Karas, D. Matthew, Muldrew, William J.
• Granted Patent: US 7,292,979 B2
• Time in Patent Office: Days
• US Class Current: 704/1
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/489 using time information
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

Methods and apparatuses in which attributes including one or more types of accents and one or more types of human languages from an audio information stream are identified. Each identified attribute from the audio information stream is encoded into a time ordered index. Each of the identified attributes shares a common time reference. Different human language models are compared at approximately the same time to generate an integrated time ordered index.

Claims

1. A method, comprising:

identifying attributes including one or more types of accents and one or more types of human languages from an audio information stream;

encoding each identified attribute from the audio information stream into a time ordered index, each of the identified attributes share a common time reference; and

comparing different human language models at approximately the same time to generate an integrated time ordered index.

2. The method of claim 1, further comprising:

comparing confidence ratings of the different human language models.

3. The method of claim 1, further comprising:

generating a transcript including each spoken word, wherein each spoken word shares the common time reference.

4. The method of claim 1, further comprising:

triggering an event to occur upon an identification of unique voice characteristics of a speaker in less than five seconds.

5. A machine-readable medium that stores instructions, which when executed by a machine, cause the machine to perform operations comprising:

identifying attributes including one or more types of accents and one or more types of human languages from an audio information stream;

encoding each identified attribute from the audio information stream into a time ordered index, each of the identified attributes share a common time reference;

comparing different human language models at approximately the same time to generate an integrated time ordered index.

6. The article of manufacture of claim 5, further comprising instructions which cause the machine to perform further operations comprising:

generating a query on one or more of the identified attributes in the time ordered indexed.

7. The article of manufacture of claim 5, further comprising instructions which cause the machine to perform further operations comprising:

correlating a first identified attribute of the information stream with a second identified attribute having a similar time code.

8. The article of manufacture of claim 5, wherein the audio information stream comes from an unstructured information source.

9. The article of manufacture of claim 5, wherein the audio information stream includes audio-visual data.

10. The article of manufacture of claim 5, wherein the audio information stream includes speech data.

11. The article of manufacture of claim 5, wherein at least one of the identified attributes further comprises a change of accent.

12. The article of manufacture of claim 5, wherein at least one of the identified attributes further comprises a change of human language.

13. The article of manufacture of claim 5, wherein at least one of the identified attributes further comprises a discrete spoken word.

14. The article of manufacture of claim 5, wherein the identified attributes are encoded via extensible markup language.

15. The article of manufacture of claim 5, wherein the time ordered index includes a start time and a duration in which each identified attribute was conveyed.

16. The article of manufacture of claim 5, wherein the common time reference comprises a time indication.

17. The article of manufacture of claim 5, wherein the common time reference comprises a frame count.

18. The article of manufacture of claim 5, further comprising instructions which cause the machine to perform further operations comprising:

correlating a first identified attribute of the information stream with a second identified attribute having a similar time code.

19. The article of manufacture of claim 18, wherein the similar time code comprises the first identified attribute possessing a start time approximately the same as the second identified attribute or an overlapping of the durations associated with the first identified and the second identified attribute.

20. The article of manufacture of claim 5, wherein the integrated time ordered index includes data from the different human language models.

21. An apparatus, comprising:

means for identifying attributes including one or more types of accents and one or more types of human languages from an audio information stream;

means for encoding each identified attribute from the audio information stream into a time ordered index, each of the identified attributes share a common time reference; and

means for comparing different human language models at approximately the same time to generate an integrated time ordered index.

22. The apparatus of claim 21, further comprising:

means for generating a query on the one or more identified attributes in the time ordered indexed.

23. A machine-readable medium that stores instructions, which when executed by a machine, cause the machine to perform operations comprising:

converting spoken words in an information stream to written text, the information stream containing audio information; and

generating a separate encoded file for every word, wherein each encoded file shares a common time reference.

24. The article of manufacture of claim 23, further comprising instructions which cause the machine to perform further operations comprising:

generating a link to relevant material based upon the spoken words and synchronizing a display of the link in less than five seconds from analyzing the information stream.

25. An apparatus comprising:

a software engine having one or more attribute filters to detect attributes from an information stream, identify the attributes, and assign a time ordered indication with each of the identified attributes, the software engine having an index control module to facilitate an integrated time order indexing of the identified attributes; and

a computer readable medium to store the software engine.

26. The apparatus of claim 25, wherein the time ordered indication comprises a start time and a duration in which the attribute was conveyed.

27. The apparatus of claim 25, wherein the one or more attribute filters generate a time ordered index of the audio information stream in real time.

28. The apparatus of claim 25, wherein the audio information stream passes through the one or more attribute filters a single time.

29. The apparatus of claim 25, further comprising:

a manipulation module to perform operations on a first set of attributes in order to manipulate a second set of attributes.

30. The apparatus of claim 29, wherein the first set of attributes compromises a section of transcripted text and the second set of attributes comprises video images having approximately the same time ordered indications as the transcripted text.

31. The apparatus of claim 25, further comprising:

a triggering and synchronization module to dynamically trigger a link and synchronize the appearance of the link based upon a transcripted text from the information stream.