Method and apparatus for retrieving audio information using primary and supplemental indexes

• US 6,345,253 B1
• Filed: 06/18/1999
• Issued: 02/05/2002
• Est. Priority Date: 04/09/1999
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Viswanathan, Mahesh
• Primary Examiner(s): Dorvil, Richemond
• Application Number:
  US09/335,627
• Time in Patent Office: 963 Days
• Field of Search: 704/231, 704/236, 704/238, 704/250, 704/251, 704/255, 704/260, 704/200, 704/270, 704/275, 704/272
• US Class Current: 704/272
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques

Abstract

An audio retrieval system and method are provided for augmenting the transcription of an audio file with one or more alternate word or phrase choices, such as next-best guesses for each word or phrase, in addition to the best word sequence identified by the transcription process. The audio retrieval system can utilize a primary index file containing the best identified words and/or phrases for each portion of the input audio stream and a supplemental index file containing alternative choices for each word or phrase in the transcript. The present invention allows words that are incorrectly transcribed during speech recognition to be identified in response to a textual query by searching the supplemental index files. During an indexing process, the list of alternative word or phrase choices provided by the speech recognition system are collected to produce a set of supplemental index files. During a retrieval process, the user-specified textual query is matched against the primary and supplemental indexes derived from the transcribed audio to identify relevant documents. An objective ranking function scales matches found in the supplemental index file(s) using a predefined scaling factor, or a value reflecting the confidence value of the corresponding alternative choice as identified by the speech recognition system.

Claims

1. A method for retrieving audio information from one or more audio sources, said method comprising the steps of:

receiving a user query specifying at least one key word; and

comparing said user query with a primary index of best choices for words in said audio source and a supplemental index of at least one alternate choice for said words in said audio source to identify audio information satisfying said user query.

2. The method of claim 1, wherein said primary and supplemental indexes are time-stamped.

3. The method of claim 1, wherein said primary and supplemental indexes include the frequency of each word in said audio source.

4. The method of claim 1, wherein said primary and supplemental indexes include the inverse document frequency (IDF) of each word in said audio source.

5. The method of claim 1, wherein said primary and supplemental indexes include the length of said audio source.

6. The method of claim 1, wherein said primary and supplemental indexes include a set of chain linked pointers to each document containing a given word.

7. The method of claim 1, wherein said comparing step further comprises the step of comparing said at least one key word to words in said primary and supplemental indexes to identify relevant documents.

8. The method of claim 1, further comprising the step of transcribing and indexing said audio source to create said primary and supplemental indexes.

9. The method of claim 1, further comprising the step of returning at least a portion of said identified audio information to a user.

10. The method of claim 1, further comprising the step of assigning a score to each segment of said identified audio information and returning at least a portion of said identified audio information in a ranked-list.

11. The method of claim 10, wherein said score evaluates a ranked document score ranking the content-based information retrieval.

12. The method of claim 10, wherein said score is scaled using a predefined scaling factor for documents matching said supplemental index.

13. The method of claim 10, wherein said score is scaled using a value reflecting the confidence value of the corresponding alternative choice for documents matching said supplemental index.

14. An audio retrieval system for retrieving audio information from one or more audio sources, comprising:

a memory that stores a primary index of best choices for words in said audio source and a supplemental index of at least one alternate choice for said words in said audio source and computer-readable code; and

a processor operatively coupled to said memory, said processor configured to implement said computer-readable code, said computer-readable code configured to;

receive a user query specifying one or more words; and

compare said user query with said primary and supplemental indexes to identify audio information satisfying said user query.

15. The audio retrieval system of claim 14, wherein said primary and supplemental indexes are time-stamped.

16. The audio retrieval system of claim 14, wherein said primary and supplemental indexes include the frequency of each word in said audio source.

17. The audio retrieval system of claim 14, wherein said primary and supplemental indexes include the inverse document frequency (IDF) of each word in said audio source.

18. The audio retrieval system of claim 14, wherein said primary and supplemental indexes include the length of said audio source.

19. The audio retrieval system of claim 14, wherein said primary and supplemental indexes include a set of chain linked pointers to each document containing a given word.

20. The audio retrieval system of claim 14, wherein said processor is further configured to compare said at least one key word to words in said primary and supplemental indexes to identify relevant documents.

21. The audio retrieval system of claim 14, wherein said processor is further configured to transcribe and index said audio source to create said primary and supplemental indexes.

22. The audio retrieval system of claim 14, wherein said processor is further configured to transcribe and index said audio source to create said primary and supplemental indexes.

23. The audio retrieval system of claim 14, wherein said processor is further configured to return at least a portion of said identified audio information to a user.

24. The audio retrieval system of claim 14, wherein said processor is further configured to assign a score to each segment of said identified audio information and returning at least a portion of said identified audio information in a ranked-list.

25. The audio retrieval system of claim 24, wherein said score evaluates a ranked document score ranking the content-based information retrieval.

26. The audio retrieval system of claim 24, wherein said score is scaled using a predefined scaling factor for documents matching said supplemental index.

27. The audio retrieval system of claim 24, wherein said score is scaled using a value reflecting the confidence value of the corresponding alternative choice for documents matching said supplemental index.

28. An article of manufacture for retrieving audio information from one or more audio sources, comprising:

a computer readable medium having computer readable code means embodied thereon, said computer readable program code means comprising;

a step to receive a user query specifying one or more words; and

a step to compare said user query with a primary index of best choices for words in said audio source and a supplemental index of at least one alternate choice for said words in said audio source to identify audio information satisfying said user query.