Methods and apparatus relating to searching of spoken audio data

• US 8,209,171 B2
• Filed: 08/07/2008
• Issued: 06/26/2012
• Est. Priority Date: 08/07/2007
• Status: Active Grant

• Current Assignee: Arlington Technologies, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Aurix Ltd. (Avaya Incorporated)
• Inventors: Ponting, Keith M, Abbott, Martin G
• Primary Examiner(s): Abebe, Daniel D
• Application Number:
  US12/222,381
• Publication Number:
  US 20090043581A1
• Time in Patent Office: 1,419 Days
• Field of Search: 704/235, 704/251, 704/255, 369/27.01
• US Class Current: 704/235
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams

Abstract

This invention relates to a method of searching spoken audio data for one or more search terms comprising performing a phonetic search of the audio data to identify likely matches to a search term and producing textual data corresponding to a portion of the spoken audio data including a likely match. An embodiment of the method comprises the steps of taking phonetic index data corresponding to the spoken audio data, searching the phonetic index data for likely matches to the search term, wherein when a likely match is detected a portion of the spoken audio data or phonetic index data is selected which includes the likely match and said selected portion of the spoken audio data or phonetic index data is processed using a large vocabulary speech recognizer. The large vocabulary speech recognizer may derive textual data which can be used for further processing or may be used to present a transcript to a user. The present invention therefore combines the benefit of phonetic searching of audio data with the advantages of large vocabulary speech recognition.

Claims

1. A method of searching spoken audio data for a search term, said method comprising the steps of:

taking phonetic index data corresponding to the spoken audio data,

searching the phonetic index data for likely matches to the search term,

selecting a portion of the spoken audio data when a likely match is detected and,

processing said selected portion of the spoken audio data for producing textual data, wherein said searching and selecting steps are implemented on a phonetic audio searcher and said processing step is implemented on a large vocabulary speech recogniser.

2. A method as claimed in claim 1 wherein the phonetic index data comprises a score for some or all of a set of reference phones for each frame time of the spoken audio data.

3. A method as claimed in claim 1 further comprising a step of processing the spoken audio data to create phonetic index data.

4. A method as claimed in claim 2 wherein the step of searching the phonetic index data for likely matches to the search term comprises a dynamic programming search.

5. A method as claimed in claim 1 wherein the output of the large vocabulary speech recogniser comprises a confidence level for the likely match.

6. A method as claimed in claim 1 wherein the output of the large vocabulary speech recogniser comprises an indication of likely possible words corresponding to the selected portion of the spoken audio data or phonetic index data.

7. A method according to claim 1 wherein the output of the large vocabulary speech recogniser comprises a textual transcript of the selected portion of the spoken audio data or phonetic index data.

8. A method as claimed in claim 7 further comprising the step of displaying the textual transcript of the portion of the spoken audio data or phonetic index file corresponding to the likely match to the search term to a user.

9. A method as claimed in claim 1 further comprising the step of using automated analysis on the large vocabulary speech recogniser output.

10. A method as claimed in claim 1 wherein said selected portion of the spoken audio or phonetic index data which includes the likely match also includes periods of any spoken audio immediately before and after the likely match.

11. A method as claimed in claim 1 wherein said selected portion of the spoken audio or phonetic index data which includes the likely match and any periods of any spoken audio immediately before and after the likely match is determined by the large vocabulary speech recogniser.

12. A method according to claim 1 further comprising the step of extending the vocabulary of the large vocabulary speech recogniser to include all words in the search term.

13. A method as claimed in claim 1 wherein further processing is performed on the output of the large vocabulary speech recogniser.

14. A method as claimed in claim 13 wherein the output of the large vocabulary speech recogniser is searched for the or each search term to derive a confidence level for each possible match.

15. A method as claimed in claim 14 wherein the confidence level for each possible match combines a confidence measure from the search of the phonetic index data with a confidence measure from the large vocabulary speech recogniser.

16. A method as claimed in claim 13 wherein the further processing comprises searching for a search query comprising one or more search terms and outputting a confidence level for a match to the query as a whole.

17. A method as claimed in claim 13 wherein the further processing of the output of the large vocabulary speech recogniser comprises searching for at least one additional search term not used in the search of the phonetic index data.

18. A method as claimed in claim 13 wherein, after performing the further processing of the output of the large vocabulary speech recogniser, a textual transcription of the portion of the spoken audio data corresponding to the likely match is produced.

19. A method as claimed in claim 1 wherein the likelihood of match determined in the search of the phonetic index data is used in processing said selected portion of spoken audio data or phonetic index file with said large vocabulary speech recogniser.

20. A method of searching spoken audio data for one or more search terms comprising the steps of:

performing a phonetic search of the audio data to identify likely matches to a search term; and

producing textual data corresponding to a portion of the spoken audio data including a likely match, wherein said performing step is performed on a phonetic audio searcher and said producing step is performed on a large vocabulary speech recogniser.

21. A method as claimed in claim 20 wherein the textual data comprises a textual transcript.

22. A hybrid audio search engine comprising:

a phonetic search engine for identifying portions of spoken audio data as likely matches to one or more search terms; and

a large vocabulary search engine, wherein the large vocabulary search engine is configured to operate on said portions of spoken audio data.