Methods and apparatus relating to searching of spoken audio data

• US 20090043581A1
• Filed: 08/07/2008
• Published: 02/12/2009
• Est. Priority Date: 08/07/2007
• Status: Active Grant

• Current Assignee: Arlington Technologies, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Aurix Ltd. (Avaya Incorporated)
• Inventors: Ponting, Keith M., Abbott, Martin G.
• Granted Patent: US 8,209,171 B2
• Time in Patent Office: Days
• US Class Current: 704/254
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams

Abstract

This invention relates to a method of searching spoken audio data for one or more search terms comprising performing a phonetic search of the audio data to identify likely matches to a search term and producing textual data corresponding to a portion of the spoken audio data including a likely match. An embodiment of the method comprises the steps of taking phonetic index data corresponding to the spoken audio data, searching the phonetic index data for likely matches to the search term, wherein when a likely match is detected a portion of the spoken audio data or phonetic index data is selected which includes the likely match and said selected portion of the spoken audio data or phonetic index data is processed using a large vocabulary speech recogniser. The large vocabulary speech recogniser may derive textual data which can be used for further processing or may be used to present a transcript to a user. The present invention therefore combines the benefit of phonetic searching of audio data with the advantages of large vocabulary speech recognition.

Claims

1. A method of searching spoken audio data for a search term comprising the steps of taking phonetic index data corresponding to the spoken audio data, searching the phonetic index data for likely matches to the search term, wherein when a likely match is detected a portion of the spoken audio data or phonetic index data is selected which includes the likely match and said selected portion of the spoken audio data or phonetic index data is processed using a large vocabulary speech recogniser.

2. A method as claimed in claim 1 wherein the phonetic index data comprises a score for some or all of a set of reference phones for each frame time of the spoken audio data.

3. A method as claimed in claim 1 further comprising a step of processing the spoken audio data to create phonetic index data.

4. A method as claimed in claim 2 wherein the step of searching the phonetic index data for likely matches to the search term comprises a dynamic programming search.

5. A method as claimed in claim 1 wherein the output of the large vocabulary speech recogniser comprises a confidence level for the likely match.

6. A method as claimed in claim 1 wherein the output of the large vocabulary speech recogniser comprises an indication of likely possible words corresponding to the selected portion of the spoken audio data or phonetic index data.

7. A method according to claim 1 wherein the output of the large vocabulary speech recogniser comprises a textual transcript of the selected portion of the spoken audio data or phonetic index data.

8. A method as claimed in claim 7 further comprising the step of displaying the textual transcript of the portion of the spoken audio data or phonetic index file corresponding to the likely match to the search term to a user.

9. A method as claimed in claim 1 further comprising the step of using automated analysis on the large vocabulary speech recogniser output.

10. A method as claimed in claim 1 wherein said selected portion of the spoken audio or phonetic index data which includes the likely match also includes periods of any spoken audio immediately before and after the likely match.

11. A method as claimed in claim 1 wherein said selected portion of the spoken audio or phonetic index data which includes the likely match and any periods of any spoken audio immediately before and after the likely match is determined by the large vocabulary speech recogniser.

12. A method according to claim 1 further comprising the step of extending the vocabulary of the large vocabulary speech recogniser to include all words in the or each search term.

13. A method as claimed in claim 1 wherein further processing is performed on the output of the large vocabulary speech recogniser.

14. A method as claimed in claim 13 wherein the output of the large vocabulary speech recogniser is searched for the or each search term to derive a confidence level for each possible match.

15. A method as claimed in claim 14 wherein the confidence level for each possible match combines a confidence measure from the search of the phonetic index data with a confidence measure from the large vocabulary speech recogniser.

16. A method as claimed in claim 13 wherein the further processing comprises searching for a search query comprising one or more search terms and outputting a confidence level for a match to the query as a whole.

17. A method as claimed in claim 13 wherein the further processing of the output of the large vocabulary speech recogniser comprises searching for at least one additional search term not used in the search of the phonetic index data.

18. A method as claimed in claim 13 wherein, after performing the further processing of the output of the large vocabulary speech recogniser, a textual transcription of the portion of the spoken audio data corresponding to the likely match is produced.

19. A method as claimed in claim 1 wherein the likelihood of match determined in the search of the phonetic index data is used in processing said selected portion of spoken audio data or phonetic index file with said large vocabulary speech recogniser.

20. A method of searching spoken audio data for one or more search terms comprising performing a phonetic search of the audio data to identify likely matches to a search term and producing textual data corresponding to a portion of the spoken audio data including a likely match.

21. A method as claimed in claim 20 wherein the textual data comprises a textual transcript.

22. A method of processing spoken audio data comprising the steps of using a phonetic search engine to identify possible matches to at least one search term and using a large vocabulary speech engine on a portion of the spoken audio data including a likely match.

23. A method as claimed in claim 22 wherein the output of the large vocabulary speech engine is used for automated analysis of the spoken audio data.

24. A hybrid audio search engine comprising a phonetic search engine and a large vocabulary search engine wherein the large vocabulary search engine is adapted to operate on portions of spoken audio data identified by the phonetic search engine as likely matches to one or more search terms.