Method and apparatus for speech recognition using a dynamic vocabulary

• US 20050055210A1
• Filed: 08/05/2004
• Published: 03/10/2005
• Est. Priority Date: 09/28/2001
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: SRI International, Inc.
• Inventors: Bercow, Douglas A., Venkataraman, Anand, Franco, Horacio E.
• Granted Patent: US 7,308,404 B2
• Time in Patent Office: Days
• US Class Current: 704/255
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models

Abstract

A method and apparatus are provided for performing speech recognition using a dynamic vocabulary. Results from a preliminary speech recognition pass can be used to update or refine a language model in order to improve the accuracy of search results and to simplify subsequent recognition passes. This iterative process greatly reduces the number of alternative hypotheses produced during each speech recognition pass, as well as the time required to process subsequent passes, making the speech recognition process faster, more efficient and more accurate. The iterative process is characterized by the use of results from one or more data set queries, where the keys used to query the data set, as well as the queries themselves, are constructed in a manner that produces more effective language models for use in subsequent attempts at decoding a given speech signal.

Claims

1. A method for decoding a spoken request for information, the method comprising the steps of:

receiving said spoken request from a user;

applying an initial language model to said spoken request to identify one or more words contained in said spoken request; and

generating a second language model that includes words in said spoken request that are not recognized by said application of said initial language model.

2. The method of claim 1, wherein said second language model is generated by updating said initial language model.

3. The method of claim 1, wherein said initial language model is biased to generate results tailored to a specified domain.

4. The method of claim 3, wherein said specified domain is at least one of music, movies, books, games, cellular phone ring tones, auction items, addresses, directory listings, library catalogs and retail catalogs.

5. The method of claim 1, where said second language model is implemented as a recognizer search graph embodying a plurality of possible ways in which said spoken request could be phrased.

6. The method of claim 5, wherein said search graph is implemented by encoding a plurality of parallel paths from a start of said search graph to an end of said search graph, each path representing one possible way in which said spoken request could be phrased.

7. The method of claim 1, wherein the step of applying an initial language model comprises:

generating one or more hypotheses representative of data items potentially corresponding to said spoken request;

computing confidence scores for each word appearing in each of said hypotheses; and

selecting one or more high-confidence words having confidence scores that at least meet a first predefined threshold.

8. The method of claim 7, wherein the step of computing confidence scores comprises:

combining said hypotheses into a wordgraph and computing posterior probability scores for each word at each position in said hypotheses.

9. The method of claim 7, further comprising the step of:

querying a data set using said high-confidence words, where said data set contains data items potentially corresponding to said spoken request.

10. The method of claim 9, further comprising the step of:

determining if a number of query results produced by querying said data set exceeds a second predefined threshold.

11. The method of claim 10, further comprising the step of:

querying said data set for a second time with a less restrictive set of high-confidence words if the number of query results produced does not exceed said second predefined threshold.

12. The method of claim 10, further comprising the step of:

querying said data set for a second time with multiple alternative sets of high-confidence words if the number of query results produced does not exceed said second predefined threshold.

13. The method of claim 10, wherein the step of generating a second language model comprises:

analyzing said query results to identify novel words not contained within said initial language model, if said number of query results exceeds said second predefined threshold;

constructing a pronunciation dictionary for word recognition using said second language model; and

adding said novel words to said pronunciation dictionary.

14. The method of claim 10, wherein the step of generating a second language model comprises:

constructing said second language model based on said query results, if said number of query results does not exceed said second predefined threshold.

15. The method of claim 14, further comprising the step of:

applying said second language model to said spoken request to identify one or more additional words contained in said spoken request.

16. The method of claim 15, wherein said step of applying said second language model comprises:

assigning probabilities to each data item in said second language model, where said probabilities represent the likelihood that a data item corresponds to said spoken request; and

returning one or more data items to said user based upon said assigned probabilities.

17. The method of claim 16, wherein said step of returning one or more data items comprises:

returning a data item having a highest assigned probability.

18. The method of claim 16, wherein said step of returning one or more data items comprises:

returning a plurality of data items, where each of said plurality of data items has an assigned probability that deviates from a highest probability by less than a predefined degree.

19. The method of claim 16, further comprising the step of:

downloading said one or more data items to a remote location.

20. The method of claim 17, wherein said one or more data items are downloaded to at least one of a desktop computer, a laptop computer, a personal digital assistant, a wristwatch, a portable music player, a car stereo, a hi-fi/entertainment center, a television, a digital video recorder, or a cable or satellite set stop box.

21. The method of claim 1, wherein the method is executed in its entirety at a single computing device.

22. The method of claim 1, wherein execution of two or more steps of the method is distributed among a plurality of processors.

23. A computer readable medium containing an executable program for decoding a spoken request for information, where the program performs the steps of:

receiving said spoken request from a user;

applying an initial language model to said spoken request to identify one or more words contained in said spoken request; and

generating a second language model that includes words in said spoken request that are not recognized by said application of said initial language model.

24. The computer readable medium of claim 23, wherein said second language model is generated by updating said initial language model.

25. The computer readable medium of claim 23, wherein said initial language model is biased to generate results tailored to a specified domain.

26. The computer readable medium of claim 25, wherein said specified domain is at least one of music, movies, books, games, cellular phone ring tones, auction items, addresses, directory listings, library catalogs and retail catalogs.

27. The computer readable medium of claim 23, wherein said second language model is implemented as a recognizer search graph embodying a plurality of possible ways in which said spoken request could be phrased.

28. The computer readable medium of claim 27, wherein said search graph is implemented by encoding a plurality of parallel paths from a start of said search graph to an end of said search graph, each path representing one possible way in which said spoken request could be phrased.

29. The computer readable medium of claim 23, wherein the step of applying an initial language model comprises:

generating one or more hypotheses representative of data items potentially corresponding to said spoken request;

computing confidence scores for each word appearing in each of said hypotheses; and

selecting one or more high-confidence words having confidence scores that at least meet a first predefined threshold.

30. The computer readable medium of claim 29, wherein the step of computing confidence scores comprises:

combining said hypotheses into a wordgraph and computing posterior probability scores for each word at each position in said hypotheses.

31. The computer readable medium of claim 29, further comprising the step of:

querying a data set using said high-confidence words, where said data set contains data items potentially corresponding to said spoken request.

32. The computer readable medium of claim 31, further comprising the step of:

determining if a number of query results produced by querying said data set exceeds a second predefined threshold.

33. The computer readable medium of claim 32, further comprising the step of:

querying said data set for a second time with a less restrictive set of high-confidence words if the number of query results produced does not exceed said second predefined threshold.

34. The computer readable medium of claim 32, further comprising the step of:

querying said data set for a second time with multiple alternative sets of high-confidence words if the number of query results produced does not exceed said second predefined threshold.

35. The computer readable medium of claim 32, wherein the step of generating a second language model comprises:

analyzing said query results to identify novel words not contained within said initial language model, if said number of query results exceeds said second predefined threshold;

constructing a pronunciation dictionary for word recognition using said second language model; and

adding said novel words to said pronunciation dictionary.

36. The computer readable medium of claim 32, wherein the step of generating a second language model comprises:

constructing said second language model based on said query results, if said number of query results does not exceed said second predefined threshold.

37. The computer readable medium of claim 36, further comprising the step of:

applying said second language model to said spoken request to identify one or more additional words contained in said spoken request.

38. The computer readable medium of claim 37, wherein said step of applying said second language model comprises:

assigning probabilities to each data item in said second language model, where said probabilities represent the likelihood that a data item corresponds to said spoken request; and

returning one or more data items to said user based upon said assigned probabilities.

39. The computer readable medium of claim 38, wherein said step of returning one or more data items comprises:

returning a data item having a highest assigned probability.

40. The computer readable medium of claim 38, wherein said step of returning one or more data items comprises:

returning a plurality of data items, where each of said plurality of data items has an assigned probability that deviates from a highest probability by less than a predefined degree.

41. The computer readable medium of claim 37, further comprising the step of:

downloading said one or more data items to a remote location.

42. The computer readable medium of claim 41, wherein said one or more data items are downloaded to at least one of a desktop computer, a laptop computer, a personal digital assistant, a wristwatch, a portable music player, a car stereo, a hi-fi/entertainment center, a television, a digital video recorder, or a cable or satellite set stop box.

43. The computer readable medium of claim 23, wherein the method is executed in its entirety at a single computing device.

44. The computer readable medium of claim 23, wherein execution of two or more steps of the method is distributed among a plurality of processors.

45. Apparatus for decoding a spoken request for information, the apparatus comprising:

means for receiving said spoken request from a user;

means for applying an initial language model to said spoken request to identify one or more words contained in said spoken request; and

means for generating a second language model that includes words in said spoken request that are not recognized by said application of said initial language model.

46. The apparatus of claim 45, wherein said generating means is configured for generating said second language model by updating said initial language model.

47. The apparatus of claim 45, further comprising:

means for downloading one or more data items corresponding to said spoken request to a remote location.

48. The apparatus of claim 45, wherein said means for receiving, means for applying and means for generating are physically disposed in a single device.

49. The apparatus of claim 45, wherein two or more of said means for receiving, means for applying and means generating are separate processors that communicate over a network.