Method and apparatus for improving human-machine dialogs using language models learned automatically from personalized data

• US 7,072,838 B1
• Filed: 03/20/2001
• Issued: 07/04/2006
• Est. Priority Date: 03/20/2001
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Ghosh, Debajit, Constantinides, Paul C.
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Opsasnick, Michael N.
• Application Number:
  US09/813,711
• Time in Patent Office: 1,932 Days
• Field of Search: 704/270.1, 704/275, 704/257, 704/277
• US Class Current: 704/257
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules

Abstract

A speech-based processing system includes a database of PIM data of a user, a set of language models, a learning unit, a recognition server, and a speech application. The learning unit uses a language model learning algorithm to provide language models based on the PIM data. The recognition server recognizes an utterance of the user by using one of the language models. The speech application identifies and accesses a subset of the PIM data specified by the utterance by using the recognition result. The language model learning algorithm may use grammar induction and/or or may train statistical language models based on the PIM data. The language model learning algorithm may be applied to generate language models periodically or on-the-fly during a session with the user.

Claims

1. A speech-based processing system comprising:

a database of personal information manager data associated with a user;

a set of language models for use by an automatic speech recognizer to recognize speech;

a learning unit to provide language models of said set of language models based on the PIM data by employing a language model learning algorithm;

a recognition server to recognize an utterance of the user by using one of the language models; and

a speech application to identify and access a subset of the personal information manager data specified by the utterance by using a result of recognizing the utterance.

2. A speech-based processing system as recited in claim 1, wherein the utterance comprises a short reference to said subset of the personal information manager data, said short reference consisting of less than all of said subset of the personal information manager data.

3. A speech-based processing system as recited in claim 1, wherein the learning unit is configured to employ the language model learning algorithm periodically to provide language models based on the personal information manager data.

4. A speech-based processing system as recited in claim 1, wherein the learning unit is configured to employ the language model learning algorithm on-the-fly to provide a language model based on the personal information manager data during a session with the user.

5. A speech-based processing system as recited in claim 1, wherein the language model learning algorithm uses grammar induction.

6. A speech-based processing system as recited in claim 1, wherein the language model learning algorithm trains statistical language models.

7. A speech-based processing system as recited in claim 1, wherein the speech application and the learning unit comply with a common set of APIs designed for accessing personal information manager data.

8. A speech-based processing system as recited in claim 1, wherein the personal information manger data is provided in one or more XML documents.

9. A speech-based processing system as recited in claim 1, wherein the personal information manager data is provided in one or more XML documents.

10. A speech-based processing system as recited in claim 1, wherein the learning unit comprises a plurality of modules, each module containing a set of heuristics tailored for acquiring language models for one of a plurality of types of personal information manager data.

11. A speech-based processing system as recited in claim 10, wherein the plurality of types of personal information manager data comprises personal address book information.

12. A speech-based processing system as recited in claim 10, wherein the plurality of types of personal information manager data comprises personal calendar information.

13. A speech-based processing system as recited in claim 10, wherein the plurality of types of personal information manger data comprises information from email messages of the user.

14. A speech-based processing system comprising:

a database of personal information manager data associated with a user;

a set of language models for use by an automatic speech recognizer to recognize speech;

means for employing a language model learning process to provide language models of said set of language models based on the personal information manager data;

means for recognizing an utterance of the user based on one of the language models, the utterance comprising a short reference to a subset of the personal information manager data; and

means for identifying and accessing the subset of the personal information manager data specified by the utterance based on a result of recognizing the utterance.

15. A speech-based processing system as recited in claim 14, wherein the means for employing a language model learning process comprises a plurality of modules, each module containing a set of heuristics tailored for acquiring language models for one of a plurality of types of personal information manager data.

16. A speech-based processing system as recited in claim 14, wherein the means for employing a language model learning process comprises means for periodically providing language models based on the personal information manager data.

17. A speech-based processing system as recited in claim 14, wherein the means for employing a language model learning process comprises means for providing a language model based on the personal information manager data on-the-fly during a session with the user.

18. A speech-based processing system as recited in claim 14, wherein the language model learning process uses grammar induction.

19. A speech-based processing system as recited in claim 14, wherein the language model learning process trains statistical language models.

20. A speech-based processing system comprising:

a database of personal information manager data associated with a user;

a set of language models for use by an automatic speech recognizer to recognize speech;

a language model server including

a learning unit to provide the set of language models based on the personal information manager data by employing a language model learning algorithm, and

a look-up unit to look-up and select one of the language models based on a specified identity of the user;

a recognition server to recognize an utterance of the user according to the selected language model; and

a speech application to trigger operation of the look-up unit and to identify and access a subset of the personal information manager data specified by the utterance using a result of recognizing the utterance.

21. A speech-based processing system as recited in claim 20, wherein the utterance comprises a short reference to said subset of the personal information manager data, said short reference consisting of less than all of said subset of the personal information manager data.

22. A speech-based processing system as recited in claim 20, wherein the learning unit comprises a plurality of modules, each module containing a set of heuristics tailored for one of a plurality of types of personal information manager data.

23. A speech-based processing system as recited in claim 20, wherein the learning unit is configured to employ the language model learning algorithm periodically to provide language models based on the personal information manager data.

24. A speech-based processing system as recited in claim 20, wherein the learning unit is configured to employ the language model learning algorithm on-the-fly to provide a language model based on the personal information manager data during a session with the user.

25. A speech-based processing system as recited in claim 20, wherein the language model learning algorithm uses grammar induction.

26. A speech-based processing system as recited in claim 20, wherein the language model learning algorithm trains statistical language models.

27. A speech-based processing system as recited in claim 20, wherein the speech application and the learning unit comply with a common set of APIs for accessing personal information manager data.

28. A speech-based processing system comprising:

processor means for executing software; and

storage means accessible to the processor means for storing software, the storage means having stored therein a learning unit to learn a set of language models for use by an automatic speech recognizer to recognize speech, based on a set of personal information manager data and a speech application to access a subset of the personal information manager data specified by a short reference to said subset uttered by a user, by using a result of recognizing the utterance.

29. A speech-based processing system as recited in claim 28, wherein the learning unit comprises a plurality of modules, each module containing a set of heuristics tailored for acquiring language models for one of a plurality of types of personal information manager data.

30. A method of facilitating speech recognition comprising:

using an automated language model learning process to acquire a set of language models for use by an automatic speech recognizer to recognize speech, based on personal information manager data associated with a user;

recognizing an utterance by the user by using one of the language models; and

using the recognized utterance of the user to identify and access a subset of the personal information manager data.

31. A speech-based processing system as recited in claim 30, wherein the utterance comprises a short reference to said subset of the personal information manager data, said short reference consisting of less than all of said subset of the personal information manager data.

32. A method as recited in claim 30, wherein said using an automated language model learning process comprises using a plurality of modules of the automated language model learning process to acquire the set of language models based on personal information manager data associated with the user, wherein each module contains a set of heuristics tailored for a particular type of personal information manager data.

33. A method as recited in claim 30, wherein said using an automated language model learning process comprises periodically providing language models based on the personal information manager data.

34. A method as recited in claim 30, wherein said using an automated language model learning process comprises providing a language model based on the personal information manager data on-the-fly during a session with the user.

35. A method as recited in claim 30, wherein the language model learning process uses grammar induction.

36. A method as recited in claim 30, wherein the language model learning process trains statistical language models.

37. A method of facilitating speech recognition comprising:

using an automated language model learning process to acquire a set of language models for use by an automatic speech recognizer to recognize speech, based on personal information manager data associated with a user;

recognizing an utterance by the user by using one of the language models; and

using a speech application to identify and access a subset of the personal information manager data based on the recognized utterance.

38. A method as recited in claim 37, wherein the utterance comprises a short reference to said subset of the personal information manager data, said short reference consisting of less than all of said subset of the personal information manager data.

39. A method as recited in claim 38, wherein said using an automated language model learning process comprises using a plurality of modules of the automated language model learning process to acquire the set of language models based on personal information manager data associated with the user, wherein each module contains a set of heuristics tailored for one of a plurality of types of personal information manager data.

40. A method as recited in claim 38, wherein said using an automated language model learning process comprises periodically providing language models based on the personal information manager data.

41. A method as recited in claim 38, wherein said using an automated language model learning process comprises providing a language model based on the personal information manger data on-the-fly during a session with the user.

42. A method as recited in claim 38, wherein the language model learning process uses grammar induction.

43. A method as recited in claim 38, wherein the language model learning process trains statistical language models.

44. A speech-based processing system as recited in claim 38, wherein the automated language model learning process and the speech application are each compliant with a set of APIs for accessing personal information manger data.

45. A speech-based processing system as recited in claim 38, wherein the personal information manager data is provided in one or more XML documents.