Method and apparatus for providing a mixed-initiative dialog between a user and a machine

• US 20040085162A1
• Filed: 11/29/2000
• Published: 05/06/2004
• Est. Priority Date: 11/29/2000
• Status: Abandoned Application

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Agarwal, Rajeev, Shahshahani, Behzad M.
• Application Number:
  US09/727,022
• Publication Number:
  US 20040085162A1
• Time in Patent Office: Days
• US Class Current: 333/196
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A method and apparatus for enabling a mixed initiative dialog to be carried out between a user and a machine are described. A speech-enabled processing system receives an utterance from the user, and the utterance is recognized by an automatic speech recognizer using a set of statistical language models. Prior to parsing the utterance, a dialog manager uses a semantic frame to identify the set of all slots potentially associated with the current task and then retrieves a corresponding grammar for each of the identified slots from an associated reusable dialog component. A natural language parser then parses the utterance using the recognized speech and all of the retrieved grammars. The dialog manager then identifies any slot which remains unfilled after parsing and causes a prompt to be played to the user for information to fill the unfilled slot. Dependencies and constraints may be associated with particular slots.

Claims

1. A method of enabling a mixed initiative dialog to be carried out between a user and a machine, the method comprising:

providing a set of reusable dialog components; and

operating a dialog manager to control use of the reusable dialog components based on a semantic frame, wherein the reusable dialog components are individually configured to carry out system initiated aspects of a dialog.

2. A method as recited in claim 1, wherein the reusable dialog components are configured to perform disambiguation and confirmation actions specific to semantic slots associated with a current task, such that the dialog manager does not perform said disambiguation and confirmation actions.

3. A method as recited in claim 1, wherein the semantic frame contains a map of tasks to corresponding semantic slots.

4. A method as recited in claim 1, wherein said operating the dialog manager comprises:

(a) parsing an utterance using grammars from the set of reusable dialog components;

(b) after said parsing, using a prompt from one of the reusable dialog components to request information from the user to fill an unfilled slot; and

(c) automatically repeating said (b), if necessary, to fill any additional unfilled slots associated with the current task.

5. A method of enabling a mixed initiative dialog to be carried out between a user and a machine, the method comprising:

(a) receiving speech from the user, the speech representing an utterance;

(b) recognizing the utterance;

(c) identifying the set of all slots potentially associated with a current task; and

(d) using a set of reusable dialog components corresponding to said set of slots to fill the slots associated with the current task, including

(d)(1) parsing the utterance using grammars from the set of reusable dialog components, and

(d)(2) after said parsing, using a prompt from one of the reusable dialog components to request information from the user to fill an unfilled slot.

6. A method as recited in claim 5, further comprising automatically repeating said (d)(2), as necessary, to fill additional unfilled slots associated with the current task.

7. A method as recited in claim 5, wherein each of the slots represents an item of information which may be acquired from the user.

8. A method as recited in claim 5, wherein said identifying the set of all slots potentially associated with a current task is carried out prior to said parsing the utterance.

9. A method as recited in claim 5, wherein said parsing the utterance comprises filling one or more of the possible slots with corresponding values.

10. A method as recited in claim 5, wherein said identifying the set of all slots potentially associated with a current task comprises using a semantic frame that maps tasks performable in response to speech from the user to corresponding slots, to identify the set of all slots potentially associated with the current task.

11. A method as recited in claim 5, wherein each of the reusable dialog components is a speech object embodying an instantiation of a speech object class.

12. A method as recited in claim 5, wherein said recognizing comprises using a set of statistical language models so as to be capable of recognizing open-ended speech.

13. A method as recited in claim 12, wherein at least one of the statistical language models is specifically adapted for a most-recently played prompt.

14. A method as recited in claim 5, wherein a dependency exists between two or more of the slots.

15. A method as recited in claim 14, further comprising identifying a dependency between two of the slots, wherein said parsing the utterance comprises filling one of the slots based on the dependency and a value used to fill another slot.

16. A method as recited in claim 5, wherein the dialog is for accomplishing a task, and wherein the method further comprises confirming and correcting slots filled during the dialog, including:

determining that one of the slots is incorrect;

prompting the user for a corrected value for the slot;

receiving the corrected value from the user; and

using the corrected value and stored information on dependencies between the slots to control further dialog for accomplishing the task.

17. A method of enabling a mixed initiative dialog to be carried out between a user and a machine, the method comprising:

(a) receiving speech from the user, the speech representing an utterance;

(b) recognizing the utterance;

(c) identifying the set of all slots potentially associated with a current task;

(d) retrieving a corresponding grammar for each of the identified slots from one of a plurality of reusable dialog components;

(e) parsing the utterance using the recognized speech and the retrieved grammars. (f) identifying one of the slots which remains unfilled after parsing the utterance;

(g) obtaining a prompt for said slot which remains unfilled from a corresponding one of the reusable dialog components;

(h) playing the prompt to the user; and

(i) repeating said (a), (b), (e), (f), (g) and (h) so as to fill all of the slots associated with the current task.

18. A method as recited in claim 17, wherein each of the slots represents an item of information which may be acquired from the user.

19. A method as recited in claim 17, wherein said identifying the set of all slots potentially associated with a current task is carried out prior to said parsing the utterance.

20. A method as recited in claim 17, wherein said parsing the utterance comprises filling one or more of the possible slots with corresponding values.

21. A method as recited in claim 17, wherein said identifying the set of all slots potentially associated with a current task comprises using a mapping of tasks performable in response to speech from the user to corresponding slots, to identify the set of all slots potentially associated with the current task.

22. A method as recited in claim 17, wherein each of the reusable dialog components is a speech object embodying an instantiation of a speech object class.

23. A method as recited in claim 17, wherein said recognizing comprises using a set of statistical language models so as to be capable of recognizing open-ended speech.

24. A method as recited in claim 23, wherein at least one of the statistical language models is specifically adapted for a most-recently played prompt.

25. A method as recited in claim 17, wherein a dependency exists between two or more of the slots.

26. A method as recited in claim 17, further comprising identifying a dependency between two of the slots, wherein said parsing the utterance comprises filling one of the slots based on the dependency and a value used to fill another slot.

27. A method as recited in claim 17, wherein the dialog is for accomplishing a task, and wherein the method further comprises confirming and correcting slots filled during the dialog, including:

determining that one of the slots is incorrect;

prompting the user for a corrected value for the slot;

receiving the corrected value from the user; and

using the corrected value and stored information on dependencies between the slots to control further dialog for accomplishing the task.

28. A method of carrying out a mixed initiative dialog between a user and a machine, the method comprising:

receiving speech from the user, the speech representing an utterance;

recognizing the utterance using an automatic speech recognizer;

identifying the set of all slots potentially associated with a current task prior to parsing the utterance, each slot representing an item of information which may be acquired from the user;

for each of the possible slots, retrieving a corresponding grammar from a corresponding one of a plurality of reusable dialog components;

using the recognized speech and the retrieved grammars to parse the utterance, including filling one or more of the possible slots with corresponding values;

identifying one of the slots which remains unfilled;

accessing a prompt for the slot which remains unfilled from a corresponding one of the reusable dialog components; and

playing the prompt to the user.

29. A method as recited in claim 28, wherein a plurality of tasks may be performed in response to speech from the user, and wherein said identifying the set of all slots potentially associated with a current task comprises using a semantic frame which includes a mapping of tasks to slots to identify the set of all slots potentially associated with the current task.

30. A method as recited in claim 29, wherein of the reusable dialog components is an instantiation of a speech object class.

31. A method as recited in claim 28, wherein said recognizing comprises using a set of statistical language models so as to be capable of recognizing open-ended speech.

32. A method as recited in claim 31, wherein at least one of the statistical language models is specifically adapted for a most-recently played prompt.

33. A method as recited in claim 28, wherein a dependency exists between two or more of the slots.

34. A method as recited in claim 33, further comprising:

identifying a dependency between two of the slots; and

filling one of the slots based on the dependency and a value used to fill another slot.

35. A method as recited in claim 28, wherein the dialog is for accomplishing a task, and wherein the method further comprises confirming and correcting slots filled during the dialog, including:

determining that one of the slots is incorrect;

prompting the user for a corrected value for the slot;

receiving the corrected value from the user; and

using the corrected value and stored information on dependencies between the slots to control further dialog for accomplishing the task.

36. An apparatus for enabling a mixed initiative dialog to be carried out between a user and a machine, the apparatus comprising:

means for receiving speech from the user, the speech representing an utterance;

means for recognizing the utterance;

means for identifying the set of all slots potentially associated with a current task; and

means for using a set of reusable dialog components corresponding to said set of slots to fill the slots associated with the current task, including

means for parsing the utterance using grammars from the set of reusable dialog components, and

means for using, after said parsing, a prompt from one of the reusable dialog components to request information from the user to fill an unfilled slot.

37. An apparatus as recited in claim 36, further comprising means for automatically repeating said using a prompt from one of the reusable dialog components to request information from the user to fill an unfilled slot, as necessary, to fill any additional unfilled slots associated with the current task.

38. An apparatus as recited in claim 36, wherein each of the slots represents an item of information which may be acquired from the user.

39. An apparatus as recited in claim 36, wherein the means for identifying the set of all slots potentially associated with a current task is carried out prior to said parsing the utterance.

40. An apparatus as recited in claim 36, wherein the means for identifying the set of all slots potentially associated with a current task comprises means for using a semantic frame that maps tasks performable in response to speech from the user to corresponding slots, to identify the set of all slots potentially associated with the current task.

41. An apparatus as recited in claim 36, wherein each of the reusable dialog components is an instantiation of a speech object class.

42. An apparatus as recited in claim 36, wherein the means for recognizing comprises means for using a set of statistical language models so as to be capable of recognizing open-ended speech.

43. An apparatus as recited in claim 42, wherein at least one of the statistical language models is specifically adapted for a most-recently played prompt.

44. An apparatus as recited in claim 36, wherein a dependency exists between two or more of the slots, the apparatus further comprising the means for identifying a dependency between two of the slots, wherein said parsing the utterance comprises filling one of the slots based on the dependency and a value used to fill another slot.

45. An apparatus as recited in claim 36, wherein the dialog is for accomplishing a task, and wherein the apparatus further comprises means for confirming and correcting slots filled during the dialog, including:

means for determining that one of the slots is incorrect;

means for prompting the user for a corrected value for the slot;

means for receiving the corrected value from the user; and

means for using the corrected value and stored information on dependencies between the slots to control further dialog for accomplishing the task.

46. A machine-readable storage medium embodying instructions for execution by a machine, which instructions configure the machine to perform a method for enabling a mixed initiative dialog to be carried out between a user and the machine, the method comprising:

providing a set of reusable dialog components; and

operating a dialog manager to control use of the reusable dialog components based on a semantic frame, wherein the reusable dialog components are individually configured to carry out system initiated aspects of a dialog.

47. A machine-readable storage medium as recited in claim 46, wherein the reusable dialog components are configured to perform disambiguation and confirmation actions specific to semantic slots associated with a current task, such that the dialog manager does not perform said disambiguation and confirmation actions.

48. A machine-readable storage medium as recited in claim 46, wherein the semantic frame contains a map of tasks to corresponding semantic slots.

49. A machine-readable storage medium as recited in claim 46, said operating the dialog manager comprises:

(a) parsing an utterance using grammars from the set of reusable dialog components;

(b) after said parsing, using a prompt from one of the reusable dialog components to request information from the user to fill an unfilled slot; and

(c) automatically repeating said (b), if necessary, to fill any additional unfilled slots associated with the current task.

50. A device for enabling a mixed initiative dialog to be carried out between a user and a machine, the device comprising:

a set of reusable dialog components individually configured to carry out system initiated aspects of a dialog;

a semantic frame; and

a dialog manager to control use of the reusable dialog components based on the semantic frame.

51. A device as recited in claim 50, wherein the reusable dialog components are configured to perform disambiguation and confirmation actions specific to semantic slots associated with a current task, such that the dialog manager does not perform such disambiguation and confirmation actions.

52. A device as recited in claim 50, wherein the semantic frame contains a map of tasks performable in response to speech from the user to corresponding semantic slots.

53. A device as recited in claim 50, wherein the dialog manager is configured to:

(a) parse an utterance using grammars from the set of reusable dialog components;

(b) after said parsing, use a prompt from one of the reusable dialog components to request information from the user to fill an unfilled slot; and

(c) automatically repeat said (b), if necessary, to fill any additional unfilled slots associated with the current task.

54. A device for carrying out a mixed initiative dialog between a user and a machine, the device comprising:

an automatic speech recognizer to recognize an utterance in speech received from the user using a set of statistical language models;

a set of reusable dialog components;

a dialog manager to use a semantic frame to identify the set of all slots potentially associated with a current task prior to parsing of the utterance, and to retrieve a corresponding grammar for each possible slot from a corresponding one of the reusable dialog components, each slot representing an item of information which may be acquired from the user; and

a natural language parser to receive the retrieved grammars and to parse the utterance using the retrieved grammars, including filling one or more of the possible slots with corresponding values;

wherein the dialog manager further is to identify one of the slots which remains unfilled following said filling, to obtain a prompt for the slot which remains unfilled from a corresponding one of the reusable dialog components, and to cause the prompt to be played to the user to request information for filling the slots which remains unfilled.

55. A device as recited in claim 54, wherein the dialog manager is a reusable dialog component.

56. A method as recited in claim 54, wherein at least one of the statistical language models is specifically adapted for a most-recently played prompt

57. A device as recited in claim 54, wherein a dependency exists between two or more of the slots, and wherein the dialog manager is further configured:

to identify a dependency between two of the slots; and

to fill one of the slots based on the dependency and a value used to fill another slot.

58. A method of confirming and correcting slots filled during a dialog between a user and a machine, the dialog for accomplishing a task, the method comprising:

determining that one of a plurality of slots is incorrect;

prompting the user for a corrected value for the slot;

receiving the corrected value from the user; and

using the corrected value and stored information on dependencies between the slots to control further dialog for accomplishing the task

59. A method as recited in claim 58, wherein said using the corrected value and information on dependencies between the slots to control a revised dialog flow comprises determining one or more reusable dialog components to be invoked, to obtain values for slots.

60. A method as recited in claim 59, wherein during the dialog, at least one of the reusable dialog components has not previously been invoked, and a corresponding slot has not previously been filled.

61. A method as recited in claim 58, wherein the information on dependencies is contained within a semantic frame including a mapping of tasks to slots.