Method and apparatus for performing dialog management in a computer conversational interface

• US 7,257,537 B2
• Filed: 01/10/2002
• Issued: 08/14/2007
• Est. Priority Date: 01/12/2001
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Ross, Steven I., Zeller, Marijane M., Hyland, Christopher A., Howard, Kathleen A., MacAllister, Jeffrey G.
• Primary Examiner(s): Dorvil; Richemond
• Assistant Examiner(s): ALBERTALLI, BRIAN LOUIS
• Application Number:
  US10/044,647
• Publication Number:
  US 20020133355A1
• Time in Patent Office: 2,042 Days
• Field of Search: 704/257, 704/270, 704/9, 379/88.18, 710/54
• US Class Current: 704/270
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A dialog management system functions to manage the dialog between a user of a computer system and one or more speech enabled software applications. The user provides spoken input to a microphone connected to the computer system, and hears responses from one or more applications through a speaker connected to the computer system. The dialog management system includes a dialog manager, a turn manager, a speak queue, dialog contexts, and dialog state. The dialog manager provides top-level control of the dialog and stores responses based on the user'"'"'s spoken input in a speak queue for later output to the user. The turn manager controls delivery of the responses to the user based on the dialog context and the dialog state, to provide a polite dialog with the user that enables the user to be in control of the dialog. The dialog context provides information about each dialog. The dialog state provides information about whose turn it is (computer or user) to speak.

Claims

1. A computerized interface for managing a dialog between a computer and a user of the computer, the computer having an audio input device, a text input device, a visual output device, and an audio output device, the computerized interface comprising:

a prioritized speak queue for retaining responses generated by the computer in response to spoken input from the user asynchronously received by the computer through the audio input device, the responses including responses that can be spoken by a text to speech device, the spoken input being interpreted by a reasoning facility which enables the spoken input to include questions by the user, the computer running multiple applications and the reasoning facility interpreting the spoken input in a manner that at least one of the multiple applications recognizes the interpreted spoken input;

a dialog manager for placing the generated responses in the prioritized speak queue; and

a turn manager for managing audible rendering of the responses from the prioritized speak queue through the audio output device, the turn manager prioritizing audible rendering of the responses according to rules other than the order in which the responses are added to the prioritized speak queue and according to corresponding contexts in a context priority queue so that the user receives each response as part of an asynchronous dialog between the computer and the user, the turn manager conducting the dialog in a polite non-interruptive manner that is subject to control by the user including allowing the user to change subjects and allowing the user to interrupt the dialog but not allowing the audible rendering of a response to interrupt the user.

2. The interface of claim 1, wherein the turn manager is subject to behavioral goals that include:

providing speech output including audible renditions of the responses when spoken to by the user;

asking permission of the user before providing speech output based on delayed answers and notifications; and

allowing the user to (i) change subject and/or (ii) interrupt in the dialog.

3. The interface of claim 1, wherein the turn manager provides the audible rendering of the responses in a delivery mode subject to selection by the user.

4. The interface of claim 3, wherein the delivery mode is one of an immediate delivery mode and a delayed delivery mode.

5. The interface of claim 1, wherein the turn manager manages the audible rendering of the responses based on dialog states that specify the current state of the dialog between the computer and the user.

6. The interface of claim 1, wherein the response is an announcement of an event of interest to the user as determined by the computer.

7. The interface of claim 1, wherein the dialog manager and the turn manager have access to application specific knowledge in an application domain model for external applications.

8. A method for managing a dialog between a computer and a user of the computer, the computer having an audio input device, a text input device, a visual output device, and an audio output device, the method comprising the computer-implemented steps of:

receiving responses generated by the computer, the responses including responses that can be spoken by a text to speech device, in response to spoken input from the user asynchronously received by the computer through the audio input device;

placing the generated responses in a prioritized speak queue, the spoken input being interpreted by a reasoning facility which enables the spoken input to include questions by the user, the computer running multiple applications and the reasoning facility interpreting the spoken input in a manner that at least one of the multiple applications recognizes the interpreted spoken input; and

managing audible rendering of the responses from the prioritized speak queue through the audio output device, prioritizing the audible rendering of the responses according to rules other than the order in which the responses are added to the prioritized speak queue and according to corresponding contexts in a context priority queue so that the user receives each response as part of an asynchronous dialog between the computer and the user, the dialog conducted in a polite non-interruptive manner that is subject to control by the user including allowing the user to change subjects and allowing the user to interrupt the dialog but not allowing the audible rendering of a response to interrupt the user.

9. The method of claim 8, wherein the step of managing the audible rendering of the responses is performed subject to behavioral goals that include:

providing speech output including audible renditions of the responses when spoken to by the user;

asking permission of the user before providing speech output based on delayed answers and notifications; and

allowing the user to (i) change subject and/or (ii) interrupt in the dialog.

10. The method of claim 8, wherein the step of managing the audible rendering of responses is performed in a delivery mode subject to selection by the user.

11. The method of claim 10, wherein the delivery mode is one ofan immediate delivery mode and a delayed delivery mode.

12. The method of claim 8, wherein the step of managing the audible rendering of the responses is based on dialog states that specify the current state of the dialog between the computer and the user.

13. The method of claim 8, wherein the response is an announcement of an event of interest to the user as determined by the computer.

14. A computer program product comprising:

a tangible computer usable program product for managing a dialog between a computer and a user of the computer; and

a set of computer program instructions embodied on the tangible computer usable program product, including instructions to;

receive responses generated by the computer, the responses including responses that can be spoken by a text to speech device, in response to spoken input from the user asynchronously received by the computer through the audio input device;

place the generated responses in a prioritized speak queue, the spoken input being interpreted by a reasoning facility which enables the spoken input to include questions by the user, the computer running multiple applications and the reasoning facility interpreting the spoken input in a manner that at least one of the multiple applications recognizes the interpreted spoken input; and

manage audible rendering of the responses from the prioritized speak queue through the audio, output device, prioritizing the audible rendering of the responses according to rules other than the order in which the responses are added to the prioritized speak queue and according to corresponding contexts in a context priority queue so that the user receives each response as part of an asynchronous dialog between the computer and the user, the dialog conducted in a polite non-interruptive manner that is subject to control by the user including allowing the user to change subjects and allowing the user to interrupt the dialog but not allowing the audible rendering of a response to interrupt the user.

15. The computer program product of claim 14, wherein the set of computer instructions comprises further instructions to manage the audible rendering of the responses subject to behavioral goals that include:

providing speech output including audible renditions of the responses when spoken to by the user;

asking permission of the user before providing speech output based on delayed answers and notifications; and

allowing the user to (i) change subject and/or (ii) interrupt in the dialog.

16. The computer program product of claim 14, wherein the set of computer instructions comprises further instructions to manage the audible rendering of responses based on a delivery mode subject to selection by the user.

17. The computer program product of claim 16, wherein the delivery mode is one of an immediate delivery mode and a delayed delivery mode.

18. The computer program product of claim 14, wherein the set of computer instructions comprises further instructions to manage the audible rendering of the responses based on dialog states that specify the current state of the dialog between the computer and the user.

19. The computer program product of claim 14, wherein the response is an announcement of an event of interest to the user as determined by the computer.

20. A computerized interface for managing a dialog between a computer and a user of the computer, the computer having an audio input device, a text input device, a visual output device, and an audio output device, the computerized interface comprising:

a prioritized speak queue for retaining responses generated by the computer the responses including responses that can be spoken by a text to speech device, in response to spoken input from the user asynchronously received by the computer through the audio input device, the spoken input being interpreted by a reasoning facility which enables the spoken input to include questions by the user, the computer running multiple applications and the reasoning facility interpreting the spoken input in a manner that at least one of the multiple applications recognizes the interpreted spoken input;

means for placing the generated responses in the prioritized speak queue; and

means for managing audible rendering of the responses from the prioritized speak queue through the audio output device, the turn manager prioritizing audible rendering of the responses according to rules other than the order in which the responses are added to the prioritized speak queue and according to corresponding contexts in a context priority queue so that the user receives each response as part of an asynchronous dialog between the computer and the user, the dialog conducted in a polite non-interruptive manner that is subject to control by the user including allowing the user to change subjects and allowing the user to interrupt the dialog but not allowing the audible rendering of a response to interrupt the user.