Method and Apparatus for Managing Dialog Management in a Computer Conversation

• US 20080015864A1
• Filed: 07/16/2007
• Published: 01/17/2008
• Est. Priority Date: 01/12/2001
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: MacAllister, Jeffrey, Hyland, Christopher, Howard, Kathleen, Zeller, Marijane, Ross, Steven
• Granted Patent: US 7,496,514 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A dialog management system functions to manage the dialog between a user of a computer system and one or more speech enabled software applications. The user provides spoken input to a microphone connected to the computer system, and hears responses from one or more applications through a speaker connected to the computer system. The dialog management system includes a dialog manager, a turn manager, a speak queue, dialog contexts, and dialog state. The dialog manager provides top-level control of the dialog and stores responses based on the user'"'"'s spoken input in a speak queue for later output to the user. The turn manager controls delivery of the responses to the user based on the dialog context and the dialog state, to provide a polite asynchronous dialog with the user that enables the user to be in control of the dialog. The dialog context provides information about each dialog. The dialog state provides information about whose turn it is (computer or user) to speak.

Claims

1. A computer apparatus for managing a dialogue between a computer and a user of the computer and a user of the computer, the computer having an audio input device and an audio output device, the computer apparatus comprising:

a speak queue retaining responses generated by the computer in response to spoken input from the user asynchronously received by the computer through the audio input device, the spoken input being interpreted by a reasoning facility which enables the spoken input to include questions by the user;

a dialog manager responsive to the reasoning facility and placing the generated responses in the speak queue, the responses being able to be spoken by a text to speech device, and the dialog manager utilizing dialog context information held in a context priority queue to track state of a dialog, the generated responses in the speak queue having corresponding contexts in the context priority queue; and

a turn manager for managing audible rendering of the responses held in the speak queue through the audio output device based on dialog state, the turn manager ordering audible rendering of the responses according to corresponding contexts in the context priority queue so that the user receives each response as part of an asynchronous dialog between the computer and the user, the turn manager conducting the asynchronous dialog in a polite non-interruptive manner that is subject to control by the user including allowing the user to change subjects and allowing the user to interrupt the asynchronous dialog but not allowing the audible rendering of a response to interrupt the user.

2. The computer apparatus of claim 1, wherein the turn manager is subject to behavioral goals that include:

providing speech output including audible renditions of the responses when spoken to by the user;

asking permission of the user before providing speech output based on delayed answers and notifications; and

allowing the user to (i) change subject and/or (ii) interrupt in the asynchronous dialog.

3. The computer apparatus of claim 1, wherein the turn manager provides the audible rendering of the responses in a delivery mode subject to selection by the user.

4. The computer apparatus of claim 3, wherein the delivery mode is one of an immediate delivery mode and a delayed delivery mode.

5. The computer apparatus of claim 1, wherein the dialog manager and the turn manager have access to application specific knowledge in an application domain model for external applications.

6. The computer apparatus of Claim 1, wherein the response is an announcement of an event of interest to the user as determined by the computer.

7. In a computer system, a method for managing a dialog between a computer and a user of the computer, the computer having an audio input device and an audio output device, the method comprising the computer-implemented steps of:

asynchrously receiving at the computer through the audio input device spoken input from the user;

interpreting the spoken input using a reasoning facility which enables the spoken input to include questions by the user;

as a result of the interpreting, generating responses by the computer in response to the spoken input;

placing the generated responses in a speak queue the responses including responses that can be spoken by a text to speech device;

using dialog context information held in a context priority queue, tracking state of a dialog, responses in the speak queue having corresponding contexts held in the context priority queue; and

based on dialog state, managing audible rendering of the responses held in the speak queue through the audio output device, ordering the audible rendering of the responses according to corresponding contexts in the context priority queue so that the user receives each response as part of an asynchronous dialog between the computer and the user, the asynchronous dialog conducted in a polite non-interruptive manner that is subject to control by the user including allowing the user to change subjects and allowing the user to interrupt the asynchronous dialog but not allowing the audible rendering of a response to interrupt the user.

8. In a computer system, the method of claim 7, wherein the step of managing the audible rendering of the responses is performed subject to behavioral goals that include:

providing speech output including audible renditions of the responses when spoken to by the user;

asking permission of the user before providing speech output based on delayed answers and notifications; and

allowing the user to (i) change subject and/or (ii) interrupt in the asynchronous dialog.

9. In a computer system, the method of claim 7 wherein the step of managing the audible rendering of responses is performed in a delivery mode subject to selection by the user.

10. In a computer system, the method of claim 9 wherein the delivery mode is one of an immediate delivery mode and a delayed delivery mode.

11. In a computer system, the method of claim 7 wherein the step of managing the audible rendering of the responses is based on dialog states that specify the current state of the dialog between the computer and the user.

12. In a computer system, the method of claim 7 wherein the response is an announcement of an event of interest to the user as determined by the computer.

13. A computer program product comprising:

a tangible computer usable program product for managing a dialog between a computer and a user of the computer; and

a set of computer program instructions embodied on the tangible computer usable program product, including instructions to;

asynchrously receive at the computer through an audio input device synchronizing input from user;

interpret the spoken input using a reasoning facility which enables the spoken input to include questions by the user;

generate responses by the computer in response to the spoken input;

place the generated responses in a speak queue, the responses including responses that can be spoken by a text to speech device;

using dialog context information held in a context priority queue, track state of a dialog, responses in the speak queue having corresponding contexts held in the context priority queue; and

based on dialog state, manage audible rendering of the responses held in the speak queue through an audio output device, ordering the audible rendering of the responses according to corresponding contexts in the context priority queue so that the user receives each response as part of an asynchronous dialog between the computer and the user, the asynchronous dialog being conducted in a polite non-interruptive manner that is subject to control by the user including allowing the user to change subjects and allowing the user to interrupt the asynchronous dialog but not allowing the audible rendering of a response to interrupt the user.

14. The computer program product of claim 13, wherein the set of computer instructions comprises further instructions to manage the audible rendering of the responses subject to behavioral goals that include:

providing speech output including audible renditions of the responses when spoken to by the user;

asking permission of the user before providing speech output based on delayed answers and notifications; and

allowing the user to (i) change subject and/or (ii) interrupt in the asynchronous dialog.

15. The computer program product of claim 13, wherein the set of computer instructions comprises further instructions to manage the audible rendering of responses based on a delivery mode subject to selection by the user.

16. The computer program product of claim 15, wherein the delivery mode is one of an immediate delivery mode and a delayed delivery mode.

17. The computer program product of claim 13, wherein the set of computer instructions comprises further instructions to manage the audible rendering of the responses based on dialog states that specify the current state of the dialog between the computer and the user.

18. The computer program product of claim 13, wherein the response is an announcement of an event of interest to the user as determined by the computer.

19. A computerized system for managing a dialog between a computer and a user of the computer, the computerized system comprising:

speak queue means for retaining responses generated by the computer in response to spoken input from the user asynchronously received by the computer through an audio input device, the spoken input being interpreted by a reasoning facility which enables the spoken input to include questions by the user;

dialog means responsive to the reasoning facility and for placing the generated responses in the speak queue means, the responses including responses that can be spoken by a text to speech device;

context means enabling the dialog means to track state of a dialog, the generated responses in the speak queue means having corresponding context in the context means; and

;

turn means for managing audible rendering of the responses held in the speak queue means through an audio output device based on dialog state, the turn means ordering audible rendering of the responses according to corresponding contexts in the context means so that the user receives each response as part of an asynchronous dialog between the computer and the user, the asynchronous dialog being conducted in a polite non-interruptive manner that is subject to control by the user including allowing the user to change subjects and allowing the user to interrupt the asynchronous dialog but not allowing the audible rendering of a response to interrupt the user.

20. The computerized system of claim 19 wherein the turn means includes any combination of:

providing speech output including audible renditions of the responses when spoken to by the user;

asking permission of the user before providing speech output based on delayed answers and notifications; and

allowing the user to (i) change subject and/or (ii) interrupt the asynchronous dialog.