Method for determining and maintaining dialog focus in a conversational speech system

• US 7,069,220 B2
• Filed: 08/13/1999
• Issued: 06/27/2006
• Est. Priority Date: 08/13/1999
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Gopalakrishnan, Popani, Ramaswamy, Ganesh N., Coffman, Daniel M., Kleindienst, Jan
• Primary Examiner(s): ARMSTRONG, ANGELA A
• Application Number:
  US09/374,374
• Publication Number:
  US 20030014260A1
• Time in Patent Office: 2,510 Days
• Field of Search: 704/251, 704/270, 704/275, 704/257, 704/9
• US Class Current: 704/275
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A system and method of the present invention for determining and maintaining dialog focus in a conversational speech system includes presenting a command associated with an application to a dialog manager. The application associated with the command is unknown to the dialog manager at the time it is made. The dialog manager determines a current context of the command by reviewing a multi-modal history of events. At least one method is determined responsive to the command based on the current context. The at least one method is executed responsive to the command associated with the application.

Claims

1. A method for determining and maintaining dialog focus in a conversational speech system comprising the steps of:

maintaining a multi-modal history of events that result from user interaction with one or more user applications of a multi-modal computing system, wherein the events are maintained in chronological order, and wherein the events are linked by event type, wherein an event type includes a change of dialog focus;

receiving and processing a user speech command directed to one of a plurality of the user applications of the multi-modal computing system;

a dialog manager determining a target of the speech command by determining a current context of the speech command by reviewing the multi-modal history of events;

determining at least one method responsive to the command based on the determined current context; and

executing the at least one method responsive to the command.

2. The method as recited in claim 1, wherein the dialog manager determines a target application type from the current context which is responsive to the command, wherein the target application type is abstracted from the user applications supported by the multi-modal system.

3. The method as recited in claim 1, comprising receiving input commands from a user interacting with the multi-modal system using a plurality of modalities including a telephone, a computer, and a personal digital assistant.

4. The method as recited in claim 1, wherein the step of determining a current context of the command by reviewing a multi-modal history of events includes searching the multi-modal history of events by traversing linked events.

5. The method as recited in claim 1, wherein the events in the multi-modal history of events are linked by time, transaction and class.

6. The method as recited in claim 1, wherein the step of determining at least one method includes the step of referencing the user applications of the multi-modal computing system using a component control to determine the at least one method of a user application which is appropriate based on the current context of the speech command.

7. The method as recited in claim 1, wherein the speech command is presented in a formal language such that a plurality of human utterances represent an action to be taken.

8. The method as recited in claim 1, wherein the step of determining a current context of the command by reviewing a multi-modal history of events includes the dialog manager maintaining a current dialog focus and a list of expected responses, which is used by the dialog manager to provide a reference for determining the current context.

9. The method as recited in claim 1, further comprising the step of querying a user for one of information needed to resolve the current context and information needed to take an appropriate action.

10. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for determining and maintaining dialog focus in a conversational speech system, the method steps comprising:

maintaining a multi-modal history of events that result from user interaction with one or more user applications of a multi-modal computing system, wherein the events are maintained in chronological order, and wherein the events are linked by event type, wherein an event type includes a change of dialog focus;

receiving and processing a user speech command directed to one of a plurality of the user applications of the multi-modal computing system;

a dialog manager determining a target of the speech command by determining a current context of the speech command by reviewing the multi-modal history of events;

determining at least one method responsive to the command based on the determined current context; and

executing the at least one method responsive to the command.

11. The program storage device as recited in claim 10, wherein the dialog manager determines a target application type from the current context which is responsive to the command, wherein the target application type is abstracted from the user applications supported by the multi-modal system.

12. The program storage device as recited in claim 10, comprising instructions for receiving input commands from a user interacting with the multi-modal system using a plurality of modalities including a telephone, a computer, and a personal digital assistant.

13. The program storage device as recited in claim 10, wherein the instructions for determining a current context of the command by reviewing a multi-modal history of events includes instructions for searching the multi-modal history of events by traversing linked events.

14. The program storage device as recited in claim 10, wherein the events in the multi-modal history of events are linked by time, transaction and class.

15. The program storage device as recited in claim 10, wherein the instructions for determining at least one method includes the step of referencing the user applications of the multi-modal computing system using a component control to determine the at least one method of a user application which is appropriate based on the current context of the speech command.

16. The program storage device as recited in claim 10, wherein the speech command is presented in a formal language such that a plurality of human utterances represent an action to be taken.

17. The program storage device as recited in claim 10, wherein the instructions for determining a current context of the command by reviewing a multi-modal history of events includes instructions for the dialog manager maintaining a current dialog focus and a list of expected responses, which is used by the dialog manager to provide a reference for determining the current context.

18. The program storage device as recited in claim 10, further comprising instructions for querying a user for one of information needed to resolve the current context and information needed to take an appropriate action.

19. A multi-modal computing system comprising:

a multi-modal history of events to maintain meta information of events that result from user interaction with one or more user applications of the multi-modal computing system, wherein the events are maintained in chronological order, and wherein the events are linked by event type, wherein an event type includes a change of dialog focus;

a dialog manager that maintains a current dialog focus for one of a plurality of user applications and a list of expected responses, wherein the dialog manager determines a target of a speech input event by determining a current context of the speech input event using the multi-modal history of events or the list of expected responses; and

a control component adapted to select at least one method responsive to the speech input event based on the target of the speech input event as determined by the dialog manager.

20. The system as recited in claim 19, wherein the control component selects a method associated with an active user application, an inactive application, an application with a graphical component and an application with other than a graphical component.

21. The system as recited in claim 19, wherein the multi-modal history of events maintains meta information of input events including speech and non-speech input events and output events generated by the multi-modal system.

22. The system as recited in claim 19, wherein the dialog manager determines a current context of the input speech event by traversing linked events in the multi-modal history of events.

23. The system as recited in claim 19, wherein events in the multi-modal history of events are further linked by time, transaction and class.

24. The system as recited in claim 19, wherein the control component references the user applications to determine the at least one method which is appropriate based on the current context of an input speech command.

25. The system as recited in claim 19, wherein an input speech event is presented to the dialog manager in a formal language such that a plurality of human utterances represent an action to be taken.