Online learning for dialog systems

• US 20060206337A1
• Filed: 06/29/2005
• Published: 09/14/2006
• Est. Priority Date: 03/08/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Paek, Timothy S., Chickering, David M., Horvitz, Eric J.
• Granted Patent: US 7,734,471 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G10L 15/065 Adaptation

Abstract

An online dialog system and method are provided. The dialog system receives speech input and outputs an action according to its models. After executing the action, the system receives feedback from the environment or user. The system immediately utilizes the feedback to update its models in an online fashion.

Claims

1. An online learning dialog system comprising:

a speech model that receives a speech input and provides speech events;

a decision engine model that selects an action based, at least in part, upon the speech events; and

,

a learning component that in an online manner modifies at least one of the parameters of the decision engine model based upon feedback associated with the selected action.

2. The system of claim 1 where the decision engine employs a heuristic technique to maximize long term expected utility.

3. The system of claim 2 where the heuristic technique is the Thompson strategy.

4. A voice-controlled web browser that employs the system of claim 1.

5. A voice-controlled mobile device that employs the system of claim 1.

6. A speech application that employs the system of claim 1.

7. The system of claim 1 with the action comprising a repair dialog.

8. The system of claim 7 with the repair dialog including a request to repeat and/or a request for confirmation.

9. The system of claim 1 with the actions comprising ignore, execute the action corresponding to a most likely command, request to repeat, and, provide information associated with a plurality of likely commands along with a request to confirm.

10. The system of claim 1, where the feedback comprises a negative input or a positive input utterance from a user of the system or the environment.

11. The system of claim 1, where the parameters of the speech model are updated based on feedback associated with the selected action.

12. The system of claim 1, where the learning component employs retrospective analysis to modify at least one of the parameters of the speech model.

13. The system of claim 1 with the feedback comprising a lack of an input from a user of the system within a threshold period of time.

14. The system of claim 1 with the decision engine model comprising a Markov decision process represented as an influence diagram.

15. The system of claim 1 with uncertainty of the parameters expressed using Dirichlet priors for conditional distributions of discrete variables of the decision engine model, and, Normal-Wishart priors for distributions of continuous variables of the decision engine model.

16. An online learning dialog method comprising:

determining a probability distribution associated with uncertainty regarding parameters of a decision engine model regarding speech events from spoken input;

selecting an action based, at least in part, upon the probability distribution;

receiving feedback associated with the selected action; and

,

updating at least one parameter of the decision engine model or speech model based, at least in part, upon feedback associated with the selected action.

17. The method of claim 16, the feedback comprising a verbal response to the selected action or a lack of verbal response to the selected action in a threshold period of time.

18. A computer readable medium having stored thereon computer executable instructions for carrying out the method of claim 16.

19. An online learning dialog system comprising:

means for selecting an action based, at upon in part, upon a probability distribution received from a means for modeling speech, the probability distribution associated with uncertainty regarding parameters of the means for modeling speech; and

means for modifying parameters of the means for modeling speech based upon feedback associated with the selected action.

20. The system of claim 19, the means for selecting an action employs a heuristic technique to maximize long term expected utility.