Method and apparatus for exploiting human feedback in an intelligent automated assistant

• US 10,366,336 B2
• Filed: 09/01/2010
• Issued: 07/30/2019
• Est. Priority Date: 09/02/2009
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Tur, Gokhan, Franco, Horacio E., Mark, William S., Winarsky, Norman D., Peintner, Bart, Wolverton, Michael J., Yorke-Smith, Neil
• Primary Examiner(s): Nilsson, Eric
• Application Number:
  US13/378,525
• Publication Number:
  US 20120173464A1
• Time in Patent Office: 3,254 Days
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

The present invention relates to a method and apparatus for exploiting human feedback in an intelligent automated assistant. One embodiment of a method for conducting an interaction with a human user includes inferring an intent from data entered by the human user, formulating a response in accordance with the intent, receiving feedback from a human advisor in response to at least one of the inferring and the formulating, wherein the human advisor is a person other than the human user, and adapting at least one model used in at least one of the inferring and the formulating, wherein the adapting is based on the feedback.

Claims

1. A method for conducting an interaction between a human user and a device, the method comprising:

with the device, receiving input indicative of a user request for information;

with a microphone coupled to the device, receiving sensed data;

with a feature extraction processor coupled to the microphone, extracting a plurality of speech features from the sensed data;

with a classifier processor coupled to the feature extraction processor, inferring an affective state of the human user based on the plurality of speech features extracted from the sensed data;

with an interaction management system coupled to the classifier processor, inferring an intent from the received input by performing one or more of automated speech recognition and natural language understanding using a learned model;

formulating a proposed response to the received input in accordance with the intent, the proposed response comprising system-generated output;

determining a measure of certainty associated with one or more of the intent and the proposed response;

presenting a final response to the received input by an output device of the device when the measure of certainty satisfies a minimum acceptable level of certainty;

with an interface coupled to the interaction management system between the inferring of the intent and the presenting of the final response, when the measure of certainty does not satisfy the minimum acceptable level of certainty and prior to presenting the final response, communicating the intent and the proposed response and the inferred affective state to a wizard, receiving feedback on the intent and the proposed response and the affective state from the wizard, incorporating the feedback into the final response, updating a model used to generate the proposed response based on the feedback, wherein the wizard is a human person who is not a source of the received input.

2. The method of claim 1, further comprising:

outputting the final response after adapting the proposed response.

3. The method of claim 1, further comprising:

inferring at least one personal characteristic of the human user; and

formatting the proposed response in accordance with the at least one personal characteristic.

4. The method of claim 3, wherein the at least one personal characteristic is inferred from data that is sensed by one or more sensors.

5. The method of claim 4, wherein the at least one personal characteristic is at least one of:

an age of the human user,

a gender of the human user,

a socioeconomic status of the human user, or

an affective state of the human user.

6. The method of claim 3, wherein the formatting comprises:

selecting a modality for outputting the response in accordance with the at least one personal characteristic.

7. The method of claim 1, further comprising delaying the response until the human person is contacted.

8. The method of claim 7, wherein the delaying comprises:

providing the human person with a summary of the interaction so far;

providing a preliminary response to the human user, based on feedback received from the human person in response to the summary; and

providing the human advisor with a full context of the interaction while the human user is reviewing the preliminary response.

9. The method of claim 8, wherein the providing the human person with the summary comprises:

generating a transcript of the interaction;

comparing the interaction to one or more learned models; and

indicating words in the transcript that are rare or atypical relative to the one or more learned models.

10. The method of claim 7, wherein the delaying comprises:

providing the human user with a canned response; and

providing the human person with a full context of the interaction while the human user is reviewing the canned response.

11. The method of claim 1, wherein the determining comprises:

labeling a plurality of past interactions according to whether each of the plurality of past interactions required a consultation with the human person;

extracting a plurality of features from each of the plurality of past interactions; and

formulating the determining as a supervised learning problem that uses results of the labeling and the extracting as inputs.

12. The method of claim 7, wherein the delaying comprises:

informing the human user that the human person is being consulted; and

providing the human person with a full context of the interaction.

13. The method of claim 11, further comprising:

associating a label with an interaction with the human user indicating whether the interaction with the human user requires a consultation with the human person, based on a result of the supervised learning problem, wherein the label is associated with a confidence; and

requesting the consultation when the confidence fails to meet at least a threshold confidence.

14. The method of claim 1, wherein the receiving comprises:

receiving a plurality of items of feedback from a plurality of human advisors; and

combining the plurality of items of feedback to generate a consensus feedback.

15. The method of claim 14, wherein the combining comprises:

weighting each of the plurality of items of feedback according to a confidence indicating a past performance of an associated human advisor.

16. The method of claim 1, wherein the measure of certainty comprises a statistical measure of dialogue certainty.

17. The method of claim 1, wherein the measure of certainty defines a minimum distance between two probabilities in a distribution.

18. The method of claim 1, wherein the measure of certainty comprises an adjustable threshold value.

19. The method of claim 1, wherein the measure of certainty comprises a posterior probability.

20. The method of claim 1, wherein the measure of certainty is determined based at least in part on machine learning.

21. The method of claim 1, wherein the determining comprises:

producing a distribution comprising a plurality of hypotheses for either the intent or the response;

identifying a best hypothesis and a second best hypothesis among the plurality of hypotheses; and

requesting a consultation when a distance between a confidence in the best hypothesis and a confidence in the second best hypothesis fails to meet at least a threshold distance.

22. The method of claim 2, wherein the determining comprises:

producing a distribution comprising a plurality of hypotheses for either the intent or the response;

identifying a best hypothesis and a second best hypothesis among the plurality of hypotheses;

identifying a result of the best hypothesis and a result of the second best hypothesis; and

requesting a consultation only when the result of the best hypothesis and the result of the second best hypothesis are different.

23. At least one non-transitory computer readable medium containing an executable program for conducting an interaction between a human user and a computing device, where the executable program performs steps comprising:

with a microphone coupled to the computing device, receiving sensed data;

with a feature extraction processor coupled to the microphone, extracting a plurality of features from the sensed data;

with a classifier processor coupled to the feature extraction processor, inferring an affective state of the human user based on the plurality of features extracted from the sensed data;

with an interaction management system coupled to the classifier processor, inferring an intent from a received input by performing one or more of automated speech recognition and natural language understanding using a learned model;

formulating a proposed response in accordance with the intent and the affective state, the proposed response comprising system-generated output;

determining a measure of certainty associated with one or more of the intent and the proposed response;

presenting a final response by an output device of the computing device when the measure of certainty satisfies a minimum acceptable level of certainty;

interposing an interface between the inferring of the intent and the presenting of the final response to, when the measure of certainty does not satisfy the minimum acceptable level of certainty and prior to presenting the final response, communicate the intent and the proposed response to a wizard, receive feedback on the intent and the proposed response from the wizard, incorporate the feedback into the final response, update a model used to generate the proposed response based on the feedback, wherein the wizard is not a source of the received input.

24. A system for conducting an interaction between a human user and a consumer computing device, the system comprising a plurality of processor-executable modules embodied in one or more non-transitory machine readable storage media, the system configured to:

with a microphone coupled to the consumer computing device, receiving sensed data;

with a feature extraction processor coupled to the microphone, extracting a plurality of features from the sensed data;

with a classifier processor coupled to the feature extraction processor, inferring an affective state of the human user based on the plurality of features extracted from the sensed data;

with an interaction management system coupled to the classifier processor, infer an intent from a received input by performing one or more of automated speech recognition and natural language understanding using a learned model;

formulate a proposed response in accordance with the intent and the affective state, the proposed response comprising system-generated output;

determine a measure of certainty associated with one or more of the intent and the proposed response;

present a final response by an output device of the consumer computing device when the measure of certainty satisfies a minimum acceptable level of certainty;

with an interface coupled between the inferring of the intent and the presenting of the final response, when the measure of certainty does not satisfy the minimum acceptable level of certainty and prior to presenting the final response, communicate the intent and the proposed response to a wizard, receive feedback on the intent and the proposed response from the wizard, incorporate the feedback into the final response, update a model used to generate the proposed response based on the feedback, wherein the wizard is not a source of the received input.