System and method for multi-agent architecture for interactive machines

• US 9,530,412 B2
• Filed: 08/29/2014
• Issued: 12/27/2016
• Est. Priority Date: 08/29/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Selfridge, Ethan
• Primary Examiner(s): Riley, Marcus T
• Application Number:
  US14/473,288
• Publication Number:
  US 20160063992A1
• Time in Patent Office: 851 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/222 Barge in, i.e. overridable guidance for interrupting prompts
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology

Abstract

Systems, methods, and computer-readable storage devices are for an event-driven multi-agent architecture improves via a semi-hierarchical multi-agent reinforcement learning approach. A system receives a user input during a speech dialog between a user and the system. The system then processes the user input, identifying an importance of the user input to the speech dialog based on a user classification and identifying a variable strength turn-taking signal inferred from the user input. An utterance selection agent selects an utterance for replying to the user input based on the importance of the user input, and a turn-taking agent determines whether to output the utterance based on the utterance, and the variable strength turn-taking signal. When the turn-taking agent indicates the utterance should be output, the system selects when to output the utterance.

Claims

1. A method comprising:

receiving a user input during a speech dialog between a user and a system, wherein the user is implicitly classified as a type of user;

processing the user input by;

identifying an importance of the user input to the speech dialog based on the type of user; and

identifying a variable strength turn-taking signal inferred from the user input;

selecting, via an utterance selection agent, an utterance for replying to the user input based on the importance of the user input;

determining, via a turn-taking agent, whether to output the utterance based on the utterance, and the variable strength turn-taking signal, to yield a determination;

when the determination indicates the utterance should be output, selecting a timing for when to output the utterance;

repeating the receiving, the processing, the selecting, and the determining throughout the speech dialog; and

updating a first rule set used for the identifying of the importance of the user input and a second rule set for the identifying of the variable strength turn-taking signal in the user input.

2. The method of claim 1, wherein the type of user is one of an expert and a novice.

3. The method of claim 1, wherein the updating occurs after the utterance is output.

4. The method of claim 1, wherein updating of the first rule set is based on reinforcing relatively short utterances.

5. The method of claim 1, wherein updating of the second rule set is based on reinforcing a minimal time between the user input and the output of the utterance.

6. The method of claim 1, wherein the utterance selection agent and the turn-taking agent are both Markov decision processes.

7. The method of claim 6, wherein the turn-taking agent is dependent on the utterance selection agent.

8. The method of claim 1, wherein the user input is silence.

9. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

receiving a user input during a speech dialog between a user and a system, wherein the user is implicitly classified as a type of user;

processing the user input by;

identifying an importance of the user input to the speech dialog based on the type of user; and

identifying a variable strength turn-taking signal inferred from the user input;

selecting, via an utterance selection agent, an utterance for replying to the user input based on the importance of the user input;

determining, via a turn-taking agent, whether to output the utterance based on the utterance, and the variable strength turn-taking signal, to yield a determination;

when the determination indicates the utterance should be output, selecting a timing for when to output the utterance;

repeating the receiving, the processing, the selecting, and the determining throughout the speech dialog; and

updating a first rule set used for the identifying of the importance of the user input and a second rule set for the identifying of the variable strength turn-taking signal in the user input.

10. The system of claim 9, wherein the type of user is one of an expert and a novice.

11. The system of claim 9, wherein the updating occurs after the utterance is output.

12. The system of claim 9, wherein updating of the first rule set is based on reinforcing relatively short utterances.

13. The system of claim 9, wherein updating of the second rule set is based on reinforcing a minimal time between the user input and the output of the utterance.

14. The system of claim 9, wherein the utterance selection agent and the turn-taking agent are both Markov decision processes.

15. The system of claim 14, wherein the turn-taking agent is dependent on the utterance selection agent.

16. The system of claim 9, wherein the user input is silence.

17. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

receiving a user input during a speech dialog between a user and a system, wherein the user is implicitly classified as a type of user;

processing the user input by;

identifying an importance of the user input to the speech dialog based on the type of user; and

identifying a variable strength turn-taking signal inferred from the user input;

selecting, via an utterance selection agent, an utterance for replying to the user input based on the importance of the user input;

determining, via a turn-taking agent, whether to output the utterance based on the utterance, and the variable strength turn-taking signal, to yield a determination;

when the determination indicates the utterance should be output, selecting a timing for when to output the utterance;

repeating the receiving, the processing, the selecting, and the determining throughout the speech dialog; and

updating a first rule set used for the identifying of the importance of the user input and a second rule set for the identifying of the variable strength turn-taking signal in the user input.

18. The computer-readable storage device of claim 17, wherein the type of user is one of an expert and a novice.