Dialog knowledge acquisition system and method

• US 10,162,815 B2
• Filed: 01/03/2017
• Issued: 12/25/2018
• Est. Priority Date: 09/02/2016
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Lehman, Jill Fain, Li, Boyang Albert, Pereira, Andre Tiago Abelho, Dos Santos Caravalho Leite, Iolanda M., Sun, Ming, Pincus, Eli
• Primary Examiner(s): Sirjani, Fariba
• Application Number:
  US15/397,659
• Publication Number:
  US 20180068658A1
• Time in Patent Office: 721 Days
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06F 9/00 Arrangements for program control, e.g. control units program control for peripheral devices G06F13/10
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: H04M 3/00 Automatic or semi-automatic exchanges

Abstract

A dialog knowledge acquisition system includes a hardware processor, a memory, and hardware processor controlled input and output modules. The memory stores a dialog manager configured to instantiate a persistent interactive personality (PIP), and a dialog graph having linked dialog state nodes. The dialog manager receives dialog initiation data, identifies a first state node on the dialog graph corresponding to the dialog initiation data, determines a dialog interaction by the PIP based on the dialog initiation data and the first state node, and renders the dialog interaction. The dialog manager also receives feedback data corresponding to the dialog interaction, identifies a second state node based on the dialog initiation data, the dialog interaction, and the feedback data, and utilizes the dialog initiation data, the first state node, the dialog interaction, the feedback data, and the second state node to train the dialog graph.

Claims

1. A dialog knowledge acquisition system comprising:

a hardware processor and a memory;

an input module and an output module controlled by the hardware processor;

the memory storing a dialog manager configured to instantiate a persistent interactive personality (PIP) and a dialog graph having a plurality of linked dialog state nodes;

the hardware processor configured to execute the dialog manager to;

receive a dialog initiation data via the input module;

identify a first state node on the dialog graph corresponding to the dialog initiation data;

determine a dialog interaction by the PIP based on the dialog initiation data and the first state node;

render the dialog interaction via the output module, wherein the output module renders the dialog interaction as at least one of a speech, a facial expression, and a gesture being performed by one of a robot and an avatar;

receive a user response corresponding to the dialog interaction via the input module;

perform a strict dialog execution policy such that the user response is only mapped to semantically equivalent children of the first state node;

perform a relaxed dialog execution policy such that the user response is mapped to the plurality of linked dialog state nodes outside the children of the first state node, upon registering a dialog failure;

identify a second state node on the dialog graph based on the dialog initiation data, the dialog interaction, and the user response; and

utilize the dialog initiation data, the first state node, the dialog interaction, the user response, and the second state node to train the dialog graph for subsequent dialog interactions by the PIP.

2. The dialog knowledge acquisition system of claim 1, wherein the dialog interaction comprises a language based communication by the PIP.

3. The dialog knowledge acquisition system of claim 1, wherein the dialog interaction comprises a non-verbal communication by the PIP.

4. The dialog knowledge acquisition system of claim 1, wherein the input module comprises at least one of a keyboard and a touchscreen display, and wherein the dialog initiation data is received as an input to one of the at least one of the keyboard and the touchscreen display.

5. The dialog knowledge acquisition system of claim 1, wherein the input module comprises at least one sensor, and wherein the dialog initiation data comprises sensor data produced by the at least one sensor.

6. The dialog knowledge acquisition system of claim 1, wherein the input module comprises a microphone and an analog-to-digital converter (ADC), and wherein the dialog initiation data is converted by the ADC from speech received by the microphone.

7. The dialog knowledge acquisition system of claim 1, wherein the output module comprises a display, and wherein the dialog interaction is rendered as text on the display.

8. The dialog knowledge acquisition system of claim 1, wherein the output module comprises an audio output device and a display showing the avatar as a representation of the PIP, and wherein the dialog interaction is rendered as at least one of the speech by the avatar, the facial expression by the avatar, and the gesture by the avatar.

9. The dialog knowledge acquisition system of claim 1, further comprising the robot as a representation of the PIP, wherein the robot includes the output module, and wherein the dialog interaction is rendered as at least one of the speech by the robot, the facial expression by the robot, and the gesture by the robot.

10. The dialog knowledge acquisition system of claim 1, further comprising a network communication module controlled by the hardware processor, wherein the hardware processor is further configured to execute the dialog manager to:

generate a narrative corresponding to at least the dialog initiation data;

send a dialog interaction recommendation request based on the narrative to a first plurality of contributors via the network communication module;

receive a second plurality of dialog interaction recommendations from at least some of the first plurality of contributors via the network communication module;

filter the second plurality of dialog interaction recommendations to identify at least one adopted dialog interaction; and

train the dialog graph to include the at least one adopted dialog interaction.

11. A method for use by a dialog knowledge acquisition system including a hardware processor, an input module and an output module controlled by the hardware processor, and a memory storing a dialog manager configured to instantiate a persistent interactive personality (PIP) and a dialog graph having a plurality of linked dialog state nodes, the method comprising:

receiving, by the dialog manager executed by the hardware processor, a dialog initiation data via the input module;

identifying, by the dialog manager executed by the hardware processor, a first state node on the dialog graph corresponding to the dialog initiation data;

determining, by the dialog manager executed by the hardware processor, a dialog interaction by the PIP based on the dialog initiation data and the first state node;

rendering, by the dialog manager executed by the hardware processor, the dialog interaction via the output module, wherein the output module renders the dialog interaction as at least one of a speech, a facial expression, and a gesture being performed by one of a robot and an avatar;

receiving, by the dialog manager executed by the hardware processor, a user response corresponding to the dialog interaction via the input module;

performing, by the dialog manager executed by the hardware processor, a strict dialog execution policy such that the user response is only mapped to semantically equivalent children of the first state node;

performing, by the dialog manager executed by the hardware processor, a relaxed dialog execution policy such that the user response is mapped to the plurality of linked dialog state nodes outside the children of the first state node, upon registering a dialog failure;

identifying, by the dialog manager executed by the hardware processor, a second state node on the dialog graph based on the dialog initiation data, the dialog interaction, and the user response; and

utilizing, by the dialog manager executed by the hardware processor, the dialog initiation data, the first state node, the dialog interaction, the user response, and the second state node to train the dialog graph for subsequent dialog interactions by the PIP.

12. The method of claim 11, wherein the dialog interaction comprises a language based communication by the PIP.

13. The method of claim 11, wherein the dialog interaction comprises a non-verbal communication by the PIP.

14. The method of claim 11, wherein the input module comprises at least one of a keyboard and a touchscreen display, and wherein the dialog initiation data is received as an input to one of the at least one of the keyboard and the touchscreen display.

15. The method of claim 11, wherein the input module comprises at least one sensor, and wherein the dialog initiation data comprises sensor data produced by the at least one sensor.

16. The method of claim 11, wherein the input module comprises a microphone and an analog-to-digital converter (ADC), and wherein the dialog initiation data is converted by the ADC from speech received by the microphone.

17. The method of claim 11, wherein the output module comprises a display, and wherein the dialog interaction is rendered as text on the display.

18. The method of claim 11, wherein the output module comprises an audio output device and a display showing the avatar as a representation of the PIP, and wherein the dialog interaction is rendered as at least one of the speech by the avatar, the facial expression by the avatar, and the gesture by the avatar.

19. The method of claim 11, wherein the dialog knowledge acquisition system further comprises the robot as a representation of the PIP, wherein the robot includes the output module, and wherein the dialog interaction is rendered as at least one of the speech by the robot, the facial expression by the robot, and the gesture by the robot.

20. The method of claim 11, wherein the dialog knowledge acquisition system further comprises a network communication module controlled by the hardware processor, and wherein the method further comprises:

generating, by the dialog manager executed by the hardware processor, a narrative corresponding to at least the dialog initiation data;

sending, by the dialog manager executed by the hardware processor, a dialog interaction recommendation request based on the narrative to a first plurality of contributors via the network communication module;

receiving, by the dialog manager executed by the hardware processor, a second plurality of dialog interaction recommendations from at least some of the first plurality of contributors via the network communication module;

filtering, by the dialog manager executed by the hardware processor, the second plurality of dialog interaction recommendations to identify at least one adopted dialog interaction; and

training, by the dialog manager executed by the hardware processor, the dialog graph to include the at least one adopted dialog interaction.