SPOKEN DIALOG SYSTEM BASED ON DUAL DIALOG MANAGEMENT USING HIERARCHICAL DIALOG TASK LIBRARY

• US 20140136212A1
• Filed: 03/25/2013
• Published: 05/15/2014
• Est. Priority Date: 11/14/2012
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: KWON, Oh Woog, ROH, Yoon Hyung, YANG, Seong Il, LEE, Ki Young, CHOI, Sung Kwon, PARK, Eun Jin, HUANG, Jinxia, KIM, Young Kil, KIM, Chang Hyun, SEO, Young Ae, JIN, Yun, PARK, Sang Kyu, JUNG, Sang Keun, NA, Seung Hoon, SHIN, Jong Hun
• Granted Patent: US 9,230,544 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

The present invention relates to a spoken dialog system and method based on dual dialog management using a hierarchical dialog task library that may increase reutilization of dialog knowledge by constructing and packaging the dialog knowledge based on a task unit having a hierarchical structure, and may construct and process the dialog knowledge using a dialog plan scheme about relationship therebetween by classifying the dialog knowledge based on a task unit to make design of a dialog service convenient, which is different from an existing spoken dialog system in which it is difficult to reuse dialog knowledge since a large amount of construction costs and time is required.

Claims

1. A spoken dialog system based on dual dialog management using a hierarchical dialog task library for a system response suitable for a user utterance input, the system comprising:

a dialog knowledge database (DB) to store dialog knowledge required to achieve a dialog goal of a domain task;

a hierarchical task based dialog knowledge managing unit to update the dialog knowledge DB with subtask dialog knowledge that is constructed or newly generated by a dialog service designer using subtask dialog knowledge; and

a spoken dialog processing engine unit to perform a dialog with a user by repeatedly performing a process of recognizing a user utterance as a user utterance sentence, generating a system utterance text referring to the dialog knowledge DB in order to achieve the dialog goal of the domain task, and thereby outputting the generated system utterance text using voice.

2. The system of claim 1, wherein the spoken dialog processing engine unit comprises:

a speech recognizing unit to generate the user utterance sentence by transcribing the user utterance;

a language understanding unit to generate a several candidates for user utterance intention that the user desires to speak by analyzing the user utterance sentence as meaning representation;

a system dialog managing unit to search for a system utterance intention and pattern by referring to the dialog knowledge DB with respect to meaning representation of several candidates for user utterance intention;

a user dialog managing unit to search the dialog knowledge DB for the subsequent several candidates for user utterance intention after the system dialog managing unit generates the system utterance intention and pattern after a current user utterance;

a dialog generating unit to search the dialog knowledge DB for the system utterance intention and pattern selected by the system dialog managing unit or the user dialog managing unit and a dialog pattern with respect to the several candidates for user utterance intention, and to generate a system utterance text using the found dialog pattern; and

a speech synthesizing unit to output the generated dialog text using the voice.

3. The system of claim 1, wherein the hierarchical task based dialog knowledge managing unit comprises:

a hierarchical task dialog knowledge library to hierarchically include the dialog knowledge stored in the dialog knowledge DB, dialog knowledge of a subtask of a domain task constructed by a designer, and dialog knowledge of an upper subtask; and

a dialog knowledge constructing unit to update the dialog knowledge DB and the hierarchical task dialog knowledge library with dialog knowledge that is newly constructed or generated by the designer based on existing dialog knowledge of the hierarchical task dialog knowledge library.

4. The system of claim 1, wherein the dialog knowledge DB comprises:

a hierarchical task flow graph DB to store a hierarchical task flow graph indicating a flow between subtasks for solving the domain task;

a slot system DB to store a slot system in which meanings that need to be mutually known between a user and a system are classified in order to achieve a goal of the domain task;

a dialog library DB to store a dialog library that is an aggregate of utterance patterns classified into an intention of the user and an intention of the system in order to generate a system speech in a predetermined situation in the domain task or to recommend the subsequent user speech candidate; and

an application domain knowledge DB to store application domain knowledge in which knowledge that is to be known by the system in order to achieve a goal of an application domain task is databased.

5. The system of claim 4, wherein the hierarchical task flow graph DB stores dialog flow graph knowledge expressing a time series context between subtasks for performing the domain task and flow information between subtasks according to a transition condition, and uses an integrated subtask as a task of an upper layer by integrating the subtasks.

6. The system of claim 4, wherein the dialog library DB includes a response dialog library that is a utterance pattern associated with a response to a dialog of a responder and a corresponding utterance intention, a utterance pattern requesting a predetermined slot value to user or system, a progress dialog library constructed the utterance pattern, and a subtask transition dialog library in which a utterance intention and a utterance pattern for each speaker in the case of start, end, or restart of a subtask are constructed.

7. The system of claim 2, wherein the system dialog managing unit comprises:

a user intention selecting block to select the best user intention among the user intention candidates using dialog history;

a user intention processing block to change the dialog history based on a dialog act type of the selected user utterance intention;

a system response utterance generating block to select a utterance pattern suitable for a dialog history using an example based scheme by searching a dialog library DB of the dialog knowledge DB for a response to the selected user utterance intention or a corresponding system speech intention;

a system progress utterance generating block to inquire the user about content in which a goal desired by the user is embodied, by inquiring the user about relevant slots required to embody a value of a target slot for completing a goal of a subtask currently in progress; and

a system transition utterance generating block to find a system utterance pattern by searching the dialog library DB for a transition dialog with respect to a current subtask using a system speech intention informing the user about a progress state of the subtask at a point in time of start, end, or restart of the subtask.

8. The system of claim 7, wherein the system dialog managing unit further comprises:

a dialog history including dialog section history that stores a speech intention between the user and the system in a dialog progress order using dialog flow information, subtask history that stores subtask progress history, and slot value history that describes a slot value of a slot that is transmitted and received between the user and the system and a state thereof; and

a dialog history management block to manage the history of the dialog history based on each user intention or system intention of the user intention processing block, the system response utterance generating block, the system progress utterance generating block, or the system transition utterance generating block.

9. The system of claim 7, wherein the system progress utterance generating block generates system utterance inquiring slot value for achieving purpose of subtask based on the average dialog turn number required to complete the subtask by initially inquiring the user about relevant slots having a large ambiguous value by referring to the application domain knowledge DB of the dialog knowledge DB, or by transferring, to the user dialog managing unit, the system speech intention to inquire the user about, in order to embody the value of the target slot of the subtask.

10. The system of claim 7, wherein the user dialog managing unit comprises:

a user response utterance generating block to generate a user utterance intention and pattern using an example based scheme by searching a response dialog of the dialog library DB of the dialog knowledge DB for a response to a system speech intention generated by the system dialog managing unit or a corresponding user intention;

a user progress utterance generating block to inquire the user about content in which a goal is embodied in order to complete a goal of a current subtask;

a user transition utterance generating block to generate a user utterance intention and pattern by searching a task transition dialog of the dialog library DB for a user utterance for verifying the content currently selected by the user or proceeding with a new subtask in the case of start or end of the subtask; and

a subtask complete estimation block to verify how many dialog turns a currently generated user utterance intention further proceeds based on a response of a system until the goal of the subtask is completed, by repeatedly executing the system dialog managing unit and the user dialog managing unit.

11. The system of claim 10, wherein the user dialog managing unit further comprises:

a temporary dialog history including dialog section history that stores a speech intention between the user and the system in a dialog progress order using dialog flow information, subtask history that stores subtask progress history, and slot value history that describes a slot value of a slot that is transmitted and received between the user and the system and a state thereof; and

a temporary dialog history management block to manage the history of the temporary dialog history based on a dialog context that is changed based on the user utterance intention through executing of the user response utterance generating block, the user progress utterance generating block, the user transition utterance generating block, or the subtask complete estimation block.