Method for managing mixed initiative human-machine dialogues based on interactive speech

• US 7,499,861 B2
• Filed: 10/25/2002
• Issued: 03/03/2009
• Est. Priority Date: 10/30/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Loquendo SpA (Microsoft Corporation)
• Inventors: Rullent, Claudio, Danieli, Morena
• Primary Examiner(s): Edouard; Patrick N
• Assistant Examiner(s): Yen; Eric
• Application Number:
  US10/494,472
• Publication Number:
  US 20050010393A1
• Time in Patent Office: 2,321 Days
• Field of Search: 704/1, 704/9, 704/251, 704/257, 704/E15.04, 704/270, 704/275, 379/88.18
• US Class Current: 704/275
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Method for managing mixed-initiative human-machine dialogues based on speech interaction exploiting the separation between general dialogue knowledge, such as communicative acts, which can be used in multiple application domains, and particular linguistic knowledge, which are domain-specific parameters, to process the dialogue as a sequence of changes of status. Each status consist in a set of features linked both to the processed parameters and to the linguistic and pragmatic context, and describes a certain instant of the communicative situation between the user and the system so to discriminate it from other situations that are also only slightly different. The method employs three components. A first component which, given the various parameters of the domain, defines the parameters on which to intervene to modify the status with the intent of converging towards a situation in which all parameters are acquired with a certain value; in parallel, the component identifies the Communicative Act (CA) which applied to these parameters can make the status evolve in the required direction. A second component creates the sentences to be conveyed to user, whereby obtaining a Communicative Act by instancing said parameters. A third component analyses the user'"'"'s reply to determine the new system status given the parameters that were provided by the user, their mutual coherence, the previous status of these parameters and other correlated parameters.

Claims

1. A method of managing mixed-initiative human-machine dialogues based on speech interaction for providing a service to a human user in a particular context or domain in which sentences uttered by the user are modeled in terms of parameters that are simple or structured informative elements while the exchanges of the machine are expressed by means of communicative acts, the method comprising the steps of:

processing the dialogue as a sequence of changes of parameter status that is a set of characteristics of the parameters dependent on the linguistic and pragmatic context, this set describing a certain instant of the communicative situation between the user and the machine;

selecting from the parameters of the domain, the parameters on which to intervene to modify the status with the intent of converging towards a situation in which all the selected parameters are acquired and confirmed;

identifying in parallel the communicative act that applied to the selected parameters makes the status evolve toward completion of the dialogue;

creating sentences to be conveyed to the user as a communicative act applied to the selected parameters; and

analyzing the user'"'"'s reply to determine a new status for the machine parameters, checking for mutual coherence among the simple or structured informative elements, their previous status, the status of parameters, and the value of the parameters provided by the user,

activating a dialogue system by the reception of an external request that can come from the user or from the application and which performs the following operations;

activation upon reception of a request;

domain task initialization by verifying whether the application knowledge requires the management of a single task or several tasks;

in the latter case, the system initializes the parameter subset forming the current task;

request classification;

interpreting a user'"'"'s request according to the context determined by the latest realized communicative acts, the parameters contained in the user'"'"'s request and the current parameter status and task status;

choice of a communicative act to be generated according to the context determined by the application parameter status and task status;

generation of instanced communicative act by selecting a configuration whose applicability threshold is not saturated;

standby or end of routine, the structure of the dialogue being represented by a general parameter status used to store parameter information, values associated with the parameters, and the status that is progressively updated for each parameter during the dialogue;

interpreting a user'"'"'s sentence by;

considering the contents of said structure of the dialogue, the communicative act issued to a user during the previous exchange and the involved parameters;

if said structure is empty, initializing a contextual variable with a first value and interpreting the user'"'"'s sentence ends;

if said structure is not empty, running the following checks starting from a selection of at least one communicative act applied to at least one parameter that is either requested and/or verified;

checking with the system if the at least one parameter is represented in the structure of the dialogue;

if it is represented, its value is analyzed with respect to the dialogue status represented in the general parameter status;

if the at least one parameter is present, the content of structure of the dialogue is considered useless as it pertains to the parameter by attributing the value NOCONT to the CURRENT_SITUATION variable at the end of the interpretation step, if more than one parameter is present in the structure of the dialogue, each one is processed for the at least one parameter;

checking the compatibility of the at least one parameter against the value of the at least one other associated parameter represented in the structure of the dialogue;

the interpretation will continue if the check is positive, otherwise the status of the at least one parameter and its associated parameter are deemed uncertain and such event will cause the selection of a communicative act CA_SELECT;

if the at least one parameter is part of a structured parameter, the coherence of the structured parameter will be checked on the basis of information represented by the at least one parameter;

interpretation will continue if the check result is positive, otherwise the concerned parameters status will be deemed uncertain, which will be solved either by activating a specific communicative act CA_SELECT or by acquiring a new parameter;

updating the status of the concerned parameter in the structure of the dialogue;

if a user'"'"'s sentence expresses at least one offered parameter that was not requested by the machine, the non-requested parameter is included in the structure and the following checks are run;

the compatibility of the offered parameter is checked against the value of at least one associated parameter;

interpretation will continue if the check result is positive and the offered parameter is represented in the structure of the dialogue, otherwise the offered parameter is rejected.

2. The method according to claim 1 wherein a set of two or more communicative acts selected from the following are employed to realize human-machine dialogue interfaces which exploit separation between domain and dialogue knowledge:

CA_INFO;

the machine provides preliminary information to introduce the user to the service;

CA_REQUEST;

the machine requests one or more parameters;

CA_VERIFY;

the machine verifies the value assumed by one or more parameters during the interaction;

CA_REQUEST_AND_VERIFY;

the machine asks for one or more parameters and verifies the value assumed by one or more parameters during the interaction;

CA_OPEN_OPTION;

the machine proposes to continue the interaction and the user can choose to continue or quit;

CA_PHATIC;

the machine generates messages to keep the communication channel open;

CA_SELECT;

in the case of ambiguous information, the machine informs the user of the ambiguity and selects the correct value;

CA_READ;

the machine reads information retrieved in database according to the needs expressed by the user during the interaction;

CA_CLOSURE;

the machine ends the communication.

3. The method according to claim 1 wherein said parameters include simple parameters that convey a complete, atomic piece of information corresponding to a concept expressed in a user'"'"'s utterance to which values consisting either of recognized words or symbolic values are associated.

4. The method according to claim 3 wherein said parameters include generalized parameters that provide a generalization of two or more parameters and that are used to process underdetermined pieces of information corresponding to a concept expressed in a user'"'"'s utterance.

5. The method according to claim 4 wherein said parameters include structured parameters consisting of sets of simple parameters that may be either simultaneously or exclusively present.

6. The method according to claim 5 wherein specific domain knowledge is associated with said structured parameters to run check procedures on the value acquired during the dialogue.

7. The method according to claim 1 wherein said parameters are defined as follows according to their properties:

“

certain”

parameters are those which do not need to be confirmed after being acquired and can be considered certain either on the basis of knowledge;

“

offered value”

parameters are those for which a standard value can be provided, which can be considered acquired, and for which immediate confirmation can be asked on the basis of the knowledge related to the application;

“

associated”

parameters are those that can have the same type of value but whose acquired value cannot be the same for the application;

“

delay”

parameters are pairs of parameters that the developer decides to delay so that confirmation of the first will only be asked once the second has been acquired;

“

necessary”

parameters are those which confirm a structured parameter consisting of “

n”

parameters.

8. The method according to claim 1 wherein the current situation of at least one parameter is augmented by information about the number of exchanges spent to make a parameter reach a status of not-affected, acquired, or confirmed.

9. The method according to claim 8 wherein information about the current situation of the parameters is augmented by information represented by metalevel variables whose values give rise to the following different behavior of the machine:

if a metalevel variable ML_SELECT_INCONSISTENCY is on, the machine selects CA_SELECT communicative act to solve incoherence of a structured parameter;

if a metalevel variable ML_SELECT_ALL is on, the machine selects a CA_SELECT communicative act to solve incoherence between two parameters;

if the value specified for ML_STRUCT_INCONS metalevel variable is NEGOTIATE, an incoherence detected in a structured parameter is solved by prompting the user for a correct value;

if the value specified for ML_STRUCT_INCONS metalevel variable is DRASTIC, an incoherence detected in a structured parameter is solved by again asking the parameter value of the user;

if the value of the metalevel variable ML_COND_VERIFY is YES, the choice of a CA_VERIFY for a certain parameter must be postponed until a predetermined new parameter has been asked of the user;

if the value of the metalevel variable ML_TASK_REDO is YES, the dialogue task is rerun and, if the value is NO, the dialogue task cannot be rerun;

the ML_DELAY_VERIFY metalevel variable is valued with the list of names of the parameters whose confirmations can be delayed.

10. The method according to claim 1 wherein a structure representing the status of different tasks is realized that is one of the following:

nonactive task labeled as “

pending,

task in progress labeled as “

running”

or “

running default,”

task completed with success labeled as “

executed, or

task completed with failure labeled (failed).

11. The method according to claim 1, further comprising the step of

processing the domain-specific knowledge comprising a set of structures for storing linguistic knowledge ordered according to parameters, communicative acts and metalevel variables for processing check procedures related to said structured parameters.

12. The method according to claim 1, an initial choice of the knowledge is made by the following steps:

selecting the parameters names that will be used to classify information provided by the user and to categorize the information;

associating procedures with non-simple parameters;

classifying the following information according to the given communicative-act categories;

the parameters involved in the communicative act,

the sentences to be used to request, confirm, etc., said parameters,

the number of times that the communicative act can be uttered, and

the utterance contexts of the sentence;

stating the metalevel variables and the names of the acts addressed to external dialogue resources and associating them with procedures, where required, and names of the involved parameters;

isolating the parameter subsets intended as specific domain tasks and assigning respective names thereto.

13. The method according to claim 1 wherein application-specific linguistic knowledge is stored in the following structures:

a structure for the parameter names;

a structure for names of the check procedures associated with each parameter;

a structure for tasks;

a structure for communicative acts and associated information

a structure for the names of the acts to be performed with reference to software modules that do not pertain to the dialogue system;

a structure for the names of the parameters associated with each act;

a structure for the formal representation of the sentence uttered by the user of each exchange of the dialogue containing the concepts expressed in the user sentence represented as a set of argument-value pairs.

14. The method according to claim 1 wherein the value of the metalevel variable specifying the task type is checked during the first operation:

the system immediately goes to request classification in the case of SINGLE-TASK;

in the case of multiple tasks, the general task structure is initialized and a non active status is assigned to the specified step.

15. The method according to claim 1 wherein the following steps are carried out during the second operation:

the machine discriminates whether the dialogue module consists of a sentence uttered by the user or consists of a message sent by other system modules;

the main algorithm for interpreting the user'"'"'s request and managing confirmation and uncertainty is run;

the system message is analyzed to discover the source and the contents and to run internal variable initialization actions.

16. The method according to claim 1 wherein the following operations are performed to select the communicative act to be generated;

resources external to the dialogue system are checked for information pertinent to the interaction in progress with the user;

the user is put on standby and CA_PHATIC is selected when waiting for an answer from said resources following a request that blocks the interaction with the user;

the interaction is closed and a CA_CLOSURE is selected if an interaction error with said resources occurred;

the availability of information data satisfying the request is verified if all parameters required for the current application task have been acquired;

otherwise the system returns to the parameter acquisition algorithm by selecting a communicative act CA_REQUEST;

the interaction proceeds by reading the information data if the information has been retrieved from said resources;

if said resources do not contain information data for a current request, the system checks whether either a new request for reacquiring one or more parameters has been run or access to external resources has been made on the basis of predefined values;

the system checks whether the knowledge compiled for the current application include task rerunning by checking the value attributed to ML_TASK_REDO;

if affirmative, a communication act CA_OPEN_OPTION is selected and the user is prompted to rerun the task;

if negative, the algorithm selects a communication act CA_CLOSURE to end the interaction.

17. The method according to claim 1 wherein, if the user responds without providing informative elements useful for confirming two or more parameters, one of the following three modes is applied to solve cases of underdetermined communication:

a communication act CA_VERIFY applied to a first parameter is selected;

if the user does not confirm the value of the first parameter the parameter value in the structure of the dialogue is updated to DENIED and its status is represented either as ACQUIRED or NOT_AFFECTED, according to whether an alternative value was supplied or not;

if the user confirms the first parameter, a second parameter in the structure of the dialogue is deleted and a new request is issued to the user by selecting a communicative act CA_REQUEST;

if more than two parameters are being verified, confirmation of the last parameter will not be required once the user has confirmed all previous parameters;

the values of the parameters being verified are rejected beforehand and the user is asked to repeat the values one at a time;

it is hypothesized that only one of the parameters is wrong and the user is asked to choice one of two alternative values;

a CA_SELECT is generated and;

CONFIRMED status is assigned to both parameters if the previous sentence is confirmed,

the value of the parameters in the structure of the dialogue is changed and status ACQUIRED is assigned if alternative values are provided for both parameters,

if only one parameter is confirmed, the nonconfirmed parameter is deleted from the structure of the dialogue and a new parameter acquisition is started.