Method for managing mixed initiative human-machine dialogues based on interactive speech

• US 20050010393A1
• Filed: 10/25/2002
• Published: 01/13/2005
• Est. Priority Date: 10/30/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Loquendo SpA (Microsoft Corporation)
• Inventors: Rullent, Claudio, Danieli, Morena
• Granted Patent: US 7,499,861 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Method for managing mixed-initiative human-machine dialogues based on speech interaction exploiting the separation between a general dialogue knowledge, such as communicative acts, which can be used in multiple application domains, and particular linguistic knowledge, which are domain-specific parameters, to process the dialogue as a sequence of changes of status. Each status consist in a set of features linked both to the processed parameters and to the linguistic and pragmatic context, and describes a certain instant of the communicative situation between the user and the system so to discriminate it from other situations that are also only slightly different. The method employs three components. A first component which, given the various parameters of the domain, defines the parameters on which to intervene to modify the status with the intent of converging towards a situation in which all parameters are acquired with a certain value; in parallel, the component identifies the Communicative Act (CA) which applied to these parameters can make the status evolve in the required direction. A second component creates the sentences to be conveyed to user, whereby obtaining a Communicative Act by instancing said parameters. A third component analyses the user'"'"'s reply to determine the new system status given the parameters that were provided by the user, their mutual coherence, the previous status of these parameters and other correlated parameters.

Claims

1. Method for managing mixed-initiative human-machine dialogues based on speech interaction for providing a service to a human user in a particular context of domain in which the sentences uttered by the user are modelled in terms of simple or structured informative elements (parameters) while the exchanges of the machine or system are expressed by means of Communicative Acts (CA), characterised in that the dialogue is processed as status change sequence, comprising the whole of the characteristics linked both to the processed parameters and to the linguistic and pragmatic context, this whole describing a certain instant of the communicative situation between the user and the system so as to discriminate it from other situations which are also only slightly different employing the following three main components:

a. a first component which, given the various parameters of the domain, defines the parameters on which to intervene to modify the status with the intent of converging towards a situation in which all parameters are acquired and confirmed;

in parallel, the component identifies the Communicative Act (CA) which applied to these parameters can make the status evolve in the required direction;

b. a second component for creating the sentences to be conveyed to user;

the Communicative Act is obtained by instancing said parameters;

c. a third component which analyses the user'"'"'s reply to determine the new system status given the parameters which were provided by the user, their mutual coherence, the previous status of these parameters or other correlated parameters.

2. Method according to claim 1, characterised in that the semantic concepts and the parameters related to the information contained in the user'"'"'s utterance are firstly analysed to obtain a possibly formal representation of its meaning, the specific information content and to interpret the specific information content by collocating it in a context according to the nature of the involved knowledge, linguistic and contextual, respectively, in which the parameter, either alone or with an associated value, provides relevant information for defining the user'"'"'s needs.

3. Method according to claim 1, characterised in that a set of two or more Communicative Acts (CA) selected from the following are employed to realise human-machine dialogue interfaces which exploit knowledge domain and dialogue separation:

a. CA_INFO;

system provides preliminary information on the potentials of the service and the interaction method and introduces the user to the service;

b. CA_REQUEST;

system requests one or more parameters;

c. CA_VERIFY;

system verifies the value assumed by one or more parameters during the interaction;

d. CA_REQUEST_AND_VERIFY;

system asks for one or more parameters and verifies the value assumed by one or more parameters during the interaction;

e. CA_OPEN_OPTION;

system proposes an option to continue the interaction leaving the choice of the initiative to the user;

f. CA_PHATIC;

system generates messages to keep the communication channel open;

g. CA_SELECT;

in the case of ambiguous information, system informs the user of the ambiguity and selects the correct value;

h. CA_READ;

system reads information retrieved in database according to the needs expressed by the user during the interaction;

i. CA_CLOSURE;

system ends the communication.

4. Method according to claim 1, characterised in that said parameters include simple parameters which convey a complete, atomic piece of information corresponding to a concept expressed in the user'"'"'s utterance to which values, normally consisting of recognised words or symbolic values, are associated.

5. Method according to claim 1, characterised in that said parameters include generalised parameters which generalise two or more parameters of the application and are used to process underdetermined information classes or generic terms and to which the various parameters of the dialogue system are explicitly associated which the generalised parameters may become once the ambiguity has been cleared.

6. Method according to claim 1, characterised in that said parameters include structured parameters consisting of sets of simple interdependent parameters which may be either simultaneously or exclusively present.

7. Method according to claim 1, characterised in that specific domain knowledge is associated to said structured parameters to run check procedures on their correct formation, on the links between component parameters and on the criteria for assessing whether a set of parameters, with certain associated values may create an instance of the structured parameter or not.

8. Method according to claim 1, characterised in that said parameters are defined as follows according to their inherent properties:

a. “

certain”

parameters are those which do not need to be confirmed after being acquired and can be considered certain either on the basis of knowledge;

b. “

offered value”

parameters are those for which a standard value can be provided, which can be considered acquired and for which immediate confirmation can be asked on the basis of the knowledge related to the application;

c. “

associated”

parameters are those which can have the same type of value but whose acquired value cannot be the same for the application;

d. “

delay”

parameters are pairs of parameters that the developer decides to delay so that confirmation of the first will only be asked once the second has been acquired;

e. “

necessary”

parameters are those which confirm a structured parameter consisting of “

n”

parameters.

9. Method according to claim 1, characterised in that the domain and the structure of the dialogue are represented online in a non-hierarchic way and in that the contextual information is represented by independent single level structures.

10. Method according to claim 9, characterised in that said independent single level structures contain triads (General Parameter Status, SGP) formed by the name of the parameters, the value associated to the parameter communicated by the user and the status which is progressively updated for each parameter during the course of the interaction.

11. Method according to claim 9, characterised in that said independent single level structures contain one of three values (CURRENT-SITUATION) which characterise specific interaction patterns, namely not affected parameter, acquired parameter, confirmed parameter, for each parameter during the course of the interaction.

12. Method according to claim 1, characterised in that one of three possible transient statuses, namely unchanged parameter, negated parameter, replaced parameter, is associated to the input parameters.

13. Method according to claim 1, characterised in that the parameter status is associated to the number of exchanges spent to make a parameter reach a certain status, determined on the basis of metalevel variables for solving case of ambiguity.

14. Method according to claim 13, characterised in that said metalevel variables are of the following type:

CA_SELECT is applicable to solve incoherence of a structured parameter (ML_SELECT_INCONSISTENCY) and incoherence between two parameters (ML_SELECT_ALL);

defining whether either acquisition of a parameter or set of parameters must be immediately followed by a confirmation request (ML_HOLES_ALLOWED=NO) or the interaction can proceed also without confirming the parameters (ML_HOLES_ALLOWED=YES);

detected incoherence in a structured parameter is solved either by negotiating the correct value (ML_STRUCT_INCONS=NEGOTIATE) or eliciting a new value for the parameter (ML_STRUCT_INCONS=DRASTIC);

the choice of a CA_VERIFY for a certain parameter must be postponed until a new parameter specified by the developer has been acquired (ML_COND_VERIFY=YES, NO);

to rerun the application task (ML_TASK_REDO=YES, NO);

to specify which parameters are involved in the rerun (ML_PARAM_REDO=one or more parameter names);

to specify the names of the parameters for which confirmation can be delayed (ML_DELAY_VERIFY=list of parameter name pairs);

to specify whether a parameter has a threshold (ML_THRESHOLD_REQ_GEN=YES) or not (ML_THRESHOLD_REQ_GEN=NO);

to specify the type of task (ML_TASK_LEVEL=SINGLE_TASK, MULTI_TASK, MENU, DATA_BROWSE).

15. Method according to claim 1, characterised in that a structured intended to contain the informative contents of single tasks which can form a domain (General Task Status, SGT) is realised, in which each active task can assume the following values:

not active task (PENDING);

task in progress (RUNNING);

completed task (EXECUTED);

failed task (FAILED);

normal task in progress (RUNNING_DEFAULT).

16. Method according to claim 1, characterised in that a module is realised to process the domain-specific knowledge to be made available to the three main components, comprising a set of structures for storing linguistic knowledge ordered according to parameters, communicative acts and metalevel variables for processing check procedures related to said structured variables.

17. Method according to claim 1, characterised in that the initial choice of the knowledge is made by means of the following steps:

1. The parameters names are selected;

the parameter names will be used to classify information provided by the user and to categorise the information (simple parameters, structured parameters;

certain, negated or associated value parameters);

2. Procedures are associated to non-simple parameters;

3. The following information is classified according to the given communicative acts (CA) categories;

the parameters involved in the CA, the sentences to be used to request, confirm, etc., said parameters, the number of times that the communicative act can be uttered, the utterance contexts of the sentence;

4. The metalevel variables and the names of the acts addressed to external dialogue resources are stated and procedures, where required, and names of the involved parameters are associated;

5. The possible parameter subsets intended as specific domain tasks are isolated and a name is assigned.

18. Method according to claim 1, characterised in that application-specific linguistic knowledge is stored in the following structures:

a structure for the parameter names (Param);

a structure for the check procedure names possibly associated to each parameter (ParamProc);

a structure for possible tasks (Task);

a structure for communicative acts (CA) and associated information (CommAct);

a structure for the names of the acts to be performed with reference to external acts (ExtAct);

a structure for the names of the parameters possibly associated to each act (ActDB);

a generic structure for possible metalevel options (Sdm1);

a structure for the formal representation of the sentence uttered by the user of each exchange of the dialogue (DeepSemDialogue, DSD).

19. Method according to claim 1, characterised in that dialogue management is activated by the reception of an external request which can come from the user or from the application and which performs the following operations:

1. activation upon reception of a request;

domain task initialisation by verifying whether the compiled application knowledge requires the management of a single task (step 110) or several tasks;

in the latter case, the system initialises the parameter subset forming the current task (step 111);

2. request classification (step 112);

this can come either from the user (step 113) or from the application (step 114);

step 113 interprets the user'"'"'s request according to the context determined by the latest realised CA, the parameters contained in the user'"'"'s request and the current parameter status and task status;

in the second case, the application request is interpreted according to the latest realised CA and the contents of the information from the application;

3. choice of communicative act to be generated according to the context determined by the application parameter status and task status (step 115);

4. generation of instanced communicative act by selecting a configuration whose applicability threshold is not saturated (step 116);

5. standby or end of routine (step 117).

20. Method according to claim 19, characterised in that the value of the metalevel variable (ML-MULTI-TASK) specifying the task type (step 110) is checked during the first operation:

the system immediately goes to request classification in the case of SINGLE-TASK (step 112);

in the case of multiple tasks, the general task structure (SGT) is initialised and a non active (PENDING) status is assigned to the specified step (step 111).

21. Method according to claim 19, characterised in that the following steps are carried out during the second operation:

the system discriminates whether the dialogue module consists of a sentence uttered by the user or consists of a message sent by other system modules (step 112),

the main algorithm for interpreting the user'"'"'s request and managing confirmation and uncertainty is run (step 113);

the system message is analysed to discover the source and the contents and to run, where needed, simple data structure and dialogue module internal variable initialisation actions (step 114).

22. Method according to claim 19, characterised in that the following steps are performed to interpret the user'"'"'s sentence:

the contents of said structure for formally representing the sentence uttered by the user (DeepSemDialogue, DSD), the communicative act instanced during the previous exchanged and the involved parameters are considered;

if said structure (DSD) is empty (step 121), a contextual variable (CURRENT_SITUATION) is initialised with a first value (NOT_UND) (step 122) and interpretation of the user'"'"'s sentence ends (step 123);

if said structure (DSD) is not empty, the following checks are run starting from the current communicative act for each parameter being either requested (step 125) and/or verified (step 126);

the concerned parameter is sought for in the DSD (step 124);

if present, its value is analysed with respect to possible general parameter status (SGP) value and status;

if none of the concerned parameters are present, the DSD content is declared useless, either by attributing the value NO_CONT to the CURRENT_SITUATION variable at the end of the interpretation step (step 127) or by processing the offered parameters;

the compatibility of the concerned parameter is checked against the values of the other associated parameters in the SGP;

the interpretation will continue if the check is positive;

the concerned parameters status will be deemed uncertain, which will be solved by activating a specific CA_SELECT if the result is negative;

—

the structured parameter coherence of the concerned parameter will be checked if the parameter is the component a structured parameter;

interpretation will continue if the check result is positive;

the concerned parameters status will be deemed uncertain, which will be solved either by activating a specific CA_SELECT or by acquiring a new parameter, if the check result is negative;

the status and the value of the concerned parameter in the respective triad (SGP) if modified, if required;

the parameters offered by the user but not requested are sought for in the structure (DSD) (step 128) and the following checks are run;

The compatibility of the offered parameter is checked against the value offered by the other parameters to which it is associated in the SGP;

interpretation will continue if the check result is positive to acquire the parameter;

if the result is negative, the offer will be rejected if it is incompatible with the previously confirmed associated parameters;

otherwise the concerned parameter status is deemed uncertain;

The structured parameter coherence of the concerned parameter will be checked if the parameters is a component of a structured parameter by applying the functions defined to verify whether the component values can coexist;

interpretation will continue if the check result is positive;

the parameter is rejected if the result is negative;

The status and the value of the concerned parameter in the respective triad (SGP) if modified.

23. Method according to claim 19, characterised in that the following operations are performed to select the communicative act to be generated (step 115):

specific resources are checked (database, DB) for information pertinent to the interaction in progress with the user;

the user is put on standby and CA_PHATIC is selected (step 132) when waiting for an answer from said resources following a request that blocks the interaction with the user (step 131);

the interaction is closed and a CA_CLOSURE is selected (step 134) if an interaction error with said resources (DB) occurred (step 133);

the availability of data satisfying the request is verified if all parameters required for the current application task have been acquired (step 135);

otherwise the system returns to the parameter acquisition algorithm by selecting a CA_REQUEST (step 136);

the interaction proceeds by reading the data (step 138) if the data has been retrieved from said resources (DB) (step 137);

if said resources (DB) do not contain data for the current request, the system checks whether either a new request for reacquiring one or more parameters has been run or access to external resources has been made on the basis of predefined values (step 139);

the system checks whether the knowledge compiled for the current application include task rerunning by checking the value attributed to ML_TASK_REDO (pass 141);

if affirmative, a CA_OPEN_OPTION is selected and the user is prompted to repeat the task (step 142);

if negative, the algorithm selects a CA_CLOSURE to end the interaction (step 143).

24. Method according to the previous claims, characterised in that one of the following three modes are applied to solve cases of underdetermined CA_VERIFY utterances if the user responds without providing informative elements useful for confirming two or more parameters (p1, p2, . . . ):

a CA_VERIFY applied to a first parameter (p1) is selected;

the parameter value is updated to DENIED if the user does not confirm the value of the first parameter and the status of the respective triad (SGP) is changed either to NOT_AFFECTED or to ACQUIRED, according to whether an alternative value was supplied or not;

otherwise, the second parameter (p2) in the respective triad (SGP) is deleted and a new request is made by selecting a CA_REQUEST if the user confirms the first parameter (p1);

if more than two parameters are being verified, confirmation of the last parameter will not be required once the user has confirmed all previous parameters;

the values of the parameters being verified are rejected beforehand and the user is asked to repeat the values, possibly one at a time;

it is hypothesised that only one of the parameters is wrong and the user is asked to choice one of two alternative values;

a CA_SELECT is generated and;

CONFIRMED status is assigned to both parameters if the previous sentence is confirmed,

the value of the parameters in the respective triads (SGP) is changed and ACQUIRED status is assigned if alternative values are provided for both parameters,

if only one parameter is confirmed, the non confirmed parameter is deleted from the triad (SGP) and a new acquisition is started.