Method and apparatus for a multi I/O modality language independent user-interaction platform

• US 9,214,156 B2
• Filed: 08/06/2013
• Issued: 12/15/2015
• Est. Priority Date: 08/06/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Gandrabur, Simona, Buist, Eric, Dragoi, Andrei, Salimi, Alireza
• Primary Examiner(s): PULLIAS, JESSE SCOTT
• Application Number:
  US13/960,755
• Publication Number:
  US 20150046168A1
• Time in Patent Office: 861 Days
• Field of Search: 704/1, 704/9, 704/10, 704233-257, 704270-275
• US Class Current: 1/1
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding

Abstract

Automated user-machine interaction is gaining attraction in many applications and services. However, implementing and offering smart automated user-machine interaction services still present technical challenges. According to at least one example embodiment, a dialogue manager is configured to handle multiple dialogue applications independent of the language, the input modalities, or output modalities used. The dialogue manager employs generic semantic representation of user-input data. At a step of a dialogue, the dialogue manager determines whether the user-input data is indicative of a new request or a refinement request based on the generic semantic representation and at least one of a maintained state of the dialogue, general knowledge data representing one or more concepts, and data representing history of the dialogue. The dialogue manager then responds to determined user-request with multi-facet output data to a client dialogue application indicating action(s) to be performed.

Claims

1. A method of automatically managing a dialogue with a user, comprising:

transforming, at a dialogue manager, user-input data received from a client dialogue application into a generic semantic representation, the generic semantic representation being independent of a language and an input modality associated with the user-input data;

determining, by the dialogue manager, whether the user-input data is indicative of a new request by the user or a refinement request by the user refining one or more previous requests by the user, based on the generic semantic representation and at least one of a maintained state of the dialogue, general knowledge data representing one or more concepts, and data representing history of the dialogue;

sending multi-facet output data, indicative of one or more actions for the client dialogue application to perform, the one or more actions being determined based on a result of said determining whether the generic semantic representation is indicative of the new request by the user or the refinement by the user refining the one or more previous requests by the user;

sending a query to a backend end system for retrieving data associated with the determined user-request, the query being generated based on the determined user-request and the maintained state of the dialogue;

receiving a response to the query from the backend system; and

updating a list of data items based on the response received from the backend system.

2. The method according to claim 1 further comprising generating the query, wherein generating the query includes:

generating a refinement query representing a refinement of another query previously sent to the backend system;

or

generating a new query.

3. The method according to claim 1 further comprising updating the history of the dialogue based on the generic semantic representation generated or the list of data updated.

4. The method according to claim 1 further comprising updating the maintained state of the dialogue based on attributes associated with the query generated.

5. The method according to claim 1, wherein determining whether the user-input data is indicative of the new request by the user or the refinement request by the user includes employing a correlation measure, the correlation measure being evaluated based on linguistic features, natural language understanding (NLU) features, the data representing history of the dialogue, dialogue context, dialogue scope, or the general knowledge.

6. The method according to claim 1 further comprising:

upon determining that a single data item is received within the response from the backend system, returning the single data item in the multi-facet output data;

upon determining that two or more data items are received within the response from the backend system, returning an indication of multiple matches in the multi-facet output data;

or

upon determining that no data item is received within the response from the backend system, returning an error message in the multi-facet output data.

7. The method according to claim 1, wherein the generic semantic representation generated includes a semantic representation of one or more concepts.

8. The method according to claim 1 further comprising:

upon determining that the user-input data is indicative of the refinement request by the user, filtering a list of data items based on the generic semantic representation; and

returning an indication of the result of filtering the list of the data items in the multi-facet output data.

9. The method according to claim 8, wherein the list of data items represents search results received from a backend system and said filtering is further based on at least one of the maintained state of the dialogue, general knowledge data representing one or more concepts, and data representing the history of the dialogue.

10. An apparatus for automatically managing a dialogue with a user, comprising:

a processor; and

a memory with computer code instructions stored thereon,

the processor and the memory, with the computer code instructions, being configured to cause the apparatus to;

transform user-input data received from a client dialogue application into a generic semantic representation, the generic semantic representation being independent of a language and an input modality associated with the user-input data;

determine whether the user-input data is indicative of a new request by the user or a refinement request by the user refining one or more previous requests by the user, based on the generic semantic representation and at least one of a maintained state of the dialogue, general knowledge data representing one or more concepts, and data representing history of the dialogue;

send multi-facet output data, indicative of one or more actions for the client dialogue application to perform, the one or more actions being determined based on a result of said determining whether the generic semantic representation is indicative of the new request by the user or the refinement by the user refining the one or more previous requests by the user;

send a query to a backend end system for retrieving data associated with the determined user-request, the query being generated based on the determined user-request and the maintained state of the dialogue;

receive a response to the query from the backend system; and

update a list of data items based on the response received from the backend system.

11. The apparatus according to claim 10, wherein the processor and the memory, with the computer code instructions, are configured to further cause the apparatus to:

generate a refinement query representing a refinement of another query previously sent to the backend system;

or

generate a new query.

12. The apparatus according to claim 10, wherein the processor and the memory, with the computer code instructions, are configured to further cause the apparatus to update the history of the dialogue based on the generic semantic representation generated or the list of data updated.

13. The apparatus according to claim 10, wherein the processor and the memory, with the computer code instructions, are configured to further cause the apparatus to update the maintained state of the dialogue based on attributes associated with the query generated.

14. The apparatus according to claim 10, wherein in determining whether the user-input data is indicative of the new request by the user or the refinement request by the user, the processor and the memory, with the computer code instructions, are configured to further cause the apparatus to employ a correlation measure, the correlation measure being evaluated based on linguistic features, natural language understanding (NLU) features, the data representing history of the dialogue, dialogue context, dialogue scope, or the general knowledge.

15. The apparatus according to claim 10, wherein the processor and the memory, with the computer code instructions, are configured to further cause the apparatus to:

upon determining that a single data item is received within the response from the backend system, return the single data item in the multi-facet output data;

upon determining that two or more data items are received within the response from the backend system, return an indication of multiple matches in the multi-facet output data;

or

upon determining that no data item is received within the response from the backend system, return an error message in the multi-facet output data.

16. The apparatus according to claim 10, wherein the generic semantic representation generated includes a semantic representation of one or more concepts.

17. The apparatus according to claim 10, wherein the processor and the memory, with the computer code instructions, are configured to further cause the apparatus to:

upon determining that the user-input data is indicative of the refinement request by the user, filter a list of data items based on the generic semantic representation; and

return an indication of the result of filtering the list of the data items in the multi-facet output data.

18. A non-transitory computer-readable medium having stored thereon a sequence of instructions which, when loaded and executed by a processor coupled to an apparatus, causes the apparatus to:

transform user-input data received from a client dialogue application into a generic semantic representation, the generic semantic representation being independent of a language and an input modality associated with the user-input data;

determine whether the user-input data is indicative of a new request by the user or a refinement request by the user refining one or more previous requests by the user, based on the generic semantic representation and at least one of a maintained state of the dialogue, general knowledge data representing one or more concepts, and data representing history of the dialogue;

send multi-facet output data, indicative of one or more actions for the client dialogue application to perform, the one or more actions being determined based on a result of said determining whether the generic semantic representation is indicative of the new request by the user or the refinement by the user refining the one or more previous requests by the user;

send a query to a backend end system for retrieving data associated with the determined user-request, the query being generated based on the determined user-request and the maintained state of the dialogue;

receive a response to the query from the backend system; and

update a list of data items based on the response received from the backend system.

19. A method of automatically managing a dialogue with a user, comprising:

transforming, at a dialogue manager, user-input data received from a client dialogue application into a generic semantic representation, the generic semantic representation being independent of a language and an input modality associated with the user-input data;

determining, by the dialogue manager, whether the user-input data is indicative of a new request by the user or a refinement request by the user refining one or more previous requests by the user, based on the generic semantic representation and at least one of a maintained state of the dialogue, general knowledge data representing one or more concepts, and data representing history of the dialogue;

sending multi-facet output data, indicative of one or more actions for the client dialogue application to perform, the one or more actions being determined based on a result of said determining whether the generic semantic representation is indicative of the new request by the user or the refinement by the user refining the one or more previous requests by the user;

upon determining that the user-input data is indicative of the refinement request by the user, filtering a list of data items based on the generic semantic representation; and

returning an indication of the result of filtering the list of the data items in the multi-facet output data.

20. The method according to claim 19, wherein the list of data items represents search results received from a backend system and said filtering is further based on at least one of the maintained state of the dialogue, general knowledge data representing one or more concepts, and data representing the history of the dialogue.

21. An apparatus for automatically managing a dialogue with a user, comprising:

a processor; and

a memory with computer code instructions stored thereon,

the processor and the memory, with the computer code instructions, being configured to cause the apparatus to;

transform user-input data received from a client dialogue application into a generic semantic representation, the generic semantic representation being independent of a language and an input modality associated with the user-input data;

determine whether the user-input data is indicative of a new request by the user or a refinement request by the user refining one or more previous requests by the user, based on the generic semantic representation and at least one of a maintained state of the dialogue, general knowledge data representing one or more concepts, and data representing history of the dialogue;

send multi-facet output data, indicative of one or more actions for the client dialogue application to perform, the one or more actions being determined based on a result of said determining whether the generic semantic representation is indicative of the new request by the user or the refinement by the user refining the one or more previous requests by the user;

upon determining that the user-input data is indicative of the refinement request by the user, filter a list of data items based on the generic semantic representation; and

return an indication of the result of filtering the list of the data items in the multi-facet output data.

22. The apparatus according to claim 21, wherein the list of data items represents search results received from a backend system, and wherein the processor and the memory, with the computer code instructions, are further configured to cause the apparatus to filter the list of data items based on at least one of the maintained state of the dialogue, general knowledge data representing one or more concepts, and data representing the history of the dialogue.