Method and apparatus for a multi I/O modality language independent user-interaction platform

• US 9,330,089 B2
• Filed: 11/06/2015
• Issued: 05/03/2016
• Est. Priority Date: 08/06/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Gandrabur, Simona, Buist, Eric, Dragoi, Andrei, Salimi, Alireza
• Primary Examiner(s): PULLIAS, JESSE SCOTT
• Application Number:
  US14/934,668
• Publication Number:
  US 20160070695A1
• Time in Patent Office: 179 Days
• Field of Search: 704/1, 704/9, 704/10, 704/257, 704270-275
• US Class Current: 1/1
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding

Abstract

Automated user-machine interaction is gaining attraction in many applications and services. However, implementing and offering smart automated user-machine interaction services still present technical challenges. According to at least one example embodiment, a dialogue manager is configured to handle multiple dialogue applications independent of the language, the input modalities, or output modalities used. The dialogue manager employs generic semantic representation of user-input data. At a step of a dialogue, the dialogue manager determines whether the user-input data is indicative of a new request or a refinement request based on the generic semantic representation and at least one of a maintained state of the dialogue, general knowledge data representing one or more concepts, and data representing history of the dialogue. The dialogue manager then responds to determined user-request with multi-facet output data to a client dialogue application indicating action(s) to be performed.

Claims

1. A method of automatically managing a dialogue with a user, the method comprising:

generating a generic semantic representation based upon received user-input data, the generic semantic representation being independent of a language and an input modality associated with the received user-input data, the generic semantic representation comprising at least one of a list of semantic slots and a sequence of nested semantic slots;

determining a user-intention based upon the received user-input data, the generic semantic representation and at least one of;

a maintained state of the dialogue, concept data representing one or more concepts, and history data representing history of the dialogue;

performing selection of a list of data items based on any of the concept data representing the one or more concepts and attribute data representing one or more attributes associated with the generic semantic representation; and

sending output data indicative of one or more actions for a dialogue application to perform, the one or more actions being determined based on a result of said determining the user-intention.

2. The method according to claim 1, wherein each slot of the semantic slots and the nested semantic slots includes a language-independent representation indicative of canonical meaning.

3. The method according to claim 2, wherein the language-independent representation includes a string of text characters.

4. The method according to claim 1, wherein the list of data items includes a hierarchical list of data items.

5. The method according to claim 1, wherein the selection is further performed based upon at least one of a rank and an instance associated with the list of data items.

6. The method according to claim 1, wherein a dialogue manager is configured to provide a measure of accuracy of the output data.

7. The method according to claim 1, wherein the input modality includes at least one of speech, text, touch, and computer device control.

8. The method according to claim 1, further comprising updating the maintained state of the dialogue based on the data representing the one or more attributes.

9. The method according to claim 1, wherein the selection is further based on at least one of the maintained state of the dialogue, the concept data representing the one or more concepts, and the history data representing the history of the dialogue.

10. An apparatus for automatically managing a dialogue with a user, the apparatus comprising:

a processor; and

a memory with computer code instructions stored thereon,

the processor and the memory, with the computer code instructions, being configured to cause the apparatus to;

generate a generic semantic representation based upon received user-input data, the generic semantic representation being independent of a language and an input modality associated with the received user-input data, the generic semantic representation comprising at least one of a list of semantic slots and a sequence of nested semantic slots;

determine a user-intention based upon the received user-input data, the generic semantic representation, and at least one of;

a maintained state of the dialogue, concept data representing one or more concepts, and history data representing history of the dialogue;

perform selection of a list of data items based on any of the concept data representing the one or more concepts and attribute data representing one or more attributes associated with the generic semantic representation; and

send output data indicative of one or more actions for a dialogue application to perform, the one or more actions being determined based on a result of said determining the user-intention.

11. The apparatus according to claim 10, wherein each slot of the semantic slots and the nested semantic slots includes a language-independent representation indicative of canonical meaning.

12. The apparatus according to claim 11, wherein the language-independent representation includes a string of text characters.

13. The apparatus according to claim 10, wherein the list of data items includes a hierarchical list of data items.

14. The apparatus according to claim 10, wherein the processor and the memory, with the computer code instructions, are configured to further cause the apparatus to perform selection based upon at least one of a rank and an instance associated with the list of data items.

15. The apparatus according to claim 10, wherein the processor and the memory, with the computer code instructions, are configured to further cause the apparatus to provide a measure of accuracy of the output data.

16. The apparatus according to claim 10, wherein the input modality includes at least one of speech, text, touch, and computer device control.

17. The apparatus according to claim 10, wherein the processor and the memory, with the computer code instructions, are configured to further cause the apparatus to update the maintained state of the dialogue based on the data representing the one or more attributes.

18. The apparatus according to claim 10, wherein the processor and the memory, with the computer code instructions, are configured to further cause the apparatus to perform selection based on at least one of the maintained state of the dialogue, the concept data representing the one or more concepts, and the history data representing the history of the dialogue.

19. A non-transitory computer-readable medium with software instructions stored thereon, the computer software instructions, when executed by a processor, cause an apparatus to:

generate a generic semantic representation based upon received user-input data, the generic semantic representation being independent of a language and an input modality associated with the received user-input data, the generic semantic representation comprising at least one of a list of semantic slots and a sequence of nested semantic slots;

determine a user-intention, based upon the received user-input data, the generic semantic representation, and at least one of;

a maintained state of the dialogue, concept data representing one or more concepts, and history data representing history of the dialogue;

perform selection of a list of data items based on any of the concept data representing the one or more concepts and attribute data representing one or more attributes associated with the generic semantic representation; and

send output data indicative of one or more actions for a dialogue application to perform, the one or more actions being determined based on a result of said determining the user-intention.

20. The non-transitory computer-readable medium according to claim 19, wherein each slot of the semantic slots and the nested semantic slots includes a language-independent representation indicative of canonical meaning.