Distributed natural language understanding and processing using local data sources

• US 9,171,066 B2
• Filed: 11/12/2012
• Issued: 10/27/2015
• Est. Priority Date: 11/12/2012
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Hebert, Matthieu, Robichaud, Jean-Philippe, Parisien, Christopher M.
• Primary Examiner(s): WOZNIAK, JAMES S
• Application Number:
  US13/674,191
• Publication Number:
  US 20140136183A1
• Time in Patent Office: 1,079 Days
• Field of Search: 704/9, 704/257, 704/270.1, 704/E15.047
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/085 Methods for reducing search complexity, pruning
• CPC Class Codes: G10L 2015/228 of application context

Abstract

An arrangement and corresponding method are described for distributed natural language processing. A set of local data sources is stored on a mobile device. A local natural language understanding (NLU) match module on the mobile device performs natural language processing of a natural language input with respect to the local data sources to determine one or more local interpretation candidates. A local NLU ranking module on the mobile device processes the local interpretation candidates and one or more remote interpretation candidates from a remote NLU server to determine a final output interpretation corresponding to the natural language input.

Claims

1. A method comprising:

configuring a client device to;

process one or more natural language inputs with respect to data sources stored on the client device to determine a first set of interpretation candidates for the one or more natural language inputs; and

to communicate, to a server, results from processing the one or more natural language inputs with respect to the data sources stored on the client device;

determining, by the server and based on the results from processing the one or more natural language inputs with respect to the data sources stored on the client device, a list of possible interpretation candidates for the one or more natural language inputs, the list comprising a second set of interpretation candidates for the one or more natural language inputs;

ranking, by the server, the list of possible interpretation candidates;

pruning, by the server, the list of possible interpretation candidates;

constraining, by the server and based on pseudo data corresponding to the data sources located on the client device, the pruning to prevent at least one interpretation candidate of the second set of interpretation candidates from being pruned from the list of possible interpretation candidates; and

communicating, by the server and to the client device, the second set of interpretation candidates for the one or more natural language inputs, for a final output interpretation of the one or more natural language inputs by the client device that comprises ranking a plurality of interpretation candidates comprising the first set of interpretation candidates and the second set of interpretation candidates.

2. The method of claim 1, wherein the results comprise semantic-classification results from processing the one or more natural language inputs with respect to the data sources stored on the client device.

3. The method of claim 1, wherein the data sources comprise at least one of a music list stored on the client device, email content stored on the client device, meeting content stored on the client device, or a contact list stored on the client device.

4. The method of claim 1, further comprising ranking, by the server, the second set of interpretation candidates.

5. The method of claim 4, further comprising configuring the client device to re-rank the second set of interpretation candidates.

6. The method of claim 1, wherein the client device comprises a mobile device remotely located from the server.

7. A system comprising:

at least one processor; and

a memory storing instructions that when executed by the at least one processor cause the system to;

configure a client device to;

process one or more natural language inputs with respect to data sources stored on the client device to determine a first set of interpretation candidates for the one or more natural language inputs; and

to communicate, to the system, results from processing the one or more natural language inputs with respect to the data sources stored on the client device;

determine, based on the results from processing the one or more natural language inputs with respect to the data sources stored on the client device, a list of possible interpretation candidates for the one or more natural language inputs, the list comprising a second set of interpretation candidates for the one or more natural language inputs;

rank the list of possible interpretation candidates;

prune the list of possible interpretation candidates;

constrain, based on pseudo data corresponding to the data sources located on the client device, the pruning to prevent at least one interpretation candidate of the second set of interpretation candidates from being pruned from the list of possible interpretation candidates; and

communicate, to the client device, the second set of interpretation candidates for the one or more natural language inputs, for a final output interpretation of the one or more natural language inputs by the client device that comprises ranking a plurality of interpretation candidates comprising the first set of interpretation candidates and the second set of interpretation candidates.

8. The system of claim 7, wherein the results comprise semantic-classification results from processing the one or more natural language inputs with respect to the data sources stored on the client device.

9. The system of claim 7, wherein the data sources comprise at least one of a music list stored on the client device, email content stored on the client device, meeting content stored on the client device, or a contact list stored on the client device.

10. The system of claim 7, wherein the instructions, when executed by the at least one processor, further cause the system to rank the second set of interpretation candidates.

11. The system of claim 10, wherein the instructions, when executed by the at least one processor, further cause the system to configure the client device to re-rank the second set of interpretation candidates.

12. The system of claim 11, wherein the client device comprises a mobile device remotely located from the system.

13. One or more non-transitory computer-readable media having instructions stored thereon that when executed by one or more computers cause the one or more computers to:

configure a client device to;

process one or more natural language inputs with respect to data sources stored on the client device to determine a first set of interpretation candidates for the one or more natural language inputs; and

to communicate, to the one or more computers, results from processing the one or more natural language inputs with respect to the data sources stored on the client device;

determine, based on the results from processing the one or more natural language inputs with respect to the data sources stored on the client device, a list of possible interpretation candidates for the one or more natural language inputs, the list comprising a second set of interpretation candidates for the one or more natural language inputs;

rank the list of possible interpretation candidates;

prune the list of possible interpretation candidates;

constrain, based on pseudo data corresponding to the data sources located on the client device, the pruning to prevent at least one interpretation candidate of the second set of interpretation candidates from being pruned from the list of possible interpretation candidates; and

communicate, to the client device, the second set of interpretation candidates for the one or more natural language inputs, for a final output interpretation of the one or more natural language inputs by the client device that comprises ranking a plurality of interpretation candidates comprising the first set of interpretation candidates and the second set of interpretation candidates.

14. The one or more non-transitory computer-readable media of claim 13, wherein the results comprise semantic-classification results from processing the one or more natural language inputs with respect to the data sources stored on the client device.

15. The one or more non-transitory computer-readable media of claim 13, wherein the data sources comprise at least one of a music list stored on the client device, email content stored on the client device, meeting content stored on the client device, or a contact list stored on the client device.

16. The one or more non-transitory computer-readable media of claim 13, wherein the instructions, when executed by the one or more computers, further cause the one or more computers to rank the second set of interpretation candidates.

17. The one or more non-transitory computer-readable media of claim 16, wherein the instructions, when executed by the one or more computers, cause the one or more computers to configure the client device to re-rank the second set of interpretation candidates.

18. The one or more non-transitory computer-readable media of claim 13, wherein the client device comprises a mobile device remotely located from the one or more computers.