Distributed server system for language understanding

• US 9,965,465 B2
• Filed: 06/20/2017
• Issued: 05/08/2018
• Est. Priority Date: 08/31/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Liu, Xiaohu, Sarikaya, Ruhi
• Primary Examiner(s): Hang, Vu B
• Application Number:
  US15/627,774
• Publication Number:
  US 20170286404A1
• Time in Patent Office: 322 Days
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

Systems and methods for training and using a natural language understanding system are provided. More specifically, the systems and methods train a natural language understanding system utilizing a distributed network of feature extractors on features servers. Further, the systems and methods for using the natural language understanding system utilize a distributed network of features extractor on features servers. Accordingly, the systems and methods provide for a more accurate natural language understanding system, a more reliable natural language understanding system, and a more efficient natural language understanding system. Further, the systems and methods provide for natural language understanding systems with better development (including update ability), productivity, and scalability.

Claims

1. A language understanding system, the language understanding system comprising:

a trained language understanding server, the trained language understanding server comprising;

at least one processor; and

memory encoding computer executable instructions that, when executed by the at least one processor, perform a method comprising;

receiving a natural language input from a client device;

sending the natural language input to a plurality of feature extractors in response to receiving the natural language input;

receiving potential features from the plurality of feature extractors after sending the natural language input to the plurality of feature extractors;

evaluating the potential features to determine input features for the natural language input;

determining a semantic meaning of the natural language input based on the input features; and

sending a response to the client device that includes the semantic meaning of the natural language input.

2. The language understanding system of claim 1, wherein the plurality of feature extractors are each located on different feature servers, and wherein the trained language understanding server is separate from the different feature servers.

3. The language understanding system of claim 2, wherein at least one feature extractor of the plurality of feature extractors is an auxiliary natural language understanding system.

4. The language understanding system of claim 2, wherein the input features include domains, intents, and entities.

5. The language understanding system claim 4, wherein the input features further include a confidence score for each of the domains, the intents, and the entities.

6. The language understanding system of claim 1, wherein the plurality of feature extractors run in parallel.

7. The language understanding system of claim 1, wherein each feature extractor of the plurality of feature extractors has a different feature specialty.

8. The language understanding system of claim 2, wherein the trained language understanding server is trained for a specific application, wherein the specific application is at least one of:

a digital assistant application;

a voice recognition application;

an email application;

a social networking application;

a collaboration application;

an enterprise management application;

a messaging application;

a word processing application;

a spreadsheet application;

a database application;

a presentation application;

a contacts application;

a gaming application;

an e-commerce application;

an e-business application;

a transactional application;

an exchange application; and

a calendaring application.

9. A method for using a trained natural language understanding system, the method comprising:

receiving a natural language input from a client device;

sending the natural language input to a plurality of feature extractors,

wherein the plurality of feature extractors are each located on different feature servers,

receiving potential features for the natural language input from the plurality of feature extractors;

evaluating the potential features utilizing a trained language understanding model to determine input features for the natural language input; and

generating a response to the natural language input based on the input features.

10. The method of claim 9, wherein a language understanding server is separate from the different feature servers.

11. The method of claim 9, wherein at least one of the plurality of feature extractors is an auxiliary natural language understanding system.

12. The method of claim 9, wherein the input features comprise domains, intents, and entities and their associated confidence scores.

13. The method of claim 9, wherein each of the plurality of feature extractors run in parallel.

14. The method of claim 9, wherein each feature extractor of the plurality of feature extractors has a different feature specialty.

15. The method of claim 9, wherein the natural language input is preprocessed with a language understanding decoder on the trained language understanding model, and

wherein the natural language input sent to the plurality of feature extractors is preprocessed.

16. The method of claim 9, further comprising:

sending the response to the client device,

wherein the response includes a semantic meaning of the natural language input, and

wherein the client device is at least one of;

a mobile telephone;

a smart phone;

a tablet;

a smart watch;

a wearable computer;

a personal computer;

a desktop computer;

a gaming system; and

a laptop computer.

17. The method of claim 9, wherein new features are added to at least one feature extractor of the plurality of feature extractors, and

wherein the trained language understanding model does not need to be re-trained in response to the addition of the new features.

18. The method of claim 9, wherein the trained language understanding model is trained for a specific application, wherein the specific application is at least one of:

a digital assistant application;

a voice recognition application;

an email application;

a social networking application;

a collaboration application;

an enterprise management application;

a messaging application;

a word processing application;

a spreadsheet application;

a database application;

a presentation application;

a contacts application;

a gaming application;

an e-commerce application;

an e-business application;

a transactional application;

an exchange application; and

a calendaring application.

19. A system comprising:

at least one processor; and

a memory encoding computer executable instructions that, when executed by the at least one processor, cause the at least one processor to perform a method for language understanding, the method comprising;

receiving a natural language input from a client device on a natural language server,

sending the natural language input to a first feature extractor on a first server from the natural language server;

sending the natural language input to a second feature extractor on a second server from the natural language server,

wherein the first server, the second server, and the natural language server are different and separate from each other;

receiving a first set of potential features for the natural language input from the first feature extractor by the natural language server;

receiving a second set of potential features for the natural language input from the second feature extractor by the natural language server;

aggregating the first set of potential features and the second set of potential features to form an aggregated set of potential features;

evaluating the aggregated set of potential features utilizing a language understanding model trained with training features from the first feature extractor and the second feature extractor;

determining at least one of a user intent, a domain, or entities with associated confidence scores based on evaluating the aggregated set of potential features; and

generating a response based on the at least one of the user intent, the domain, and the entities with the associated confidence scores.

20. The system of claim 19, wherein the first feature extractor is an auxiliary language understanding system and the second feature extractor is a location extractor.