Determining domains for natural language understanding

• US 10,453,117 B1
• Filed: 06/29/2016
• Issued: 10/22/2019
• Est. Priority Date: 06/29/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Reavely, Simon Peter, Prasad, Rohit, Kiss, Imre Attila, Sindhwani, Manoj
• Primary Examiner(s): Shah, Paras D
• Assistant Examiner(s): Blankenagel, Bryan S
• Application Number:
  US15/197,022
• Time in Patent Office: 1,210 Days
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/027 Frames
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 10/02 Reservations, e.g. for tickets, services or events
• CPC Class Codes: G06Q 30/0645 Rental transactions; Leasing transactions
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A system capable of performing natural language understanding (NLU) using different application domains in parallel. A model takes incoming query text and determines a list of potential supplemental intent categories corresponding to the text. Supplemental applications within those categories are then identified as likely candidates for responding to the query. Application specific domains, including NLU components for the particular supplemental applications, are then activated and process the query text in parallel. Further, certain system default domains may also process incoming queries substantially in parallel with the supplemental applications. The different results are scored and ranked to determine highest scoring NLU results.

Claims

1. A computer-implemented method comprising:

receiving text corresponding to a query;

processing the text with a first natural-language understanding (NLU) component to determine a first NLU result;

at least partially in parallel with processing the text with the first NLU component, processing the text with a first trained model to determine;

a first intent category score corresponding to a first likelihood that the text is associated with a first intent category, and

a second intent category score corresponding to a second likelihood that the text is associated with a second intent category;

processing the first intent category score and the second intent category score with a second trained model to select;

a first NLU domain associated with the first intent category, and

a second NLU domain associated with the second intent category; and

after processing the first intent category score and the second intent category score with the second trained model;

processing the text with a second NLU component associated with the first NLU domain to determine a second NLU result,

processing the text with a third NLU component associated with the second NLU domain to determine a third NLU result,

processing the first NLU result, the second NLU result, and the third NLU result to select the second NLU result, and

causing a first command associated with the second NLU result to be executed.

2. The computer-implemented method of claim 1, further comprising:

processing the text with a first named entity recognizer associated with the second NLU component to identify a first portion of the text as corresponding to first data used to execute the first command; and

processing the text with a second named entity recognizer associated with the third NLU component to identify the first portion of the text as corresponding to second data used to execute a second command associated with the third NLU component.

3. The computer-implemented method of claim 2, further comprising:

determining the first portion of text corresponds to first data in a knowledge base, the first data corresponding to a first entity identifier;

determining the first portion of text corresponds to second data in the knowledge base, the second data corresponding to a second entity identifier;

processing the second NLU result and the first entity identifier with a rescoring component to determine a first result score; and

processing the third NLU result and the second entity identifier with the rescoring component to determine a second result score,

wherein determining that the text most closely corresponds to the second NLU result comprises determining the first result score is higher than the second result score.

4. The computer-implemented method of claim 2, further comprising:

processing a representation of the text to determine a likelihood that the first portion of the text corresponds to a word used for further command processing;

determining that the likelihood that the first portion of the text corresponds to the word is above a second threshold; and

creating an indication corresponding to the first portion,

wherein processing the text with the first named entity recognizer further comprises processing the indication with the first named entity recognizer to identify the first portion of the text as corresponding to data used to execute the first command.

5. The computer-implemented method of claim 2, wherein causing the first command to be executed comprises one of:

sending, to a device associated with the second NLU component, the first portion of text and an instruction to initiate an action using the first portion of text.

6. The computer-implemented method of claim 1, wherein a third NLU domain associated with the first intent category is not within the first intent category or within the second intent category.

7. A system comprising:

at least one processor; and

a memory including instructions that, when executed by the at least one processor, cause the system to;

receive text corresponding to a query;

process the text with a first natural-language understanding (NLU) component to determine a first NLU result;

at least partially in parallel with processing the text with the first NLU component, process the text with a first trained model to determine;

a first intent category score corresponding to a first likelihood that the text is associated with a first intent category, and

a second intent category score corresponding to a second likelihood that the text is associated with a second intent category;

process the first intent category score and the second intent category score with a second trained model to select;

a first NLU domain associated with the first intent category, and

a second NLU domain associated with the second intent category; and

after processing the first intent category score and the second intent category score with the second trained model;

process the text with a second NLU component associated with the first NLU domain to determine a second NLU result,

process the text with a third NLU component associated with the second NLU domain to determine a third NLU result,

process the first NLU result, the second NLU result, and the third NLU result to select the second NLU result, and

cause a first command associated with the second NLU result to be executed.

8. The system of claim 7, wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

process the text with a first named entity recognizer associated with the second NLU component to identify a first portion of the text as corresponding to first data used to execute the first command; and

process the text with a second named entity recognizer associated with the third NLU component to identify the first portion of the text as corresponding to second data used to execute a second command associated with the third NLU component.

9. The system of claim 8, wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

determine the first portion of text corresponds to first data in a knowledge base, the first data corresponding to a first entity identifier;

determine the first portion of text corresponds to second data in a knowledge base, the second data corresponding to a second entity identifier;

process the second NLU result and the first entity identifier with a rescoring component to determine a first result score; and

process the third NLU result and the second entity identifier with the rescoring component to determine a second result score,

wherein the instructions that configure the at least one processor to determine that the text most closely corresponds to the second NLU result comprise instructions that configure the at least one processor to determine that the first result score is higher than the second result score.

10. The system of claim 8, wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

process a representation of the text to determine a likelihood that the first portion of the text corresponds to a word used for further command processing;

determine that the likelihood that the first portion of the text corresponds to the word is above a second threshold; and

create an indication corresponding to the first portion,

wherein the instructions that, when executed by the at least one processor, further cause the system to process the text with the first named entity recognizer further configure the at least one processor to process the indication with the first named entity recognizer to identify the first portion of the text as corresponding to data used to execute the first command.

11. The system of claim 8, wherein the instructions that configure the at least one processor to cause the first command to be executed further comprise instructions that, when executed by the at least one processor, further cause the system to:

send, to a device associated with the second NLU component, the first portion of text and an instruction to initiate an action using the first portion of text.

12. The system of claim 7, wherein a third NLU domain associated with the first intent category is not within the first intent category or within the second intent category.

13. The system of claim 7, wherein the instructions to process the text with the first trained model include instructions that, when executed by the at least one processor, further cause the system to:

determine, using a recurrent neural network, a multi-dimensional vector representing at least one feature of the text; and

determine, using a classifier and the multi-dimensional vector, an application associated with the text.

14. The system of claim 7, wherein the instructions to select the second NLU domain include instructions that, when executed by the at least one processor, further cause the system to:

determine that the second intent category score is above a threshold.

15. The system of claim 7, wherein the first trained model includes at least one of a recurrent neural network (RNN), a deep neural network (DNN), or a long short-term memory (LSTM) model.

16. The system of claim 7, wherein the second NLU component comprises one of a named-entity recognition component or an intent classification component.

17. The system of claim 16, wherein the second NLU component comprises the named-entity recognition component that includes a conditional-random field model and wherein the instructions that cause the system to process the text with the second NLU component comprise instructions that, when executed by the at least one processor, cause the system to:

process, using the conditional-random field model, the text to determine a first score representing a likelihood that an entity is represented in the text,

wherein the second NLU result includes the first score.

18. The system of claim 16, wherein the second NLU component comprises the intent classification component that includes a classifier and wherein the instructions that cause the system to process the text with the second NLU component comprise instructions that, when executed by the at least one processor, cause the system to:

process, using the classifier, the text to determine a first score representing a likelihood that the text corresponds to an intent,

wherein the second NLU result includes the first score.

19. The system of claim 7, wherein the instructions that cause the system to process the first NLU result, the second NLU result, and the third NLU result to select the second NLU result comprise instructions that, when executed by the at least one processor, cause the system to:

determine a first number of recognized entities in the first NLU result;

determine a second number of recognized entities in the second NLU result;

determine a third number of recognized entities in the third NLU result; and

determine that the second number is greater than the first number and the third number.

20. The system of claim 7, wherein:

the first NLU component is one of a first plurality of NLU components, wherein the first plurality of NLU components are configured to process input text data at least partially in parallel to operation of the first trained model; and

the first NLU domain and the second NLU domain are included in a plurality of domains, each of which is associated with at least one respective NLU component, the respective NLU components being configured to be operated only after a respective domain is selected using the second trained model.

21. The system of claim 7, wherein the instructions that cause the system to select the second NLU domain comprise instructions that, when executed by the at least one processor, cause the system to:

determine an application score corresponding to a third likelihood that an application is associated with the second intent category;

determine that the application score is greater than a first threshold; and

determine that the second intent category score is greater than a second threshold.