Underspecification of intents in a natural language processing system

• US 10,216,832 B2
• Filed: 12/19/2016
• Issued: 02/26/2019
• Est. Priority Date: 12/19/2016
• Status: Active Grant

• Current Assignee: Interactions, LLC
• Original Assignee: Interactions, LLC
• Inventors: Bangalore, Srinivas, Chen, John
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US15/384,275
• Publication Number:
  US 20180174578A1
• Time in Patent Office: 799 Days
• Field of Search: 704 9, 704257, 704251, 704244, 704245, 704 10, 704243, 707763
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/137 Hierarchical processing, e.g. outlines
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models

Abstract

A natural language processing system has a hierarchy of user intents related to a domain of interest, the hierarchy having specific intents corresponding to leaf nodes of the hierarchy, and more general intents corresponding to ancestor nodes of the leaf nodes. The system also has a trained understanding model that can classify natural language utterances according to user intent. When the understanding model cannot determine with sufficient confidence that a natural language utterance corresponds to one of the specific intents, the natural language processing system traverses the hierarchy of intents to find a more general user intent that is related to the most applicable specific intent of the utterance and for which there is sufficient confidence. The general intent can then be used to prompt the user with questions applicable to the general intent to obtain the missing information needed for a specific intent.

Claims

1. A non-transitory computer-readable storage medium comprising instructions executable by a processor for identifying an intent associated with a natural language utterance, the instructions comprising:

instructions for receiving a natural language utterance specified by a user;

instructions for accessing a domain hierarchy of intents comprising leaf nodes and ancestor nodes, wherein each of a plurality of the leaf nodes has a corresponding intent understanding model trained through supervised machine learning based on respective training sets of utterances that are labeled with user intents that the sets of utterances are known to represent;

instructions for applying a plurality of intent understanding models of the leaf nodes to the natural language utterance, thereby producing a corresponding plurality of leaf node confidence scores;

instructions for determining that none of the leaf node confidence scores exceeds a given minimum confidence threshold;

instructions for, responsive to the determining, determining ancestor node confidence scores for the ancestor nodes wherein each of the ancestor node confidence scores is determined by;

identifying an intent understanding model corresponding to the ancestor node; and

applying the identified intent understanding model corresponding to the ancestor node to the natural language utterance, wherein the intent understanding model corresponding to the ancestor node produces a confidence score for the ancestor node;

instructions for identifying a most applicable ancestor node based on the ancestor node confidence scores; and

instructions for identifying, as a general intent of the natural language utterance, an intent corresponding to the identified most applicable ancestor node.

2. The non-transitory computer-readable storage medium of claim 1, the instructions further comprising:

instructions for identifying, based on the identified general intent, a prompt for additional information; and

instructions for providing the prompt to the user over the computer network.

3. The non-transitory computer-readable storage medium of claim 1, wherein any descendant node in the hierarchy of intents corresponds to a more specific user intent than the intents corresponding to any ancestor nodes from which the descendant node is descended.

4. The non-transitory computer-readable storage medium of claim 1, wherein applying the intent understanding models comprises associating the natural language utterance with a verb from a domain hierarchy of actions and with a noun from a domain hierarchy of intents related to items.

5. The non-transitory computer-readable storage medium of claim 1, wherein applying the intent understanding models comprises associating the natural language utterance with two or more intents, each intent selected from its own intent hierarchy.

6. A computer-implemented method for identifying an intent associated with a natural language utterance comprising:

receiving a natural language utterance specified by a user over a computer network;

accessing a domain hierarchy of intents comprising leaf nodes and ancestor nodes, wherein each of a plurality of the leaf nodes has a corresponding intent understanding model trained through supervised machine learning based on respective training sets of utterances that are labeled with user intents the sets of utterances are known to represent;

applying the intent understanding models corresponding to each of the plurality of leaf nodes to the natural language utterance, thereby producing a corresponding plurality of leaf node confidence scores;

determining that none of the leaf node confidence scores exceeds a given minimum confidence threshold;

responsive to the determining, determining ancestor node confidence scores for the ancestor nodes wherein each of the ancestor node confidence scores is determined by;

identifying an intent understanding model corresponding to the ancestor node; and

applying the identified intent understanding model corresponding to the ancestor node to the natural language utterance, wherein the intent understanding model corresponding to the ancestor node produces a confidence score for the ancestor node;

identifying a most applicable ancestor node based on the ancestor node confidence scores; and

identifying, as a general intent of the natural language utterance, an intent corresponding to the identified most applicable ancestor node.

7. The computer-implemented method of claim 6, further comprising:

identifying, based on the identified general intent, a prompt for additional information; and

providing the prompt to the user over the computer network.

8. The computer-implemented method of claim 6, wherein the leaf nodes in the hierarchy of intents correspond to more specific user intents than the intents corresponding to the ancestor nodes from which the leaf nodes are descended.

9. The computer-implemented method of claim 6, wherein applying the intent understanding models comprises associating the natural language utterance with a verb from a domain hierarchy of actions and with a noun from a domain hierarchy of intents related to items.

10. The computer-implemented method of claim 6, wherein applying the intent understanding models comprises associating the natural language utterance with two or more intents, each intent selected from its own intent hierarchy.