Neural latent variable model for spoken language understanding
First Claim
1. A method for configuring a natural language (NL) understanding system, the NL understanding system including at least an intent classifier, the intent classifier comprising a neural network configurable with configuration data including neural network weights to distinguish a plurality of intents based on a linguistic input, the method comprising:
- configuring the intent classifier with first configuration data, the first configuration data having been determined from first collected data comprising input data items each annotated with a specific intent of a plurality of intents, the neural network weights of the first configuration data having been determined to best match the input data items and the annotated specific intents;
processing a second plurality of user inputs to the system, wherein the processing includes, for each of the second plurality of user inputs, using the intent classifier configured with the first configuration data to determine a recognized intent corresponding to said input, and causing determination of a corresponding response to the input based on the recognized intent;
storing second collected data, the second collected data comprising, for each of the second plurality of user inputs, a representation of the user input and a corresponding response;
receiving manual annotation for the second collected data, an annotation for each item of the second collected data indicating whether the corresponding response is consistent with the user input, thereby forming second annotated data without requiring annotating a correct intent for user inputs for which the caused action does not sufficiently match the user input;
determining second configuration data for the intent classifier, the second configuration data being determined to distinguish the plurality of intents by computing the second configuration data to match the second annotated data, including treating correct intents as latent variables that are not represented in the second annotated data, the determining including incrementally updating the neural network weights of the second configuration data; and
configuring the intent classifier with the second configuration data.
1 Assignment
0 Petitions
Accused Products
Abstract
A linguist classifier, for instance intent or slot classifier, is updated using data with only partial annotation indicating overall correctness rather that specific correct intent or slot values, which are treated as “latent” (i.e., unknown) variables. Full annotation of the data is not required. A small amount of fully annotated data may be combined with a substantially larger amount of partially annotated data to update the linguistic classifier. In a specific implementation, the linguistic classifier is a neural network and the weights are trained using a reinforcement learning approach.
39 Citations
20 Claims
-
1. A method for configuring a natural language (NL) understanding system, the NL understanding system including at least an intent classifier, the intent classifier comprising a neural network configurable with configuration data including neural network weights to distinguish a plurality of intents based on a linguistic input, the method comprising:
-
configuring the intent classifier with first configuration data, the first configuration data having been determined from first collected data comprising input data items each annotated with a specific intent of a plurality of intents, the neural network weights of the first configuration data having been determined to best match the input data items and the annotated specific intents; processing a second plurality of user inputs to the system, wherein the processing includes, for each of the second plurality of user inputs, using the intent classifier configured with the first configuration data to determine a recognized intent corresponding to said input, and causing determination of a corresponding response to the input based on the recognized intent; storing second collected data, the second collected data comprising, for each of the second plurality of user inputs, a representation of the user input and a corresponding response; receiving manual annotation for the second collected data, an annotation for each item of the second collected data indicating whether the corresponding response is consistent with the user input, thereby forming second annotated data without requiring annotating a correct intent for user inputs for which the caused action does not sufficiently match the user input; determining second configuration data for the intent classifier, the second configuration data being determined to distinguish the plurality of intents by computing the second configuration data to match the second annotated data, including treating correct intents as latent variables that are not represented in the second annotated data, the determining including incrementally updating the neural network weights of the second configuration data; and configuring the intent classifier with the second configuration data.
-
-
2. A method for configuring a natural language (NL) understanding system, the NL understanding system including at least a linguistic classifier, the linguistic classifier being configurable with configuration data to distinguish a plurality of linguistic categories based on a linguistic input, the method comprising:
-
configuring the linguistic classifier with first configuration data; processing a second plurality of user inputs to the system, wherein the processing includes, for each of the second plurality of user inputs, using the linguistic classifier configured with the first configuration data to determine a recognized linguistic category corresponding to said input, and causing determination of a corresponding response to the input based on the recognized linguistic category; storing second collected data, the second collected data comprising, for each of the second plurality of user inputs, a representation of the user input and the corresponding responses; receiving annotations for the second collected data, an annotation for each item of the second collected data indicating whether the corresponding response is consistent with the user input; determining second configuration data for the first linguistic classifier, the second configuration data being determined to distinguish the plurality of linguistic categories by computing the second configuration data to match the second annotated data; and configuring the linguistic classifier with the second configuration data. - View Dependent Claims (3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A natural language (NL) understanding system comprising:
-
a speech recognizer configured to process a speech input to produce an output; a linguistic feature extractor configured to process the output of the speech recognizer to produce a linguistic representation of the speech input, the linguistic representation comprising a fixed-length numerical vector; a linguistic classifier, the linguistic classifier being configurable with configuration data to distinguish a plurality of linguistic categories based on a linguistic input; a classifier initializer for determining first configuration data for the first linguist classifier; a classifier updater for determining second configuration data for the first linguistic classifier based on a processing of second collected data including a second plurality of user inputs to the system and corresponding actions determined by the system for those inputs, and annotations for each item of the second collected data indicating whether the action matches the user input, the classifier updater being configured to determining second configuration data for the first linguistic classifier to distinguish the plurality of linguistic categories by computing the second configuration data to match the second annotated data. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification