Updating language understanding classifier models for a digital personal assistant based on crowd-sourcing
First Claim
1. A server computer, comprising:
- a processing unit; and
memory coupled to the processing unit;
the server computer configured to perform operations for updating language understanding classifier models, the operations comprising;
receiving from at least one computing device of a plurality of computing devices communicatively coupled to the server computer, a first user selection of at least one of the following;
at least one intent of a plurality of available intents and/or at least one slot for the at least one intent, wherein;
the at least one intent is associated with at least one action used to perform at least one function of a category of functions for a domain;
the at least one slot indicating a value used for performing the at least one action; and
the first user selection associated with a digital voice input received at the at least one computing device; and
upon receiving from at least another computing device of the plurality of computing devices, a plurality of subsequent user selections that are identical to the first user selection and a plurality of subsequent digital voice inputs corresponding to the plurality of subsequent user selections, wherein the plurality of subsequent digital voice inputs are substantially similar to the digital voice input;
generating a labeled data set by pairing the digital voice input with the first user selection;
selecting a language understanding classifier from a plurality of available language understanding classifiers associated with one or more agent definitions, the selecting based at least on the at least one intent; and
updating the selected language understanding classifier based on the generated labeled data set.
1 Assignment
0 Petitions
Accused Products
Abstract
A method for updating language understanding classifier models includes receiving via one or more microphones of a computing device, a digital voice input from a user of the computing device. Natural language processing using the digital voice input is used to determine a user voice request. Upon determining the user voice request does not match at least one of a plurality of pre-defined voice commands in a schema definition of a digital personal assistant, a GUI of an end-user labeling tool is used to receive a user selection of at least one of the following: at least one intent of a plurality of available intents and/or at least one slot for the at least one intent. A labeled data set is generated by pairing the user voice request and the user selection, and is used to update a language understanding classifier.
-
Citations
19 Claims
-
1. A server computer, comprising:
-
a processing unit; and memory coupled to the processing unit; the server computer configured to perform operations for updating language understanding classifier models, the operations comprising; receiving from at least one computing device of a plurality of computing devices communicatively coupled to the server computer, a first user selection of at least one of the following;
at least one intent of a plurality of available intents and/or at least one slot for the at least one intent, wherein;the at least one intent is associated with at least one action used to perform at least one function of a category of functions for a domain; the at least one slot indicating a value used for performing the at least one action; and the first user selection associated with a digital voice input received at the at least one computing device; and upon receiving from at least another computing device of the plurality of computing devices, a plurality of subsequent user selections that are identical to the first user selection and a plurality of subsequent digital voice inputs corresponding to the plurality of subsequent user selections, wherein the plurality of subsequent digital voice inputs are substantially similar to the digital voice input; generating a labeled data set by pairing the digital voice input with the first user selection; selecting a language understanding classifier from a plurality of available language understanding classifiers associated with one or more agent definitions, the selecting based at least on the at least one intent; and updating the selected language understanding classifier based on the generated labeled data set. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A method for updating language understanding classifier models, the method comprising:
-
receiving via one or more microphones of a computing device, a digital voice input from a user of the computing device; performing natural language processing using the digital voice input to determine a user voice request; upon determining the user voice request does not match at least one of a plurality of pre-defined tasks in an agent definition of a digital personal assistant running on the computing device; receiving using a graphical user interface of an end-user labeling tool (EULT) of the computing device, a user selection of at least one of the following;
an intent of a plurality of available intents and at least one slot for the intent, wherein;the intent is associated with at least one action used to perform at least one function of a category of functions for a domain; and the at least one slot indicating a value used for performing the at least one action; generating a labeled data set by pairing the user voice request and the user selection; selecting a language understanding classifier from a plurality of available language understanding classifiers associated with the agent definition, the selecting based at least on the intent selected by the user; and updating the selected language understanding classifier based on the generated labeled data set. - View Dependent Claims (10, 11, 12, 13, 14, 15)
-
-
16. A computer-readable storage medium storing computer-executable instructions for causing a computing device to perform operations for updating language understanding classifier models, the operations comprising:
-
determining a user request based on user input received at a computing device, the user request received via at least one of text input and voice input, the request for a functionality of a digital personal assistant running on the computing device; determining the user request does not match at least one of a plurality of pre-defined voice commands in an agent definition of the digital personal assistant; generating a confidence score by applying a plurality of available language understanding classifiers associated with the agent definition to the user request; upon determining that the confidence score is less than a threshold value; receiving using a graphical user interface of an end-user labeling tool (EULT) of the computing device, a user selection of at least one of the following;
at least one intent of a plurality of available intents and at least one slot for the at least one intent, wherein;the at least one intent is associated with at least one action used to perform at least one function of a category of functions for a domain; and the at least one slot indicating a value used for performing the at least one action; generating a labeled data set by pairing the user voice request and the user selection; selecting a language understanding classifier from the plurality of available language understanding classifiers associated with the agent definition, the selecting based at least on the at least one intent selected by the user; and generating an updated language understanding classifier by training the selected language understanding classifier using the generated labeled data set. - View Dependent Claims (17, 18, 19)
-
Specification