Efficient generation of personalized spoken language understanding models
First Claim
Patent Images
1. A system comprising:
- a data harvesting service comprising one or more server computing devices configured to at least;
store spoken language understanding personalization data associated with a user, wherein;
the spoken language understanding personalization data comprises a plurality of separate data sets; and
each of the plurality of separate data sets is accessible independently from each remaining data set of the plurality of data sets;
receive a new data set of the spoken language understanding personalization data, wherein;
the new data set is associated with the user; and
the new data set comprises activity data received from a plurality of services, wherein the activity data is related to actions initiated by the user that cause one or more of the plurality of services to change user-specific data associated with the user; and
transmit, to a model generation service, a notification that the new data set associated with the user is available; and
a model generation service comprising one or more server computing devices configured to at least;
generate a personal language model, associated with the user, based at least in part on at least a portion of the plurality of separate data sets;
receive the notification that the new data set associated with the user is available;
determine a predicted action of the user based at least in part on the activity data of the new data set;
determine that a word associated with the predicted action is not present in a general language model; and
in response to determining that the word is not present in the general language model, modify the personal language model based at least partly on a likelihood that the user will use the word; and
a speech recognition service comprising one or more computing devices configured to at least;
receive audio data representing a user utterance;
access the personal language model that is modified; and
generate speech recognition results using the audio data and the personal language model that is modified.
1 Assignment
0 Petitions
Accused Products
Abstract
Features are disclosed for maintaining data that can be used to personalize spoken language understanding models, such as speech recognition or natural language understanding models. The personalization data can be used to update the models based on some or all of the data. The data may be obtained from various data sources, such as applications or services used by the user. Personalized spoken language understanding models may be generated or updated based on updates to the personalization data or some other portion of the stored personalization data. Generation of personalized spoken language understanding models may be prioritized such that the generation process accommodates multiple users.
79 Citations
33 Claims
-
1. A system comprising:
-
a data harvesting service comprising one or more server computing devices configured to at least; store spoken language understanding personalization data associated with a user, wherein; the spoken language understanding personalization data comprises a plurality of separate data sets; and each of the plurality of separate data sets is accessible independently from each remaining data set of the plurality of data sets; receive a new data set of the spoken language understanding personalization data, wherein; the new data set is associated with the user; and the new data set comprises activity data received from a plurality of services, wherein the activity data is related to actions initiated by the user that cause one or more of the plurality of services to change user-specific data associated with the user; and transmit, to a model generation service, a notification that the new data set associated with the user is available; and a model generation service comprising one or more server computing devices configured to at least; generate a personal language model, associated with the user, based at least in part on at least a portion of the plurality of separate data sets; receive the notification that the new data set associated with the user is available; determine a predicted action of the user based at least in part on the activity data of the new data set; determine that a word associated with the predicted action is not present in a general language model; and in response to determining that the word is not present in the general language model, modify the personal language model based at least partly on a likelihood that the user will use the word; and a speech recognition service comprising one or more computing devices configured to at least; receive audio data representing a user utterance; access the personal language model that is modified; and generate speech recognition results using the audio data and the personal language model that is modified. - View Dependent Claims (2, 3, 4)
-
-
5. A computer-implemented method comprising:
-
obtaining, by a first service comprising one or more computing devices, spoken language understanding personalization data associated with a first user of a plurality of users, wherein the spoken language understanding personalization data comprises activity data received from one or more of a plurality of services, and wherein the activity data is related to actions initiated by the first user that cause one or more of the plurality of services to change user-specific data associated with the first user; storing, by the first service, the spoken language understanding personalization data in association with previously stored spoken language understanding personalization data associated with the first user; providing, to a second service comprising one or more computing devices separate from the one or more computing devices of the first service, a notification that the spoken language understanding personalization data is available, and the spoken language understanding personalization data; determining, by the second service, a predicted action of the first user based at least in part on the activity data of the spoken language understanding personalization data; determining, by the second service, that a word associated with the predicted action is not present in a general spoken language understanding model; in response to determining that the word is not present in the general spoken language understanding model, modifying, by the second service, a personal spoken language understanding model associated with the first user based at least on a likelihood that the first user will use the word; receiving audio data representing an utterance of the first user; and generating speech recognition results using the audio data and the personal spoken language understanding model that is modified. - View Dependent Claims (6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31)
-
-
32. One or more non-transitory computer readable media comprising executable code that, when executed, cause one or more computing devices to perform a process comprising:
-
obtaining, by a first service, spoken language understanding personalization data associated with a first user of a plurality of users, wherein the spoken language understanding personalization data comprises activity data received from a plurality of services, and wherein the activity data is related to actions initiated by the first user that cause one or more of the plurality of services to change user-specific data associated with the first user; storing, by the first service, the spoken language understanding personalization data in association with previously stored spoken language understanding personalization data associated with the first user; providing, to a second service, a notification that the spoken language understanding personalization data is available, and the spoken language understanding personalization data; determining, by the second service, a predicted action of the first user based at least in part on the activity data of the spoken language understanding personalization data; determining, by the second service, that a word associated with the predicted action is not present in a general spoken language understanding model; in response to determining that the word is not present in the general spoken language understanding model, modifying, by the second service, a personal spoken language understanding model associated with the first user based at least on a likelihood that the first user will use the word; receiving audio data representing an utterance of the first user; and generating speech recognition results using the audio data and the personal spoken language understanding model that is modified. - View Dependent Claims (33)
-
Specification