Generation of predictive natural language processing models

• US 10,049,656 B1
• Filed: 09/20/2013
• Issued: 08/14/2018
• Est. Priority Date: 09/20/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Barton, William Folwell, Prasad, Rohit, Potter, Stephen Frederick, Strom, Nikko, Watanabe, Yuzo, Jampani, Madan Mohan Rao, Rastrow, Ariya, Rajasekaram, Arushan
• Primary Examiner(s): Shah, Paras D
• Assistant Examiner(s): Blankenagel, Bryan S
• Application Number:
  US14/033,346
• Time in Patent Office: 1,789 Days
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Features are disclosed for generating predictive personal natural language processing models based on user-specific profile information. The predictive personal models can provide broader coverage of the various terms, named entities, and/or intents of an utterance by the user than a personal model, while providing better accuracy than a general model. Profile information may be obtained from various data sources. Predictions regarding the content or subject of future user utterances may be made from the profile information. Predictive personal models may be generated based on the predictions. Future user utterances may be processed using the predictive personal models.

Claims

1. A system comprising:

a computer-readable memory storing executable instructions; and

one or more processors in communication with the computer-readable memory, wherein the one or more processors are programmed by the executable instructions to at least;

obtain natural language processing personalization data associated with a user, the natural language processing personalization data comprising data regarding items in a user-specific content catalog associated with the user;

generate a personal language model using at least the data regarding items in the user-specific content catalog, wherein the personal language model is specific to the user, wherein the personal language model includes a first subset of items in a general language model, and wherein the general language model is not associated with any specific user;

determine, using at least the data regarding items in the user-specific content catalog, a plurality of user-specific predicted items about which the user is predicted to make a future utterance, wherein the plurality of user-specific predicted items are not in the user-specific content catalog;

generate a predictive language model based at least on the plurality of user-specific predicted items, wherein the predictive language model is associated with the user, and wherein the predictive language model includes a second subset of items in the general language model;

generate a weighting factor for the general language model, wherein the weighting factor, when applied to the general language model, reduces probabilities associated with individual items in the general language model that are determined to be acoustically confusable with at least portion of the user-specific predicted items; and

subsequently;

process an utterance using the personal language model, the predictive language model, the general language model, and the weighting factor, wherein the utterance includes a first item of the plurality of user-specific predicted items;

recognize the first item based at least on the personal language model, the predictive language model, and the general language model, wherein the first item is recognized based at least partly on a first probability for the first item being higher than a second probability for a second item and a third probability for a third item, wherein a value of the first probability comprises a probability value from the personal language model, wherein a value of the second probability comprises a probability value from the predictive language model, and wherein a value of the third probability comprises a product of the weighting factor and a probability value from the general language model; and

play, on a user computing device, audio content associated with the first item.

2. The system of claim 1, wherein the user-specific catalog comprises one of:

a music catalog, a video catalog, a contact list, a calendar, a shopping history, or a browsing history.

3. The system of claim 1, wherein the one or more processors are further configured to:

generate, using the utterance, a lattice of processed results; and

generate a transcription of the utterance from an automatic speech recognition model and the lattice of processed results.

17. The system of claim 1, wherein the executable instructions to process the utterance using the personal language model in combination with the weighting factor and the general language model comprise instructions to multiply the weighting factor by at least a portion of the probabilities associated with items in the general language model that are determined to be acoustically confusable with the user-specific predicted items.

20. The system of claim 1, wherein the weighting factor comprises a numerical value.

4. A computer-implemented method comprising:

under control of one or more computing devices configured to execute specific instructions,

obtaining content catalog information specific to a user profile;

generating a personal language model based at least on the content catalog information, wherein the personal language model is specific to the user profile, wherein the personal language model includes a first subset of items in a general language model, and wherein the general language model is not associated with any specific user profile;

determining, based at least on the content catalog information, predicted utterance content, wherein the content catalog information does not include the predicted utterance content;

generating a predictive language model based at least on the predicted utterance content, wherein the predictive language model is specific to the user profile, and wherein the predictive language model includes a second subset of items in the general language model;

generating a weighting factor that, when applied to the general language model, reduces probabilities associated with at least a portion of items in the general language model that are determined to be acoustically confusable for at least;

a portion of items of the predicted utterance content, or a portion of items of the content catalog information;

recognizing a first item in a first utterance using the general language model in combination with the personal language model and the predictive language model, wherein the first item is recognized based at least partly on a first probability for the first item being higher than a second probability for a second item and a third probability for a third item, wherein a value of the first probability comprises a probability value from of the personal language model, wherein a value of the second probability comprises a probability value from the predictive language model, and wherein a value of the third probability comprises a product of the weighting factor and a probability value from the general language model; and

causing audio content associated with the first item to be played by an output device.

5. The computer-implemented method of claim 4, wherein the content catalog information comprises at least one of a group consisting of:

a catalog specific to the user profile;

data regarding demographics;

data regarding historical utterances; and

data regarding historical behaviors.

6. The computer-implemented method of claim 4, further comprising performing at least one of automatic speech recognition or natural language understanding on a second utterance using the predictive language model interpolated with a second language model.

7. The computer-implemented method of claim 6, wherein the second language model is one of a content domain-specific personal language model or the general language model.

8. The computer-implemented method of claim 6, further comprising generating a second weighting factor for the predictive language model.

9. The computer-implemented method of claim 6, further comprising determining that the second utterance relates to a predicted item based at least on the predictive language model.

10. The computer-implemented method of claim 9, wherein the predicted item is related to an item in a catalog specific to the user profile.

11. The computer-implemented method of claim 4, further comprising analyzing content catalog information associated with a plurality of user profiles in comparison with the content catalog information associated with the user profile.

15. The computer-implemented method of claim 4 further comprising:

identifying a first domain of a plurality of domains of predicted utterance content based at least partly on a first common classification of a group of items of the predicted utterance content, wherein the first common classification is associated with an age group of user profiles associated with the first group of items; and

identifying a second domain of the plurality of domains of predicted utterance content based at least partly on a second common classification of a second group of items of the predicted utterance content, wherein the second common classification is associated with a common music artist.

16. The computer-implemented method of claim 4, wherein the first subset of items of the personal language model is distinct from the second subset of items of the predictive language model, and wherein the personal language model associates probabilities with items of the content catalog information that are higher than probabilities associated with the items of the content catalog information included in the general language model.

18. The computer-implemented method of claim 4, further comprising multiplying the weighting factor by the probabilities associated with the portion of items in the general language model to generate weighted probabilities, wherein the recognizing the first item comprises recognizing the first item using at least one of the weighted probabilities.

21. The computer-implemented method of claim 4, wherein the recognizing the first item in the first utterance comprises:

generating preliminary natural language processing results using the general language model; and

modifying one or more scores of the preliminary natural language processing results using the predictive language model.

22. The computer-implemented method of claim 4, further comprising generating output data representing at least one of:

an audio content item, or a visual content item.

12. One or more non-transitory computer readable media comprising executable code that, when executed, cause one or more computing devices to perform a process comprising:

obtaining catalog information associated with a user profile;

generating a personal language model based at least on the catalog information, wherein the personal language model is specific to the user profile, wherein the personal language model includes a first subset of items in a general language model, and wherein the general language model is not associated with any specific user profile;

determining, based at least on the catalog information, predicted natural language processing input content, wherein the catalog information does not include the predicted natural language processing input content;

generating a predictive language model based at least on the predicted natural language processing input content, wherein the predictive language model is associated with the user profile, and wherein the predictive language model includes a subset of items in the general language model;

generating a weighting factor that, when applied to the predictive language model, modifies probabilities associated with at least a portion of items of the predicted natural language processing input content that are determined to be acoustically confusable for at least a portion of items in the general language model;

recognizing a first item in a first utterance using the general language model in combination with the personal language model and the predictive language model, wherein the first item is recognized based at least partly on a first probability for the first item being higher than a second probability for a second item and a third probability for a third item, wherein a value of the first probability comprises a product of the weighting factor and a probability value from the personal language model, wherein a value of the second probability comprises a probability value from the predictive language model, and wherein a value of the third probability comprises a probability value from the general language model; and

causing audio content associated with the first item to be played by an output device.

13. The one or more non-transitory computer readable media of claim 12, wherein the process further comprises performing automatic speech recognition on a second utterance using the predictive language model interpolated with a second language model.

14. The one or more non-transitory computer readable media of claim 13, wherein the second language model is one of a domain-specific personal language model or the general language model.

19. The one or more non-transitory computer readable media of claim 12, the process further comprising multiplying the weighing factor by at least a portion of the probabilities associated with items of the predicted natural language processing input content to generate weighted probabilities, wherein the recognizing the first item in the first utterance comprises recognizing the first item using at least one of the weighted probabilities.