Named entity recognition with personalized models

• US 9,190,055 B1
• Filed: 03/14/2013
• Issued: 11/17/2015
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Kiss, Imre Attila, Mathias, Lambert, Adams, Jeffrey Penrod
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US13/828,759
• Time in Patent Office: 978 Days
• Field of Search: 704/9, 704/10, 704/257, 704/7, 704/251, 707/999.005, 707/673, 707/999.006, 707/999.003, 726/26, 715/748
• US Class Current: 1/1
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting

Abstract

Features are disclosed for generating and using personalized named entity recognition models. A personalized model can be trained for a particular user, and then interpolated with a general model for use in named entity recognition. In some embodiments, a model may be trained for a group of users, where the users share some similarity relevant to language processing. In some embodiments, various base models may be trained so as to provide better accuracy for certain types of language input than a general model. Users may be associated with any number of base models, and the associated based models may then be interpolated for use in named entity recognition on input from the corresponding user.

Claims

1. A system comprising:

a computer-readable memory storing executable instructions; and

one or more processors in communication with the computer-readable memory, wherein the one or more processors are programmed by the executable instructions to at least;

obtain usage data regarding prior usage, by a user, of a natural language processing system;

train, based at least partly on the usage data, a personal model for use in named entity recognition, the personal model comprising a conditional random field model;

interpolate the personal model and a general model to obtain a composite model, the general model comprising a conditional random field model, wherein interpolation of the personal model and the general model comprises interpolation of an element of the personal model with a corresponding element of the general model;

receive audio data regarding an utterance of the user;

generate a transcription of the utterance using automatic speech recognition;

process, based at least partly on the composite model, the transcription with a named entity recognition module to determine at least one named entity from the transcription; and

generate a response based at least partly on the at least one named entity.

2. The system of claim 1, wherein the element of the personal model comprises a first parameter vector, the corresponding element of the general model comprises a second parameter vector, and interpolation of the element of the personal model with the corresponding element of the general model comprises performing a weighted average of the first parameter vector and the second parameter vector.

3. The system of claim 1, wherein the one or more processors are further programmed by the executable instructions to:

obtain user data regarding one or more characteristics associated with the user, wherein training the personal model is further based at least partly on the user data.

4. A computer-implemented method comprising:

under control of one or more computing devices configured with specific computer-executable instructions,

training, based at least partly on user data regarding a user, a personal model for use in named entity recognition, the personal model comprising a named entity recognition model;

interpolating the personal model and a general model to obtain a composite model, the general model comprising a named entity recognition model;

receiving audio input of an utterance, the audio input captured via a microphone;

generating a transcription of the utterance based at least partly on the audio input and

performing named entity recognition on the transcription using the composite model to generate a sequence of named entities.

5. The computer-implemented method of claim 4, wherein the personal model comprises one of:

a conditional random field model or a maximum entropy Markov model.

6. The computer-implemented method of claim 4, wherein interpolation of the personal model and the general model comprises interpolation of an element of the personal model with a corresponding element of the general model.

7. The computer-implemented method of claim 6, wherein the element of the personal model comprises a parameter vector.

8. The computer-implemented method of claim 4, wherein the interpolating further comprises interpolating the personal model and the general model and one or more additional models to obtain the composite model.

9. The computer-implemented method of claim 8, wherein interpolating the personal model and the general model and the one or more additional models is based at least partly on a weight associated with each additional model.

10. The computer-implemented method of claim 8, further comprising:

selecting a first model to include in the one or more additional models based at least partly on the user data.

11. The computer-implemented method of claim 4, wherein the user data comprises at least one of:

data regarding a personal characteristic of the user, data regarding a device characteristic of a computing device associated with the user, or data regarding prior natural language processing initiated by the user.

12. The computer-implemented method of claim 4, wherein the user data comprises one of:

geographic location, age, gender, occupation, or a personal interest.

13. The computer-implemented method of claim 4, further comprising:

receiving, from a user device associated with the user, audio data regarding an utterance of the user; and

performing automatic speech recognition on the utterance to generate a transcription.

24. The computer-implemented method of claim 4, wherein interpolating the personal model and the general model comprises summing a first likelihood score from a parameter vector of the personal model and a second likelihood score from a corresponding parameter vector of the general model, wherein the parameter vector of the personal model comprises parameters for recognizing a named entity, and wherein the corresponding parameter vector of the general model also comprises parameters for recognizing the named entity.

14. One or more non-transitory computer readable media comprising executable code that, when executed, cause one or more computing devices to perform a process comprising:

training, based at least partly on user data regarding a user, a personal model for use in named entity recognition, the personal model comprising a named entity recognition model;

interpolating the personal model and a general model to obtain a composite model, the general model comprising a named entity recognition model;

receiving audio input of an utterance, the audio input captured via a microphone;

generating a transcription of the utterance based at least partly on the audio input and

performing named entity recognition on the transcription using the composite model to generate a sequence of named entities.

15. The one or more non-transitory computer readable media of claim 14, wherein the personal model comprises one of:

a conditional random field model or a maximum entropy Markov model.

16. The one or more non-transitory computer readable media of claim 14, wherein interpolation of the personal model and the general model comprises interpolation of an element of the personal model with a corresponding element of the general model.

17. The one or more non-transitory computer readable media of claim 16, wherein the element of the personal model comprises a parameter vector.

18. The one or more non-transitory computer readable media of claim 14, wherein the interpolating further comprises interpolating the personal model and the general model and one or more additional models to obtain the composite model.

19. The one or more non-transitory computer readable media of claim 18, further comprising:

selecting a first model to include in the one or more additional models based at least partly on the user data.

20. The one or more non-transitory computer readable media of claim 14, wherein the user data comprises at least one of:

data regarding a personal characteristic of the user, data regarding a device characteristic of a computing device associated with the user, or data regarding prior natural language processing initiated by the user.

21. The one or more non-transitory computer readable media of claim 14, wherein the user data comprises one of:

geographic location, age, gender, occupation, or a personal interest.

22. The one or more non-transitory computer readable media of claim 14, the further comprising:

receiving, from a user device associated with the user, audio data regarding an utterance of the user; and

performing automatic speech recognition on the utterance to generate a transcription.

23. The one or more non-transitory computer readable media of claim 14, the process further comprising:

generating an interpretation of the transcription based at least partly on the sequence of named entities.

25. The one or more non-transitory computer readable media of claim 14, wherein interpolating the personal model and the general model comprises summing a first likelihood score from a parameter vector of the personal model and a second likelihood score from a corresponding parameter vector of the general model, wherein the parameter vector of the personal model comprises parameters for recognizing a named entity, and wherein the corresponding parameter vector of the general model also comprises parameters for recognizing the named entity.