Dynamic gazetteers for personalized entity recognition

• US 10,224,030 B1
• Filed: 03/14/2013
• Issued: 03/05/2019
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Kiss, Imre Attila, Toth, Arthur Richard, Mathias, Lambert
• Primary Examiner(s): Opsasnick, Michael N
• Application Number:
  US13/828,063
• Time in Patent Office: 2,182 Days
• Field of Search: 704246, 704247
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology

Abstract

In speech processing systems personalization is added in the Natural Language Understanding (NLU) processor by incorporating external knowledge sources of user information to improve entity recognition performance of the speech processing system. Personalization in the NLU is effected by incorporating one or more dictionaries of entries, or gazetteers, with information personal to a respective user, that provide the user'"'"'s information to permit disambiguation of semantic interpretation for input utterances to improve quality of speech processing results.

Claims

1. A method for providing personalized natural language understanding, comprising:

receiving an audio signal comprising an utterance;

obtaining a textual representation of the utterance using automatic speech recognition;

obtaining an identity of a speaker of the utterance;

obtaining a first speaker-specific gazetteer associated with the speaker of the utterance;

creating a feature vector for a first word of the textual representation, the feature vector including a first feature indicating whether a second word of the textual representation is present in the first speaker-specific gazetteer;

determining, based at least in part on the feature vector and a named entity recognition module, a plurality of labels associated with the textual representation, the plurality of labels corresponding to a semantic interpretation of the textual representation;

determining a computer executable command using the textual representation and the plurality of labels;

executing the computer executable command; and

outputting audio data.

2. The method of claim 1, wherein the first speaker-specific gazetteer was created by combining a speaker-independent gazetteer with a second speaker-specific gazetteer.

3. The method of claim 1, wherein the named entity recognition model comprises a conditional random field and wherein the named entity recognition model was trained using a speaker-independent gazetteer.

4. The method of claim 1, wherein the named entity recognition model comprises a conditional random field, and wherein the named entity recognition model was trained using a speaker-specific gazetteer.

5. A computer implemented method, comprising:

obtaining a textual representation of a user request;

obtaining an identity of the user;

obtaining a first gazetteer associated with the user;

creating a feature vector for a first word of the textual representation, the feature vector including a first feature indicating whether a second word of the textual representation is present in the first gazetteer;

determining, based at least in part on the feature vector and a named entity recognition model, at least one label associated with the textual representation;

determining a computer executable command using the textual representation and the at least one label;

executing the computer executable command; and

outputting audio data.

6. The computer implemented method of claim 5, wherein the first gazetteer comprises a gazetteer configured specifically for the user and the named entity recognition model is trained for general use.

7. The computer implemented method of claim 6, wherein the named entity recognition model was trained using a second gazetteer.

8. The computer implemented method of claim 5, wherein the first gazetteer comprises a second gazetteer originally configured for general use augmented with information specific to the user.

9. The computer implemented method of claim 5, wherein the named entity recognition model is trained based at least in part on textual representations corresponding to previous requests of the user.

10. The computer implemented method of claim 9, wherein the named entity recognition model comprises at least one of a conditional random field or a maximum entropy Markov model.

11. The computer implemented method of claim 5, wherein the first feature is a boolean.

12. The computer implemented method of claim 5, wherein the textual representation is obtained from an output of an automatic speech recognition component.

13. The computer implemented method of claim 5, wherein obtaining an identity of the user comprises one or more of performing a speaker identification operation, receiving an identity of the user or receiving an identity of a device of the user.

23. The computer implemented method of claim 5, wherein

the first gazetteer includes a plurality of music artist names, and

the feature vector indicates a portion of the plurality of music artist names that are associated with the user.

24. The computer implemented method of claim 5, wherein the feature vector includes a second feature indicating whether the second word is present in a second gazetteer associated with the user.

25. The computer implemented method of claim 5, wherein executing the computer executable command further comprises:

playing music based on the textual representation.

14. A computing device, comprising:

at least one processor;

a memory device including instructions operable to be executed by the at least one processor to perform a set of actions, configuring the at least one processor;

to obtain a textual representation of a user request;

to obtain an identity of the user;

to obtain a first gazetteer associated with the user;

to create a feature vector for a first word of the textual representation, the feature vector including a first feature indicating whether a second word of the textual representation is present in the first gazetteer;

to determine, based at least in part on the feature vector and a named entity recognition model, at least one label associated with the textual representation;

to determine a computer executable command using the textual representation and the at least one label; and

to execute the computer executable command; and

to output audio data.

15. The computing device of claim 14, wherein the first gazetteer comprises a gazetteer configured specifically for the user and the named entity recognition model is trained for general use.

16. The computing device of claim 15, wherein the named entity recognition model was trained using a second gazetteer.

17. The computing device of claim 14, wherein the first gazetteer comprises a second gazetteer originally configured for general use augmented with information specific to the user.

18. The computing device of claim 14, wherein the named entity recognition model is trained based at least in part on textual representations corresponding to previous requests of the user.

19. The computing device of claim 18, wherein the named entity recognition model comprises at least one of a conditional random field or a maximum entropy Markov model.

20. The computing device of claim 14, wherein the first feature is a boolean.

21. The computing device of claim 14, wherein the textual representation is obtained from an output of an automatic speech recognition component.

22. The computing device of claim 14, wherein the at least one processor is configured to obtain an identity of the user by one or more of performing a speaker identification operation, receiving an identity of the user or receiving an identity of a device of the user.

26. The computing device of claim 14, wherein

the first gazetteer includes a plurality of music artist names, and

the feature vector indicates a portion of the plurality of music artist names that are associated with the user.

27. The computing device of claim 14, wherein the feature vector includes a second feature indicating whether the second word is present in a second gazetteer associated with the user.

28. The computing device of claim 14, wherein the at least one processor is configured to play music based on the textual representation.