Collaborative language model biasing

• US 9,704,483 B2
• Filed: 07/28/2015
• Issued: 07/11/2017
• Est. Priority Date: 07/28/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sharifi, Matthew, Foerster, Jakob Nicolaus
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US14/811,190
• Publication Number:
  US 20170032781A1
• Time in Patent Office: 714 Days
• Field of Search: 704 9, 704 10, 704235, 704243, 704239, 704257, 704231, 704245
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology

Abstract

Methods, including computer programs encoded on a computer storage medium, for collaborative language model biasing. In one aspect, a method includes receiving (i) data including a set of terms associated with a target user, and, (ii) from each of multiple other users, data including a set of terms associated with the other user, selecting a particular other user based at least on comparing the set of terms associated with the target user to the sets of terms associated with the other users, selecting one or more terms from the set of terms that is associated with the particular other user, obtaining, based on the selected terms that are associated with the particular other user, a biased language model, and providing the biased language model to an automated speech recognizer.

Claims

1. A computer-implemented method comprising:

receiving, by a term pre-processor module of a server-based automated speech recognition system that includes (a) the term pre-processor module, (b) a user similarity determiner module, (c) a vocabulary generator module, (d) a language model biaser module, and (e) an automated speech recognizer, and from a device associated with a target user, (i) data including a set of terms associated with the target user that includes terms from one or more queries previously submitted by the target user, and, (ii) from each of multiple other users, data including a set of terms associated with the other user;

selecting, by the user similarity determiner module of the server-based automated speech recognition system, a particular other user based at least on comparing the set of terms associated with the target user to the sets of terms associated with the other users;

selecting, by the vocabulary generator module of the server-based automated speech recognition system, one or more terms from the set of terms that is associated with the particular other user;

obtaining, by the language model biaser module of the server-based automated speech recognition system and based on the selected terms that are associated with the particular other user, a biased language model;

using, by the automated speech recognizer of the server-based automated speech recognition system, the biased language model generate a speech recognition output; and

providing, by the server-based automated speech recognition system, the speech recognition output to the device associated with the target user.

2. The method of claim 1, wherein selecting a particular other user based at least on comparing the set of terms associated with the target user to the sets of terms associated with the other users comprises selecting multiple particular other users based at least on comparing the set of terms associated with the target user to the sets of terms associated with the multiple other particular users.

3. The method of claim 1, wherein the particular other user is a user that is identified as similar to the target user.

4. The method of claim 1, wherein obtaining the biased language model comprises adjusting a respective score associated with each of the selected terms in a baseline language model.

5. The method of claim 1, wherein, for each user, the set of terms associated with the user includes terms that are indicated as terms that have been accepted by the user.

6. The method of claim 1, wherein, for each term of the set of terms, the data indicates whether the term comprises an n-gram or an entity name.

7. The method of claim 1, wherein the set of terms associated with a target user includes transcriptions of prior utterances by the target user.

8. The method of claim 7, comprising:

receiving an utterance from the target user; and

transcribing the utterance using the biased language model.

9. The method of claim 1, further comprising:

receiving, from the device associated with the target user, a subsequent query of the target user;

using, by the automated speech recognizer, the biased language model to generate an output responsive to the subsequent query of the target user; and

providing the output to the device associated with the target user.

10. A computer-implemented method comprising:

receiving, by a term pre-processor module of a server-based automated speech recognition system that includes (a) the term pre-processor module, (b) a vocabulary generator module, (c) a language model biaser module, and (d) an automated speech recognizer, and from a device associated with a target user, (i) data including a set of terms associated with the target user that includes terms from one or more queries previously submitted by the target user, and, (ii) from each of multiple other users, data including a set of terms associated with the other user;

selecting, by the vocabulary generator module of the server-based automated speech recognition system, a particular term from the set of terms associated with the target user;

identifying, by the vocabulary generator module of the server-based automated speech recognition system, a particular other term, from among the sets of terms that are associated with the multiple other users, that is indicated as related to the particular term;

obtaining, by the language model biaser module of the server-based automated speech recognition system and based on the particular other term, from among the sets of terms that are associated with the multiple other users, that is indicated as related to the particular term, a biased language model;

using, by the automated speech recognizer of the server-based automated speech recognition system, the biased language model generate a speech recognition output; and

providing, by the server-based automated speech recognition system, the speech recognition output to the device associated with the target user.

11. The method of claim 10, wherein, for each user, the set of terms associated with the user includes terms that are indicated as terms that have been accepted by the user.

12. The method of claim 10, wherein, for each term of the set of terms, the data indicates whether the term comprises an n-gram or an entity name.

13. The method of claim 10, wherein identifying the particular other term that is indicated as related to the particular term comprises:

identifying an identifier of a cluster of related terms that (i) includes the particular term from the set of terms that are associated with the target user, and (ii) includes one or more terms that are associated with the multiple other users; and

identifying, based at least on comparing the identifier of the cluster of related terms to one or more cluster identifiers associated with the multiple other users, a particular other term that is indicated as related to the particular term.

14. The method of claim 10, wherein selecting a particular term from the set of terms associated with the target user comprises selecting multiple particular terms from the set of terms associated with the target user.

15. The method of claim 14, wherein identifying the particular other term that is indicated as related to the particular term comprises identifying the particular other term that is indicated as related to the multiple particular terms.

16. A computer-implemented method comprising:

receiving, by a term pre-processor module of a server-based automated speech recognition system that includes (a) the term pre-processor module, (b) a cluster determiner module, (c) a user similarity determiner module, (d) a vocabulary generator module, (e) a language model biaser module, and (f) an automated speech recognizer, and from a device associated with a target user, (i) data including a set of terms associated with the target user that includes terms from one or more queries previously submitted by the target user, and, (ii) from each of multiple other users, data including a set of terms associated with the other user;

selecting, by the cluster determiner module of the server-based automated speech recognition system, a particular term from the set of terms associated with the target user;

identifying, by the cluster determiner module of the server-based automated speech recognition system, an identifier of a cluster of related terms that (i) includes the particular term from the set of terms that are associated with the target user, and (ii) includes one or more terms that are associated with the multiple other users;

selecting, by the user similarity determined module of the server-based automated speech recognition system, a particular other user based at least on comparing the identifier of the cluster of related terms to one or more cluster identifiers associated with the multiple other users;

selecting, by the vocabulary generator module of the server-based automated speech recognition system, one or more terms from the set terms that is associated with the particular other user;

obtaining, by the language biaser module of the server-based automated speech recognition system and based on the selected terms that are associated with the particular other user, a biased language model; and

using, by the automated speech recognizer of the server-based automated speech recognition system, the biased language model generate a speech recognition output; and

providing, by the server-based automated speech recognition system, the speech recognition output to the device associated with the target user.

17. The method claim 16, wherein the set of terms that is associated with the particular other user is part of the cluster of related terms that (i) includes the particular term from the set of terms that are associated with the target user, and (ii) includes one or more terms that are associated with the multiple particular other users.

18. The method of claim 16, wherein selecting one or more terms from the set terms that is associated with the particular other user comprises selecting one or more terms, from among the set of terms that is associated with the particular other user, that are identified as being rejected by the particular other user.

19. The method of claim 18, wherein obtaining the biased language model comprises obtaining, based on the selected terms that are identified as being rejected by the particular other user, a biased language model that is biased away from the selected terms that are identified as being rejected by the particular other user.