Adapting enhanced acoustic models

• US 9,858,917 B1
• Filed: 02/02/2016
• Issued: 01/02/2018
• Est. Priority Date: 07/13/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Strope, Brian Patrick, Beeferman, Douglas H.
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US15/013,471
• Time in Patent Office: 700 Days
• Field of Search: 704231, 704236, 704239, 704251, 704252
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for enhancing speech recognition accuracy. In one aspect, a method includes receiving voice queries, obtaining, for one or more of the voice queries, feedback information that references an action taken by a user that submitted the voice query after reviewing a result of the voice query, generating, for the one or more voice queries, a posterior recognition confidence measure that reflects a probability that the voice query was correctly recognized, wherein the posterior recognition confidence measure is generated based at least on the feedback information for the voice query, selecting a subset of the one or more voice queries based on the posterior recognition confidence measures, and adapting an acoustic model using the subset of the voice queries.

Claims

1. A computer-implemented method comprising:

receiving a particular voice query submitted by a user;

receiving feedback information from a user after the user is presented with a set of search results that was generated in response to a search query that included a transcription of the particular voice query;

generating a score indicative of a probability that the transcription of the voice query is correct based at least on the feedback information;

determining that the score that is generated based at least on the feedback information satisfies a threshold; and

in response to determining that the score that is generated based at least on the feedback information satisfies the threshold, adapting an acoustic model using the particular voice query.

2. The method of claim 1, wherein generating a score indicative of the probability that the transcription of the voice query is correct comprises:

generating scores based on multiple different types of user actions indicated by feedback information, each of the multiple different types of user actions being taken by a respective user that submitted the voice query.

3. The method of claim 2, wherein generating the scores based on the multiple different types of user actions comprises:

making different adjustments to the scores based on the type of user action performed by the user that submitted the voice query.

4. The method of claim 1, wherein generating a score indicative of the probability that the transcription of the voice query is correct comprises:

determining, for at least one of the voice queries, a score indicative of a probability that a transcription of the voice query is correct, wherein the score is generated based on user feedback indicating that the user has selected a particular search result from the set of search results.

5. The method of claim 1, wherein generating a score indicative of the probability that the transcription of the voice query is correct comprises:

determining, for at least one of the voice queries, a score indicative of a probability that a transcription of the voice query is correct, wherein the score is generated based on user feedback indicating that, after submitting the voice query, the user that submitted the voice query did not submit a voice query for at least a predetermined period of time.

6. The method of claim 1, wherein generating a score indicative of the probability that the transcription of the voice query is correct comprises:

determining, for at least one of the voice queries, a score indicative of a probability that a transcription of the voice query is correct, wherein the score is generated based on user feedback indicating that the user has entered one or more characters into a query box after the set of search results are presented to the user.

7. The method of claim 1, further comprising:

receiving a voice query from a second user that is different than the user that previously submitted the particular voice query; and

in response to receiving the voice query from the second user that is different than the user that previously submitted the particular voice query, using the adapted acoustic model to transcribe the voice query received from the second user.

8. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving a particular voice query submitted by a user;

receiving feedback information from a user after the user is presented with a set of search results that was generated in response to a search query that included a transcription of the particular voice query;

generating a score indicative of a probability that the transcription of the voice query is correct based at least on the feedback information;

determining that the score that is generated based at least on the feedback information satisfies a threshold; and

in response to determining that the score that is generated based at least on the feedback information satisfies the threshold, adapting an acoustic model using the particular voice query.

9. The system of claim 8, wherein generating a score indicative of the probability that the transcription of the voice query is correct comprises:

generating scores based on multiple different types of user actions indicated by feedback information, each of the multiple different types of user actions being taken by a respective user that submitted the voice query.

10. The system of claim 8, wherein generating a score indicative of the probability that the transcription of the voice query is correct comprises:

determining, for at least one of the voice queries, a score indicative of a probability that a transcription of the voice query is correct, wherein the score is generated based on user feedback indicating that the user has selected a particular search result from the set of search results.

11. The system of claim 8 wherein generating a score indicative of the probability that the transcription of the voice query is correct comprises:

determining, for at least one of the voice queries, a score indicative of a probability that a transcription of the voice query is correct, wherein the score is generated based on user feedback indicating that, after submitting the voice query, the user that submitted the voice query did not submit a voice query for at least a predetermined period of time.

12. The system of claim 8, wherein generating a score indicative of the probability that the transcription of the voice query is correct comprises:

determining, for at least one of the voice queries, a score indicative of a probability that a transcription of the voice query is correct, wherein the score is generated based on user feedback indicating that the user has entered one or more characters into a query box after the set of search results are presented to the user.

13. The system of claim 8, further comprising:

receiving a voice query from a second user that is different than the user that previously submitted the particular voice query; and

in response to receiving the voice query from the second user that is different than the user that previously submitted the particular voice query, using the adapted acoustic model to transcribe the voice query received from the second user.

14. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving a particular voice query submitted by a user;

receiving feedback information from a user after the user is presented with a set of search results that was generated in response to a search query that included a transcription of the particular voice query;

generating a score indicative of a probability that the transcription of the voice query is correct based at least on the feedback information;

determining that the score that is generated based at least on the feedback information satisfies a threshold; and

in response to determining that the score that is generated based at least on the feedback information satisfies the threshold, adapting an acoustic model using the particular voice query.

15. The computer-readable medium of claim 14, wherein generating a score indicative of the probability that the transcription of the voice query is correct comprises:

generating scores based on multiple different types of user actions indicated by feedback information, each of the multiple different types of user actions being taken by a respective user that submitted the voice query.

16. The computer-readable medium of claim 14, wherein generating a score indicative of the probability that the transcription of the voice query is correct comprises:

determining, for at least one of the voice queries, a score indicative of a probability that a transcription of the voice query is correct, wherein the score is generated based on user feedback indicating that the user has selected a particular search result from the set of search results.

17. The computer-readable medium of claim 14, wherein generating a score indicative of the probability that the transcription of the voice query is correct comprises:

determining, for at least one of the voice queries, a score indicative of a probability that a transcription of the voice query is correct, wherein the score is generated based on user feedback indicating that, after submitting the voice query, the user that submitted the voice query did not submit a voice query for at least a predetermined period of time.

18. The computer-readable medium of claim 14, wherein generating a score indicative of the probability that the transcription of the voice query is correct comprises:

determining, for at least one of the voice queries, a score indicative of a probability that a transcription of the voice query is correct, wherein the score is generated based on user feedback indicating that the user has entered one or more characters into a query box after the set of search results are presented to the user.

19. The computer-readable medium of claim 14, further comprising:

receiving a voice query from a second user that is different than the user that previously submitted the particular voice query; and

in response to receiving the voice query from the second user that is different than the user that previously submitted the particular voice query, using the adapted acoustic model to transcribe the voice query received from the second user.