Follow-up voice query prediction

• US 10,026,398 B2
• Filed: 07/08/2016
• Issued: 07/17/2018
• Est. Priority Date: 07/08/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Behzadi, Behshad, Osmakov, Dmitry, Baeuml, Martin, Skobeltsyn, Gleb
• Primary Examiner(s): ALBERTALLI, BRIAN LOUIS
• Application Number:
  US15/205,505
• Publication Number:
  US 20180012594A1
• Time in Patent Office: 739 Days
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/14 using statistical models, e.g. Hidden Markov Models [HMMs] G10L15/18 takes precedence
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for predicting follow-up queries to an initial transcription of an utterance. In some implementations, one or more follow-up queries that are pre-associated with a transcription of an initial utterance of a user are identified. A new or modified language model in which a respective probability associated with one or more of the follow-up queries is increased with respect to an initial language model is obtained. Subsequent audio data corresponding to a subsequent utterance of the user is then received. The subsequent audio data is processed using the new or modified language model to generate a transcription of the subsequent utterance. The transcription of the subsequent utterance is then provided for output to the user.

Claims

1. A computer-implemented method comprising:

obtaining an initial utterance of a user that is provided at a particular location;

identifying one or more follow-up queries that are pre-associated with a term that matches a transcription of an initial utterance of the user, wherein the follow-up queries are (i) different than the term, and (ii) are pre-associated with the term based on query log data indicating that other users of a search engine have previously submitted the follow-up queries at the particular location after submitting an initial query that includes the term at the particular location;

adjusting an initial language model to generate a modified language model, the modified language model specifying a respective probability associated with one or more of the follow-up queries that is increased with respect to the initial language model;

receiving subsequent audio data corresponding to a subsequent utterance of the user that is provided at the particular location;

processing the subsequent audio data using the modified language model to generate a transcription of the subsequent utterance; and

providing the transcription of the subsequent utterance for output to the user.

2. The method of claim 1, further comprising:

receiving initial audio data corresponding to an initial utterance of a user; and

processing the audio data using the initial language model to generate a transcription of the initial utterance.

3. The method of claim 1, wherein adjusting the initial language model comprises:

accessing a database including one or more follow-up query mappings, each follow-up query mapping specifying a list of candidate follow-up queries for (i) prior transcriptions of utterances of the user, (ii) prior search results associated with the user, and (iii) data associated with the prior search results associated the user;

determining that one or more terms of the transcription of the initial utterance of the user are included within the one or more follow-up query mappings; and

adjusting the initial language model to increase the respective probability associated with the list of candidate follow-up queries that correspond to the one or more terms of the transcription of the initial utterance of the user that are included within the one or more follow-up query mappings.

4. The method of claim 3, wherein the list of candidate follow-up queries that are included within the one or more follow-up query mappings is based at least on a user location associated with the initial utterance of the user.

5. The method of claim 3, wherein determining that one or more terms of the transcription of the initial utterance of the user are included within the one or more follow-up query mappings comprises determining that the particular context associated with the initial utterance of the user matches a context associated with a particular transcription of a prior utterance of the user.

6. The method of claim 3, wherein each of the one or more follow-up query mappings further specifies one or more follow-up terms for (i) prior transcriptions of utterances of the user, (ii) prior search results associated with the user, and (iii) data associated with the prior search results associated the user.

7. The method of claim 1, wherein processing the audio data using the modified language model comprises determining that a particular context associated with the initial utterance of the user matches a particular context associated with the subsequent utterance of the user.

8. A computer-implemented method comprising:

obtaining an initial utterance of a user that is provided at a particular location;

identifying one or more follow-up queries that are pre-associated with a term that matches a transcription of an initial utterance of the user, wherein the follow-up queries are (i) different than the term, and (ii) are pre-associated with the term based on query log data indicating that other users of a search engine have previously submitted the follow-up queries at the particular location after submitting an initial query that includes the term at the particular location;

adjusting an initial language model to generate a modified language model, the modified language model specifying a respective probability associated with one or more of the follow-up queries that is increased with respect to the initial language model;

receiving subsequent audio data corresponding to a subsequent utterance of the user that is provided at the particular location;

processing the subsequent audio data using the modified language model to generate a transcription of the subsequent utterance; and

providing the transcription of the subsequent utterance for output to the user.

9. The method of claim 8, further comprising:

receiving initial audio data corresponding to an initial utterance of a user; and

processing the audio data using the initial language model to generate one or more search results associated with a transcription of an initial utterance of the user.

10. The method of claim 8, wherein adjusting the initial language model comprises:

accessing a database including one or more follow-up query mappings, each follow-up query mapping specifying a list of candidate follow-up queries for (i) prior transcriptions of utterances of the user, (ii) prior search results associated with the user, and (iii) data associated with the prior search results associated the user;

determining that one or more terms of the transcription of the initial utterance of the user are included within the one or more follow-up query mappings; and

adjusting the initial language model to increase the respective probability associated with the list of candidate follow-up queries that correspond to the one or more terms of the transcription of the initial utterance of the user that are included within the one or more follow-up query mappings.

11. The method of claim 10, wherein the list of candidate follow-up queries that are included within the one or more follow-up query mappings is based at least on a user location associated with the initial utterance of the user.

12. The method of claim 10, wherein determining that one or more terms of the transcription of the initial utterance of the user are included within the one or more follow-up query mappings comprises determining that the particular context associated with the initial utterance of the user matches a context associated with a particular transcription of a prior utterance of the user.

13. The method of claim 10, wherein each of the one or more follow-up query mappings further specifies one or more follow-up terms for (i) prior transcriptions of utterances of the user, (ii) prior search results associated with the user, and (iii) data associated with the prior search results associated the user.

14. The method of claim 8, wherein processing the audio data using the modified language model comprises determining that a particular context associated with the initial utterance of the user matches a particular context associated with the subsequent utterance of the user.

15. A computer-implemented method comprising:

obtaining an initial utterance of a user that is provided at a particular location;

identifying one or more follow-up queries that are pre-associated with a term that matches a transcription of an initial utterance of the user, wherein the follow-up queries are (i) different than the term, and (ii) are pre-associated with the term based on query log data indicating that other users of a search engine have previously submitted the follow-up queries at the particular location after submitting an initial query that includes the term at the particular location;

adjusting an initial language model to generate a modified language model, the modified language model specifying a respective probability associated with one or more of the follow-up queries that is increased with respect to the initial language model;

receiving subsequent audio data corresponding to a subsequent utterance of the user that is provided at the particular location;

processing the subsequent audio data using the modified language model to generate a transcription of the subsequent utterance; and

providing the transcription of the subsequent utterance for output to the user.

16. The method of claim 15, further comprising:

receiving initial audio data corresponding to an initial utterance of a user; and

processing the audio data using the initial language model to generate one or more search results associated with a transcription of an initial utterance of the user.

17. The method of claim 15, wherein adjusting the initial language model comprises:

accessing a database including one or more follow-up query mappings, each follow-up query mapping specifying a list of candidate follow-up queries for (i) prior transcriptions of utterances of the user, (ii) prior search results associated with the user, and (iii) data associated with the prior search results associated the user;

determining that one or more terms of the transcription of the initial utterance of the user are included within the one or more follow-up query mappings; and

adjusting the initial language model to increase the respective probability associated with the list of candidate follow-up queries that correspond to the one or more terms of the transcription of the initial utterance of the user that are included within the one or more follow-up query mappings.

18. The method of claim 17, wherein the list of candidate follow-up queries that are included within the one or more follow-up query mappings is based at least on a user location associated with the initial utterance of the user.