Automatic language model update

• US 8,423,359 B2
• Filed: 05/24/2010
• Issued: 04/16/2013
• Est. Priority Date: 04/03/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Cohen, Michael H., Baluja, Shumeet, Moreno, Pedro J.
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Serrou, Abdelali
• Application Number:
  US12/786,102
• Publication Number:
  US 20110213613A1
• Time in Patent Office: 1,058 Days
• Field of Search: 704/231, 704/235, 704/251, 704/270.1, 704/270, 704/271
• US Class Current: 704/231
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting

Abstract

A method for generating a speech recognition model includes accessing a baseline speech recognition model, obtaining information related to recent language usage from search queries, and modifying the speech recognition model to revise probabilities of a portion of a sound occurrence based on the information. The portion of a sound may include a word. Also, a method for generating a speech recognition model, includes receiving at a search engine from a remote device an audio recording and a transcript that substantially represents at least a portion of the audio recording, synchronizing the transcript with the audio recording, extracting one or more letters from the transcript and extracting the associated pronunciation of the one or more letters from the audio recording, and generating a dictionary entry in a pronunciation dictionary.

Claims

1. A computer-implemented method, comprising:

obtaining, at a server system, information that identifies a level of inclusion of a particular word in queries that were received over a recent period of time;

modifying, by the server system, a speech recognition model to revise occurrence data for the particular word based on the level of inclusion of the particular word in the queries that were received over the recent period of time,

wherein revising occurrence data for the particular word comprises assigning weights to occurrences of the particular word in the queries that were received over the recent period of time, the assigned weights being higher for occurrences of the particular word received over the recent period of time than weights assigned to occurrences of the particular word received outside of the recent period of time and before the recent period of time;

receiving, at the server system and from a mobile computing device, a verbal query;

translating, by the server system and using the modified speech recognition model, the verbal query into”

textual query, wherein the textual query includes the particular word; and

determining whether the textual query represents a verbal search query or whether the textual query represents a verbal system command.

2. The computer-implemented method of claim 1, wherein the queries that were received over the recent period of time is a portion of all queries that were received and that include the particular word.

3. The computer-implemented method of claim 1, wherein the recent period of time when the queries were received is a period of time of receipt of the queries by a search engine system, and wherein the queries were received from a plurality of computerized devices.

4. The computer-implemented method of claim 1, wherein the recent period of time when the queries were received is a period of time of transmission of the queries by a plurality of computerized devices to a search engine system.

5. The computer-implemented method of claim 1, wherein the recent period of time when the queries were received is a period of time of audio recordings of the queries.

6. The computer-implemented method of claim 1, wherein the queries are textural search queries.

7. The computer-implemented method of claim 1, wherein the queries are verbal search queries.

8. The computer-implemented method of claim 1, further comprising:

collecting search results that are responsive to the textual query by accessing an index of documents in a search engine system; and

transmitting the collected search results for use by the mobile computing device.

9. The computer-implemented method of claim 8, further comprising transmitting, with the collected search results and for use by the mobile computing device, the textual query.

10. The computer-implemented method of claim 8, wherein the collected search results and the textual query are formatted for transmission so as to cause the mobile computing device to automatically display, in a concurrent view, the collected search results and the textual query.

11. The computer-implemented method of claim 1, further comprising modifying, by the server system, the speech recognition model to revise a concurrence weighting for the particular word, wherein the concurrence weighting provides a determination of the likelihood of appearance of the particular word and a second word being adjacent, and wherein the concurrence weighting is generated based on the presence of the particular word and the second word together in one or more search queries.

12. The computer-implemented method of claim 11, wherein the concurrence weighting is generated based on the particular word and the second word being adjacent in one or more search queries.

13. A system comprising:

a server system including one or more machine-readable storage devices that are programmed with instructions that, when executed by one or more programmable processors, obtain information that identifies a level of inclusion of a particular word in queries that were received over a defined recent period of time;

an updater, embodied in one or more machine-readable storage devices that are programmed with instructions that, when executed by one or more programmable processors, modify a speech recognition model to revise occurrence data for the particular word based on the level of inclusion of the particular word in the queries that were received over the defined recent period of time,

wherein inclusion of the particular word in the queries that were received over the defined recent period of time influences the occurrence data more strongly than does inclusion of the particular word in queries that were received over time outside of the defined recent period of time and before the defined recent period of time; and

a speech recognition system, embodied in one or more machine-readable storage devices that are programmed with instructions that, when executed by one or more programmable processors;

(i) receive a verbal query, (ii) translate, using the modified speech recognition model, the verbal query into a textual query, wherein the textual query includes the particular word, and (iii) determine whether the textual query represents a verbal search query or whether the textual query represents a verbal system command.

14. The system of claim 13, wherein the updater is stored on a mobile computing device.

15. The system of claim 14, wherein the server system transmits information to the mobile computing device that enables the speech recognition model to be modified by the updater.

16. A system comprising:

a server system that includes one or more machine-readable storage devices that are programmed with instructions that, when executed by one or more programmable processors, receive information that identifies when queries that include a particular word were received;

means for modifying a speech recognition model to revise occurrence data for the word based on when the particular word was received in the queries so that a recent occurrence of the particular word in first queries more strongly influences the occurrence data for the particular word than a second occurrence of the particular word in a second query of the queries based on information identifying that the recent query was received after the second query; and

a speech recognition system, embodied in one or more machine-readable storage devices that are programmed with instructions that, when executed by one or more programmable processors;

(i) receive a verbal query, (ii) translate, using the modified speech recognition model, the verbal query into a textual query, wherein the textual query includes the particular word, and (iii) determine whether the textual query represents a verbal search query or whether the textual query represents a verbal system command.

17. The system of claim 16, further comprising a search engine system that includes one or more machine-readable storage devices that are programmed for execution by one or more programmable processors to receive the queries as search requests.

18. A computer-implemented method, comprising:

obtaining, at a server system, information that identifies a level of inclusion of a particular term in queries that were received by a search engine system as search requests and that are associated with a recent period of time, to an exclusion of queries that were received by the search engine system as search requests and that are associated with time outside of the recent period of time and before the recent period of time;

modifying, by the server system, a speech recognition model to revise occurrence data for the particular term based on the level of inclusion of the term in the queries that are associated with the recent period of time wherein inclusion of the particular term in the queries that are associated with the recent period of time influences the occurrence data more strongly than use of the particular term in queries that are associated with time outside of the recent period of time and before the recent period of time;

receiving, at the server system and from a mobile computing device, a verbal query;

translating, by the server system and using the modified speech recognition model, the verbal query into a textual query, wherein the textual query includes the particular term; and

determining whether the textual query represents a verbal search query or whether the textual query represents a verbal system command.

19. The computer-implemented method of claim 18, further comprising

transmitting, by the server system and for use by the remote computing device, the textual query.

20. The computer-implemented method of claim 18, wherein the queries that are associated with the recent period of time are queries that were received by the search system during the recent period of time.

21. A computer-implemented method, the method comprising:

transmitting queries, by a computing device to a remote server system over a recent period of time, so as to cause the server system to;

obtain information identifying a level of inclusion of a particular word in the queries, and

modify a speech recognition model to revise occurrence data for the particular word based on the level of inclusion of the particular word in the queries that were transmitted over the recent period of time,

wherein revising occurrence data for the particular word comprises assigning weights to occurrences of the particular word in the queries that were received over the recent period of time, the assigned weights being higher for occurrences of the particular word received over the recent period of time than weights assigned to occurrences of the particular word received outside of the recent period of time and before the recent period of time; and

transmitting, by the computing device to the server system, a verbal query, so as to cause the server system to;

translate the verbal query into a textual query using the modified speech recognition model, wherein the textual includes the particular word, and

determine whether the textual query represents a verbal search query or whether the textual query represents a verbal system command.

22. The computer-implemented method of claim 21, wherein the queries that were transmitted over the recent period of time is a portion of all queries that were transmitted by the computing device and that include the particular word.

23. The computer-implemented method of claim 21, wherein the transmitting of the verbal search query to the server system further causes the server system to collect search results that are responsive to the textual query by accessing an index of documents in a search engine system;

and further comprising receiving, at the particular computing device and from the remote server system, the collected search results.

24. A computer-implemented method, comprising:

obtaining, at a server system, information that identifies a level of inclusion of a particular word in queries that were received over a recent period of time, wherein the recent period of time when the queries were received is a period of time of transmission of the queries by a plurality of computerized devices to a search engine system, or a period of time of audio recordings of the queries; and

modifying, by the server system, a speech recognition model to revise occurrence data for the particular word based on the level of inclusion of the particular word in the queries that were received over the recent period of time,

wherein revising occurrence data for the particular word comprises assigning weights to occurrences of the particular word in the queries that were received over the recent period of time, the assigned weights being higher for occurrences of the particular word received over the recent period of time than weights assigned to occurrences of the particular word received outside the recent period of time and before the recent period of time.

25. The computer-implemented method of claim 24, wherein the recent period of time when the queries were received is the period of time of transmission of the queries by the plurality of computerized devices to the search engine system.

26. The computer-implemented method of claim 24, wherein the recent period of time when the queries were received is the period of time of audio recordings of the queries.

27. The computer-implemented method of claim 24, further comprising modifying, by the server system, the speech recognition model to revise a concurrence weighting for the particular word, wherein the concurrence weighting provides a determination of the likelihood of appearance of the particular word and a second word being adjacent, and wherein the concurrence weighting is generated based on the presence of the particular word and the second word together in one or more search queries.

28. The computer-implemented method of claim 27, wherein the concurrence weighting is generated based on the particular word and the second word being adjacent in one or more search queries.

29. A system comprising:

a server system including one or more machine-readable storage devices that are programmed with instructions that, when executed by one or more programmable processors, obtain information that identifies a level of inclusion of a particular word in queries that were received over a defined recent period of time, wherein the defined recent period of time when the queries were received is a defined period of time of transmission of the queries by a plurality of computerized devices to a search engine system, or a defined period of time of audio recordings of the queries; and

an updater, embodied in one or more machine readable storage devices that are programmed with instructions that, when executed by one or more programmable processors, modify a speech recognition model to revise occurrence data for the particular word based on the level of inclusion of the particular word in queries that were received over the defined recent period of time,

wherein inclusion of the particular word in the queries that were received over the defined period of time influences the occurrence data more strongly than does inclusion of the particular word in queries that were received over time outside of the defined recent period of time and before the recent period of time.

30. A system comprising:

a server system that includes one or more machine-readable storage devices that are programmed with instructions that, when executed by one or more programmable processors, receive information that identifies when queries that include a particular word were received, wherein the time when the queries were received is a time of transmission of the queries by a plurality of computerized devices to a search engine system, or a time of audio recordings of the queries; and

means for modifying a speech recognition model to revise occurrence data for the particular word based on when the particular word was received in queries so that a first occurrence of the particular word in a recent query of the queries more strongly influences the occurrence data for the particular word than a second occurrence of the particular word in a second query of the queries based on the information identifying that the recent query was received after the second query.

31. The system of claim 30, further comprising a search engine system that includes one or more machine-readable storage devices that are programmed for execution by one or more programmable processors to receive the queries as search requests.

32. A computer-implemented method, comprising:

obtaining, at a server system, information that identifies a level of inclusion of a particular term in queries that were received by a search engine system as search requests and that are associated with a recent period of time, to an exclusion of queries that were received by the search engine system as search requests and that are associated with time outside of the recent period of time and before the recent period of time, wherein the recent period of time is a period of time of transmission of the queries by a plurality of computerized devices to a search engine system, or a period of time of audio recordings of the queries; and

modifying, by the server system, a speech recognition model to revise occurrence data for the particular term based on the level of inclusion of the term in the queries that are associated with the recent period of time, wherein inclusion of the particular term in the queries that are associated with the recent period of occurrence data more strongly than use of the particular term in queries that are associated with time outside of the recent period of time and before the recent period of time.

33. The computer-implemented method of claim 32, further comprising:

receiving, at the server system and from a remote computing device, a verbal query;

translating, by the server system, the verbal query into textual data using the modified speech recognition model; and

transmitting, by the server system and for use by the remote computing device, the textual query.

34. A computer-implemented method, the method comprising:

transmitting queries, by a computing device to a remote server system over a recent period of time, so as to cause the server system to;

obtain information identifying a level of inclusion of a particular word in the queries, and

modify a speech recognition model to revise occurrence data for the particular word based on the level of inclusion of the particular word in the queries that were transmitted over the recent period of time,

wherein revising occurrence data for the particular word comprises assigning weights to occurrences of the particular word in the Queries that were received over the recent period of time, the assigned weights being higher for occurrences of the particular word received over the recent period of time than weights assigned to occurrences of the particular word received outside of the recent period of time and before the recent period of time, wherein the recent period of time is a period of time of transmission of the queries by the computing device to a search engine system, or a period of time of audio recordings of the queries.