Caching speech recognition scores

• US 9,858,922 B2
• Filed: 06/23/2014
• Issued: 01/02/2018
• Est. Priority Date: 06/23/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Weinstein, Eugene, Kumar, Sanjiv, Moreno, Ignacio L., Senior, Andrew W., Bhat, Nikhil Prasad
• Primary Examiner(s): Safaipour, Houshang
• Assistant Examiner(s): Shah, Bharatkumar S
• Application Number:
  US14/311,557
• Publication Number:
  US 20150371631A1
• Time in Patent Office: 1,289 Days
• Field of Search: 704256
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/285 Memory allocation or algorithm optimisation to reduce hardware requirements

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for caching speech recognition scores. In some implementations, one or more values comprising data about an utterance are received. An index value is determined for the one or more values. An acoustic model score for the one or more received values is selected, from a cache of acoustic model scores that were computed before receiving the one or more values, based on the index value. A transcription for the utterance is determined using the selected acoustic model score.

Claims

1. A method performed by one or more computers of an automated speech recognition system, the method comprising:

receiving, by the one or more computers of the automated speech recognition system, one or more values comprising data that indicates acoustic characteristics of an utterance;

determining, by the one or more computers of the automated speech recognition system, an index value for the one or more values;

accessing, by the one or more computers of the automated speech recognition system, a cache of acoustic model scores that were computed before receiving the one or more values, wherein the cache indicates a mapping of the acoustic model scores to index values, the mapping being determined before receiving the one or more values;

selecting, by the one or more computers of the automated speech recognition system, a subset of the acoustic model scores in the cache based on the determined index value;

determining, by the one or more computers of the automated speech recognition system, a transcription for the utterance using the selected subset of the acoustic model scores; and

providing, by the one or more computers of the automated speech recognition system, an output of the automated speech recognition system, wherein the output of the automated speech recognition system includes data indicating the transcription determined using the selected subset of the acoustic model scores.

2. The method of claim 1, wherein determining the index value comprises generating a hash value for the one or more values using a locality sensitive hash function.

3. The method of claim 2, wherein selecting the subset of the acoustic model scores in the cache comprises selecting an acoustic model score previously generated for a set of values that matches the received one or more values.

4. The method of claim 2, wherein selecting the subset of the acoustic model scores in the cache comprises selecting an acoustic model score previously generated for a set of values that does not exactly match the received one or more values, the acoustic model score approximating an acoustic model score for the received one or more values.

5. The method of claim 1, wherein the one or more values comprise information derived from a particular speech frame and information derived from contextual speech frames occurring before or after the particular frame;

and

wherein determining the index value for the one or more values comprises determining, as the index value, a hash value for the particular speech frame based on the information derived from a particular speech frame and information derived from the contextual speech frames occurring before or after the particular frame.

6. The method of claim 5, wherein selecting the subset of the acoustic model scores in the cache comprises selecting an acoustic model score indicating a likelihood that the particular speech frame is an occurrence of a particular phonetic unit in the utterance.

7. The method of claim 1, further comprising selecting multiple acoustic model scores from the cache based on the index value, each of the multiple acoustic model scores indicating a likelihood that the utterance includes a different phonetic unit in a portion corresponding to the one or more values.

8. The method of claim 1, wherein receiving the one or more values comprises, receiving the one or more values from a client device over a network;

and

wherein the method further comprises providing the transcription to the client device over the network.

9. The method of claim 1, wherein the one or more values comprise results of compressing speech features for the utterance using vector quantization.

17. The method of claim 1, wherein selecting a subset of the acoustic model scores in the cache based on the determined index value comprises determining one or more scores that the mapping indicates correspond to the determined index value.

18. The method of claim 1, wherein receiving the one or more values comprising data about an utterance comprises receiving the one or more values, by the one or more computers of the automated speech recognition system, from a client device over a network;

and

wherein providing the data indicating the transcription as an output of the automated speech recognition system comprises providing the data indicating the transcription, by the one or more computers of the automated speech recognition system, to the client device over the network.

19. The method of claim 1, wherein receiving the one or more values comprising data about an utterance comprises receiving, by the one or more computers of the automated speech recognition system, data including a particular portion of the utterance;

wherein selecting a subset of the acoustic model scores in the cache comprises selecting, by the one or more computers of the automated speech recognition system, a particular acoustic model score indicating a likelihood corresponding to a particular phonetic unit; and

wherein determining the transcription for the utterance comprises determining the transcription using the particular acoustic model score to represent a likelihood that the particular portion of the utterance is an occurrence of the particular phonetic unit.

20. The method of claim 1, wherein the cache maps an index value to a set of multiple acoustic model scores, where each of the acoustic model scores indicates a likelihood for a different phonetic unit.

10. An automated speech recognition system comprising:

one or more computers; and

one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving one or more values comprising data that indicates acoustic characteristics of an utterance;

determining an index value for the one or more values;

accessing a cache of acoustic model scores that were computed before receiving the one or more values, wherein the cache indicates a mapping of the acoustic model scores to index values, the mapping being determined before receiving the one or more values;

selecting a subset of the acoustic model scores in the cache based on the determined index value;

determining a transcription for the utterance using the selected subset of the acoustic model scores; and

providing an output of the automated speech recognition system, wherein the output of the automated speech recognition system includes data indicating the transcription determined using the selected subset of the acoustic model scores.

11. The system of claim 10, wherein determining the index value comprises generating an index of a hash table.

12. The system of claim 11, wherein selecting the subset of the acoustic model scores in the cache comprises selecting an acoustic model score previously generated for a set of values that does not exactly match the received one or more values, the acoustic model score approximating an acoustic model score for the received one or more values.

13. The system of claim 10, wherein the one or more values comprise information derived from a particular speech frame and information derived from contextual speech frames occurring before or after the particular frame;

and

wherein determining the index value for the one or more values comprises determining, as the index value, a hash value for the particular speech frame based on the information derived from a particular speech frame and information derived from the contextual speech frames occurring before or after the particular frame.

14. A computer storage device storing instructions that, when executed by one or more computers of an automated speech recognition system, cause the one or more computers to perform operations comprising:

receiving one or more values comprising data that indicates acoustic characteristics of an utterance;

determining an index value for the one or more values;

accessing a cache of acoustic model scores that were computed before receiving the one or more values, wherein the cache indicates a mapping of the acoustic model scores to index values, the mapping being determined before receiving the one or more values;

selecting a subset of the acoustic model scores in the cache based on the determined index value;

determining a transcription for the utterance using the selected subset of the acoustic model scores; and

providing an output of the automated speech recognition system, wherein the output of the automated speech recognition system includes data indicating the transcription determined using the selected subset of the acoustic model scores.

15. The computer storage device of claim 14, wherein the index value corresponds to a storage location in the cache;

wherein selecting the subset of the acoustic model scores in the cache based on the determined index value comprises;

using the mapping to determine the storage location corresponding to the index value;

retrieving, from the storage location corresponding to the index value, a set of multiple acoustic model scores, each of the multiple acoustic model scores indicating a likelihood of occurrence for a different phonetic unit; and

selecting the set of acoustic model scores retrieved from the storage location corresponding to the index value as the subset.

16. The computer storage device of claim 14, wherein the one or more values comprise information derived from a particular speech frame and information derived from contextual speech frames occurring before or after the particular frame;

wherein determining the index value for the one or more values comprises determining a hash value for the particular speech frame based on the information derived from a particular speech frame and information derived from the contextual speech frames occurring before or after the particular frame;

wherein selecting the subset of the acoustic model scores comprises retrieving one or more acoustic model scores from the cache using the hash value; and

wherein determining a transcription for the utterance using the selected subset of the acoustic model scores comprises determining a phonetic unit corresponding to the particular speech frame based on the retrieved one or more acoustic model scores.