Speech recognition using non-parametric models

• US 9,336,771 B2
• Filed: 05/16/2013
• Issued: 05/10/2016
• Est. Priority Date: 11/01/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Chelba, Ciprian I.
• Primary Examiner(s): Pham, Tammy
• Assistant Examiner(s): Leland, III, Edwin S
• Application Number:
  US13/895,582
• Publication Number:
  US 20150371633A1
• Time in Patent Office: 1,090 Days
• Field of Search: 704/240
• US Class Current: 1/1
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/144 Training of HMMs
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for using non-parametric models in speech recognition. In some implementations, speech data is accessed. The speech data represents utterances of a particular phonetic unit occurring in a particular phonetic context, and the speech data includes values for multiple dimensions. Boundaries are determined for a set of quantiles for each of the multiple dimensions. Models for the distribution of values within the quantiles are generated. A multidimensional probability function is generated. Data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function are stored.

Claims

1. A method performed by one or more computers, the method comprising:

receiving, from a client device over a computer network, audio data that describes an utterance of a user;

accessing, by the one or more computers, stored data of an acoustic model that was generated by;

accessing, by a data processing apparatus, speech data that represents utterances of a particular phonetic unit occurring in a particular phonetic context, the speech data comprising values for multiple dimensions;

determining, by the data processing apparatus, boundaries for a set of quantiles for each of the multiple dimensions;

generating, by the data processing apparatus and for each of the quantiles, a model that models the distribution of values within the quantile;

generating, by the data processing apparatus, a multidimensional probability function that indicates, for input speech data representing speech occurring in the particular phonetic context, a probability that the input speech data will have values that correspond to a given set of the quantiles for the multiple dimensions, wherein generating the multidimensional probability function comprises generating an n-gram model wherein the n-grams are sequences of quantile identifiers and the sequences include quantile identifiers for quantiles in at least two dimensions; and

storing, by the data processing apparatus, data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function;

using, by the one or more computers, the stored data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function to perform speech recognition for an utterance; and

providing, to the client device and over the computer network, the transcription for the utterance.

2. The method of claim 1, wherein generating the multidimensional probability function comprises generating a multidimensional probability function that indicates, for an input speech frame representing speech occurring in the particular phonetic context, a probability that the input speech frame will have values that correspond to a given set of the quantiles for the multiple dimensions, where the input speech frame is a multidimensional real-valued vector.

3. The method of claim 1, wherein determining the boundaries for the set of quantiles for each of the multiple dimensions comprises determining the boundaries for equal occupancy quantiles such that each of the equal occupancy quantiles includes an equal number of values from the accessed speech data, each equal occupancy quantile representing a range of values in a dimension that does not overlap with any other equal occupancy quantile.

4. The method of claim 1, wherein generating models for the distribution of values within the quantiles comprises generating, for each of the quantiles, a univariate Gaussian distribution based on values in the speech data that correspond to the quantile.

5. The method of claim 1, wherein generating the generating the multidimensional probability function comprises determining the number and identity of the n-grams for which data is included in the n-gram model based on the accessed speech data.

6. The method of claim 1, wherein determining boundaries for the set of quantiles for each of the multiple dimensions comprises determining the boundaries such that the number of quantiles in the set of quantiles for each of the multiple dimensions is based on the accessed speech data.

7. The method of claim 1, wherein generating the multi-dimensional probability function comprises determining conditional probabilities that each indicate a probability that a value in a first dimension will be in a particular quantile of the first dimension given at least that a value in a second dimension occurs in a particular quantile of the second dimension.

8. The method of claim 1, further comprising generating other portions of the acoustic model using a plurality of processing nodes, each processing node being assigned different sequences of phonetic units, each of the different processing nodes performing accessing, determining, generating of models, generating of a multi-dimensional probability function, and storing for each of its assigned sequences of phonetic units to generate portions of the acoustic model corresponding to the assigned sequences of phonetic units.

9. The method of claim 1, wherein using the stored data to determine the transcription for the utterance comprises using the portions of the acoustic model generated by the different processing nodes to determine a transcription for the utterance.

10. The method of claim 1, wherein accessing the speech data comprises accessing speech data that represents multiple utterances of a particular phonetic unit that each occur in a same, particular phonetic context that comprises one or more additional phonetic units, the speech data comprising values for multiple dimensions for each of the multiple utterances;

wherein determining the boundaries for a set of quantiles comprises determining boundaries for a set of quantiles for each of the multiple dimensions based on the speech data that represents multiple utterances of the particular phonetic unit occurring in the particular phonetic context that includes the one or more additional phonetic units;

wherein generating the multidimensional probability function comprises generating a multidimensional probability function that indicates, for input speech data representing speech occurring in the particular phonetic context that includes the one or more additional phonetic units, a probability that the input speech data will have values that correspond to a given set of the quantiles for the multiple dimensions; and

wherein storing the data comprises storing data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function as a portion of an acoustic model corresponding to the particular phonetic unit occurring in the particular phonetic context that include the one or more additional phonetic units.

11. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, from a client device over a computer network, audio data that describes an utterance of a user;

accessing stored data of an acoustic model that was generated by;

accessing speech data that represents utterances of a particular phonetic unit occurring in a particular phonetic context, the speech data comprising values for multiple dimensions;

determining boundaries for a set of quantiles for each of the multiple dimensions;

generating, for each of the quantiles, a model that models the distribution of values within the quantile;

generating a multidimensional probability function that indicates, for input speech data representing speech occurring in the particular phonetic context, a probability that the input speech data will have values that correspond to a given set of the quantiles for the multiple dimensions, wherein generating the multidimensional probability function comprises generating an n-gram model wherein the n-grams are sequences of quantile identifiers and the sequences include quantile identifiers for quantiles in at least two dimensions;

storing data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function;

using the stored data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function to determine a transcription for the utterance; and

providing, to the client device and over the computer network, the transcription for the utterance.

12. The system of claim 11, wherein generating the multidimensional probability function comprises generating a multidimensional probability function that indicates, for an input speech frame representing speech occurring in the particular phonetic context, a probability that the input speech frame will have values that correspond to a given set of the quantiles for the multiple dimensions, where the input speech frame is a multidimensional real-valued vector.

13. The system of claim 11, wherein determining the boundaries for the set of quantiles for each of the multiple dimensions comprises determining the boundaries for equal occupancy quantiles such that each of the equal occupancy quantiles includes an equal number of values from the accessed speech data, each equal occupancy quantile representing a range of values in a dimension that does not overlap with any other equal occupancy quantile.

14. The system of claim 11, wherein generating models for the distribution of values within the quantiles comprises generating, for each of the quantiles, a univariate Gaussian distribution based on values in the speech data that correspond to the quantile.

15. The system of claim 11, wherein generating the generating the multidimensional probability function comprises determining the number and identity of the n-grams for which data is included in the n-gram model based on the accessed speech data.

16. The system of claim 11, wherein determining boundaries for the set of quantiles for each of the multiple dimensions comprises determining the boundaries such that the number of quantiles in the set of quantiles for each of the multiple dimensions is based on the accessed speech data.

17. The system of claim 11, wherein accessing the speech data comprises accessing speech data that represents multiple utterances of a particular phonetic unit that each occur in a same, particular phonetic context that comprises one or more additional phonetic units, the speech data comprising values for multiple dimensions for each of the multiple utterances;

wherein determining the boundaries for a set of quantiles comprises determining boundaries for a set of quantiles for each of the multiple dimensions based on the speech data that represents multiple utterances of the particular phonetic unit occurring in the particular phonetic context that includes the one or more additional phonetic units;

wherein generating the multidimensional probability function comprises generating a multidimensional probability function that indicates, for input speech data representing speech occurring in the particular phonetic context that includes the one or more additional phonetic units, a probability that the input speech data will have values that correspond to a given set of the quantiles for the multiple dimensions; and

wherein storing the data comprises storing data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function as a portion of an acoustic model corresponding to the particular phonetic unit occurring in the particular phonetic context that include the one or more additional phonetic units.

18. A non-transitory computer storage medium storing a computer program, the program comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving, from a client device over a computer network, audio data that describes an utterance of a user;

accessing stored data of an acoustic model that was generated by;

accessing speech data that represents utterances of a particular phonetic unit occurring in a particular phonetic context, the speech data comprising values for multiple dimensions;

determining boundaries for a set of quantiles for each of the multiple dimensions;

generating, for each of the quantiles, a model that models the distribution of values within the quantile;

generating a multidimensional probability function that indicates, for input speech data representing speech occurring in the particular phonetic context, a probability that the input speech data will have values that correspond to a given set of the quantiles for the multiple dimensions, wherein generating the multidimensional probability function comprises generating an n-gram model wherein the n-grams are sequences of quantile identifiers and the sequences include quantile identifiers for quantiles in at least two dimensions;

storing data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function;

using the stored data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function to determine a transcription for the utterance; and

providing, to the client device and over the computer network, the transcription for the utterance.

19. The non-transitory computer storage medium of claim 18, wherein generating the generating the multidimensional probability function comprises determining the number and identity of the n-grams for which data is included in the n-gram model based on the accessed speech data.

20. The non-transitory computer storage medium of claim 18, wherein accessing the speech data comprises accessing speech data that represents multiple utterances of a particular phonetic unit that each occur in a same, particular phonetic context that comprises one or more additional phonetic units, the speech data comprising values for multiple dimensions for each of the multiple utterances;

wherein determining the boundaries for a set of quantiles comprises determining boundaries for a set of quantiles for each of the multiple dimensions based on the speech data that represents multiple utterances of the particular phonetic unit occurring in the particular phonetic context that includes the one or more additional phonetic units;

wherein generating the multidimensional probability function comprises generating a multidimensional probability function that indicates, for input speech data representing speech occurring in the particular phonetic context that includes the one or more additional phonetic units, a probability that the input speech data will have values that correspond to a given set of the quantiles for the multiple dimensions; and

wherein storing the data comprises storing data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function as a portion of an acoustic model corresponding to the particular phonetic unit occurring in the particular phonetic context that include the one or more additional phonetic units.