SPEECH RECOGNITION USING NON-PARAMETRIC MODELS

• US 20150371633A1
• Filed: 05/16/2013
• Published: 12/24/2015
• Est. Priority Date: 11/01/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Chelba, Ciprian I.
• Granted Patent: US 9,336,771 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/144 Training of HMMs
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for using non-parametric models in speech recognition. In some implementations, speech data is accessed. The speech data represents utterances of a particular phonetic unit occurring in a particular phonetic context, and the speech data includes values for multiple dimensions. Boundaries are determined for a set of quantiles for each of the multiple dimensions. Models for the distribution of values within the quantiles are generated. A multidimensional probability function is generated. Data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function are stored.

Claims

1. A method performed by data processing apparatus, the method comprising:

accessing, by the data processing apparatus, a aratus speech data that represents utterances of a particular phonetic unit occurring in a particular phonetic context that comprises one or more additional phonetic units, the speech data comprising values for multiple dimensions for each of the utterances;

determining, by the data processing apparatus, boundaries for a set of quantiles for each of the multiple dimensions based on the speech data that represents utterances of the particular phonetic unit occurring in the particular phonetic context;

generating, by the data processing apparatus and for each of the quantiles, a model that models the distribution of values within the quantile;

generating, by the data processing apparatus, a multidimensional probability function that indicates, for input speech data representing speech occurring in the particular phonetic context, a probability that the input speech data will have values that correspond to a given set of the quantiles for the multiple dimensions;

storing, by the data processing apparatus, data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function as a portion of an acoustic model corresponding to the particular phonetic unit occurring in the particular phonetic context; and

using, by the data processing apparatus, the stored data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function to perform speech recognition for an utterance.

2. The method of claim 1, wherein generating the multidimensional probability function comprises generating a multidimensional probability function that indicates, for an input speech frame representing speech occurring in the particular phonetic context, a probability that the input speech frame will have values that correspond to a given set of the quantiles for the multiple dimensions, where the input speech frame is a multidimensional real-valued vector.

3. The method of claim 1, wherein determining the boundaries for the set of quantiles for each of the multiple dimensions comprises determining the boundaries for equal occupancy quantiles such that each of the equal occupancy quantiles includes an equal number of values from the accessed speech data, each equal occupancy quantile representing a range of values in a dimension that does not overlap with any other equal occupancy quantile.

4. The method of claim 1, wherein generating models for the distribution of values within the quantiles comprises generating, for each of the quantiles, a univariate Gaussian distribution based on values in the speech data that correspond to the quantile.

5. The method of claim 1, wherein generating the multidimensional probability function comprises generating an n-gram model wherein the n-grams are sequences of quantile identifiers and the sequences include quantile identifiers for quantiles in at least two dimensions.

6. The method of claim 5, wherein generating the generating the multidimensional probability function comprises determining the number and identity of the n-grams for which data is included in the n-gram model based on the accessed speech data.

7. The method of claim 1, wherein determining boundaries for the set of quantiles for each of the multiple dimensions comprises determining the boundaries such that the number of quantiles in the set of quantiles for each of the multiple dimensions is based on the accessed speech data.

21. The method of claim 1, wherein generating the multi-dimensional probability function comprises determining conditional probabilities that each indicate a probability that a value in a first dimension will be in a particular quantile of the first dimension given at least that a value in a second dimension occurs in a particular quantile of the second dimension.

22. The method of claim 1, further comprising generating other portions of the acoustic model using a plurality of processing nodes, each processing node being assigned different sequences of phonetic units, each of the different processing nodes performing accessing, determining, generating of models, generating of a multi-dimensional probability function, and storing for each of its assigned sequences of phonetic units to generate portions of the acoustic model corresponding to the assigned sequences of phonetic units.

8. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, from a client device over a computer network, audio data that describes an utterance of a user;

accessing stored data of an acoustic model that was generated by;

accessing speech data that represents utterances of a particular phonetic unit occurring in a particular phonetic context that comprises one or more additional phonetic units, the speech data comprising values for multiple dimensions for each of the utterances;

determining boundaries for a set of quantiles for each of the multiple dimensions based on the speech data that represents utterances of the particular phonetic unit occurring in the particular phonetic context;

generating, for each of the quantiles, a model that models the distribution of values within the quantile;

generating a multidimensional probability function that indicates, for input speech data representing speech occurring in the particular phonetic context, a probability that the input speech data will have values that correspond to a given set of the quantiles for the multiple dimensions;

storing data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function as a portion of an acoustic model corresponding to the particular phonetic unit occurring in the particular phonetic context;

using the stored data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function to determine a transcription for the utterance; and

providing, to the client device and over the computer network, the transcription for the utterance.

9. The system of claim 8, wherein generating the multidimensional probability function comprises generating a multidimensional probability function that indicates, for an input speech frame representing speech occurring in the particular phonetic context, a probability that the input speech frame will have values that correspond to a given set of the quantiles for the multiple dimensions, where the input speech frame is a multidimensional real-valued vector.

10. The system of claim 8, wherein determining the boundaries for the set of quantiles for each of the multiple dimensions comprises determining the boundaries for equal occupancy quantiles such that each of the equal occupancy quantiles includes an equal number of values from the accessed speech data, each equal occupancy quantile representing a range of values in a dimension that does not overlap with any other equal occupancy quantile.

11. The system of claim 8, wherein generating models for the distribution of values within the quantiles comprises generating, for each of the quantiles, a univariate Gaussian distribution based on values in the speech data that correspond to the quantile.

12. The system of claim 9, wherein generating the multidimensional probability function comprises generating an n-gram model wherein the n-grams are sequences of quantile identifiers and the sequences include quantile identifiers for quantiles in at least two dimensions.

13. The system of claim 12, wherein generating the generating the multidimensional probability function comprises determining the number and identity of the n-grams for which data is included in the n-gram model based on the accessed speech data.

14. The system of claim 8, wherein determining boundaries for the set of quantiles for each of the multiple dimensions comprises determining the boundaries such that the number of quantiles in the set of quantiles for each of the multiple dimensions is based on the accessed speech data.

15. A non-transitory computer storage medium storing a computer program, the program comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving, from a client device over a computer network, audio data that describes an utterance of a user;

accessing stored data of an acoustic model that was generated by;

accessing speech data that represents utterances of a particular phonetic unit occurring in a particular phonetic context that comprises one or more additional phonetic units, the speech data comprising values for multiple dimensions for each of the utterances;

determining boundaries for a set of quantiles for each of the multiple dimensions based on the speech data that represents utterances of the particular phonetic unit occurring in the particular phonetic context;

generating, for each of the quantiles, a model that models the distribution of values within the quantile;

generating a multidimensional probability function that indicates, for input speech data representing speech occurring in the particular phonetic context, a probability that the input speech data will have values that correspond to a given set of the quantiles for the multiple dimensions;

storing data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function as a portion of an acoustic model corresponding to the particular phonetic unit occurring in the particular phonetic context;

using the stored data indicating the boundaries of the quantiles, the models for the distribution of values in the quantiles, and the multidimensional probability function to determine a transcription for the utterance; and

providing, to the client device and over the computer network, the transcription for the utterance.

19. The non-transitory computer storage medium of claim 15, wherein generating the multidimensional probability function comprises generating an n-gram model wherein the n-grams are sequences of quantile identifiers and the sequences include quantile identifiers for quantiles in at least two dimensions.

20. The non-transitory computer storage medium of claim 19, wherein generating the generating the multidimensional probability function comprises determining the number and identity of the n-grams for which data is included in the n-gram model based on the accessed speech data.

16. -18. (canceled)

23. The method of claim 23, wherein using the stored data to determine the transcription for the utterance comprises using the portions of the acoustic model generated by the different processing nodes to determine a transcription for the utterance.