Dynamically configurable acoustic model for speech recognition system

• US 6,141,641 A
• Filed: 04/15/1998
• Issued: 10/31/2000
• Est. Priority Date: 04/15/1998
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Huang, Xuedong D., Hwang, Mei-Yuh
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Azad, Abul K.
• Application Number:
  US09/060,654
• Time in Patent Office: 930 Days
• Field of Search: 704/231, 704/254, 704/255, 704/256, 704/257, 704/243, 704/244, 704/245, 704/251, 704/253, 704/242
• US Class Current: 704/243
• CPC Class Codes: G10L 15/144 Training of HMMs
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering

Abstract

The present invention includes a system for recognizing speech based on an input data stream. The system includes an acoustic model which has a model size. The model is adjustable to a desired size based on characteristics of a computer system on which the recognition system is run.

Claims

1. A method of recognizing speech based on an input data stream, the method comprising:

providing a recognition system including an acoustic model having a model size;

adjusting the model size to a desired size based on characteristics of a computer system on which the recognition system is run based only on criteria other than a training data corpus used to train the acoustic model;

receiving the input data stream; and

performing speech recognition based on the acoustic model having the desired size.

2. The method of claim 1 wherein the acoustic model includes a plurality of deep senones, indicative of leaf senones on a senone tree, having ancestor senones, indicative of predecessor senones on the senone trees, and being characterized by at least one parameter, and wherein adjusting the model size comprises:

selecting an ancestor senone;

identifying a plurality of deep senones which are descendents of the selected ancestor senone; and

clustering parameters in the plurality of deep senones.

3. The method of claim 2 wherein identifying a plurality of deep senones comprises:

identifying a pair of parameters to be merged corresponding to the plurality of deep senones.

4. The method of claim 3 wherein each of the deep senones is represented by a single discrete output distribution and wherein identifying a pair of parameters to be merged comprises:

identifying a pair of output distributions to be merged based on an amount of reduction in likelihood of generating a dataset aligned with the pair of output distributions which results from merging the pair of output distributions.

5. The method of claim 4 wherein identifying a pair of output distributions to be merged comprises:

selecting first and second output distributions;

determining a likelihood of generating a first set of data and a second set of data, before merging the first and second selected output distributions, wherein the first set of data is aligned with the first selected output distribution and the second set of data is aligned with the second selected output distribution;

determining a reduction in the likelihood of generating the first and second sets of data after merging the first and second selected output distributions; and

identifying the pair of output distributions to be merged based on the reduction in the likelihood of generating the first and second sets of data.

6. The method of claim 3 wherein each deep senone is represented by at least one continuous density function, wherein the parameters comprise characteristics of the continuous density function, and wherein identifying a pair of parameters to be merged comprises:

identifying a pair of characteristics to be merged based on a reduction in likelihood of generating a set of data corresponding to the pair of characteristics which results from merging the pair of characteristics.

7. The method of claim 6 wherein the continuous density function comprises a Gaussian density function, wherein the characteristics of the Gaussian density function comprise a mean and covariance corresponding to the Gaussian density function and wherein identifying a pair of parameters to be merged based on a reduction in likelihood comprises:

selecting a first Gaussian density function and a second Gaussian density function;

determining a likelihood of generating a first set of data corresponding to the first Gaussian density function and a second set of data corresponding to the second Gaussian density function before merging the means and covariances of the first and second Gaussian density functions;

determining a reduction in the likelihood of generating the first and second sets of data after merging the means and covariances of the first and second Gaussian density functions; and

identifying a pair of at least one of means and covariances to be merged based on the reduction in the likelihood of generating the first and second sets of data.

8. The method of claim 7 wherein identifying a pair of at least one of means and covariances comprises:

initially identifying means and covariances to be merged; and

then identifying additional covariances to be merged.

9. The method of claim 8 and further comprising:

identifying merged means to be expanded into unmerged means and still maintain the acoustic model at the desired size.

10. The method of claim 1 wherein adjusting the model size comprises:

adjusting the model size based on preselected computational resources on the computer system.

11. The method of claim 10 wherein adjusting the model size based on preselected computational resources comprises:

adjusting the model size based on a memory capacity and a speed of the computer system.

12. The method of claim 10 wherein adjusting the model size based on preselected computational resources, comprises:

providing a user request requesting system information indicative of the preselected computational resources;

receiving a user input containing the system information; and

adjusting the model size based on the system information.

13. The method of claim 12 wherein the system information comprises a level of computer allocations the user desires to allocate to the recognition system.

14. A method of configuring a speech recognition system on a first computer used to run the speech recognition system wherein the speech recognition system was created on a second computer, the method comprising:

providing an acoustic model including a plurality of parameters and having a model size based on a number of the parameters included in the acoustic model, the acoustic model including leaf senones on a senone tree and ancestor senones indicative of predecessor senones on the senone tree;

obtaining model size criteria; and

adjusting the model size based on the size criteria by merging parameters from leaf senones to reduce a number of parameters in the acoustic model.

15. The method of claim 14 wherein obtaining model size criteria comprises:

obtaining system information indicative of operational characteristics of the first computer; and

wherein adjusting the model size comprises reducing the number of parameters in the acoustic model to a desired number based on the system information.

16. The method of claim 15 wherein obtaining system information comprises:

obtaining memory capacity and operational speed of the first computer.

17. The method of claim 15 wherein obtaining system information comprises:

obtaining a user input indicative of a portion of computational resources of the first computer the user wishes to allocate to speech recognition.

18. The method of claim 14 wherein merging parameters comprises:

merging parameters from only leaf senones having a common ancestor senone.

19. The method of claim 14 wherein each leaf senone is represented by a plurality of continuous density function, each continuous density functions being parameterized by a plurality of different parameters, and wherein merging parameters comprises:

selecting two continuous density functions to be merged; and

merging all of the parameters associated with the two selected continuous density functions.

20. The method of claim 14 wherein each leaf senone is represented by a plurality of continuous density functions, each continuous density function being parameterized by a plurality of different parameters, and wherein merging parameters comprises:

selecting two continuous density functions to be merged; and

merging only a subset of the plurality of parameters associated with the selected continuous density functions.

21. The method of claim 20 wherein the selected continuous density functions comprise Gaussian density functions and wherein merging only a subset comprises:

merging covariances of the selected Gaussian density functions.

22. The method of claim 21 wherein merging only a subset comprises:

merging means of the selected Gaussian density functions.

23. The method of claim 14 wherein the parameters each have an associated data set and wherein merging comprises:

merging certain of the parameters based on a reduction in a likelihood of generating the first and second data sets which results from merging the parameters.

24. A speech recognition system for recognizing input speech, comprising:

a feature extractor configured to extract features from the input speech;

an acoustic model having a model size and including models of speech units;

a search engine determining a likely output based on the features extracted and based on the acoustic model; and

a configuration module configured to adjust the model size based on system information corresponding to a computer system on which the speech recognition system is run, wherein the acoustic model is trained based on a training corpus and wherein the configuration module is configured to adjust the model size without accessing the training corpus.

25. The speech recognition system of claim 24 wherein the models in the acoustic model each include a plurality of parameters and wherein the configuration module is configured to obtain the system information by obtaining operational characteristics of the computer system and to reduce the number of parameters in the acoustic model to a desired number based on the operational characteristics.

26. The speech recognition system of claim 24 wherein the models of speech units include a plurality of leaf senones from a senone tree having ancestor senones, each leaf senone having at least one corresponding parameter and wherein the configuration module is configured to merge parameters in the leaf senone to reduce the number of parameters in the acoustic model to a desired size.

27. The speech recognition system of claim 26 wherein the configuration module is configured to merge parameters from only leaf senones having a common ancestor senone.

28. The speech recognition system of claim 26 wherein the configuration module is configured to selectively merge the parameters based on a reduction in likelihood of generating data sets aligned with the parameters to be merged, wherein the reduction results from merging the parameters.