Speech recognition system having a quantizer using a single robust codebook designed at multiple signal to noise ratios

• US 6,003,003 A
• Filed: 06/27/1997
• Issued: 12/14/1999
• Est. Priority Date: 06/27/1997
• Status: Expired due to Term

• Current Assignee: RPX Corporation
• Original Assignee: Advanced Micro Devices, Inc.
• Inventors: Cong, Lin, Asghar, Safdar M.
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Storm, Donald L.
• Application Number:
  US08/883,979
• Time in Patent Office: 900 Days
• Field of Search: 704/221, 704/222, 704/227, 704/226, 704/231, 704/233, 704/243, 704/244, 704/256
• US Class Current: 704/243
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering

Abstract

In one embodiment, a speech recognition system is organized with a fuzzy matrix quantizer with a single codebook representing u codewords. The single codebook is designed with entries from u codebooks which are designed with respective words at multiple signal to noise ratio levels. Such entries are, in one embodiment, centroids of clustered training data. The training data is, in one embodiment, derived from line spectral frequency pairs representing respective speech input signals at various signal to noise ratios. The single codebook trained in this manner provides a codebook for a robust front end speech processor, such as the fuzzy matrix quantizer, for training a speech classifier such as a u hidden Markov models and a speech post classifier such as a neural network. In one embodiment, a fuzzy Viterbi algorithm is used with the hidden Markov models to describe the speech input signal probabilistically.

Claims

1. A speech recognition system comprising:

a first quantizer to receive a representation of a speech input signal and generate quantization output data representing the closeness of the represented speech input signal to codewords in a single codebook, the single codebook of the quantizer representing a vocabulary of u words, where u is a non-negative integer greater than one and the single codebook further having a mixture of s signal to noise ratio codeword entries; and

a speech signal processor to receive the quantization output data from the quantizer, the speech signal processor having processes trained with respective quantization output data for the u vocabulary words and having speech input signal classifying output data as recognized speech.

2. The speech recognition system of claim 1 further comprising:

a speech classifier to receive the speech input signal classifying output data and having output data to classify the speech input signal.

3. The speech recognition system of claim 2 wherein the speech classifier is a multilevel perceptron neural network trained solely with the speech input signal classifying output data from the speech signal processor.

4. The speech recognition system of claim 1 further comprising:

a second quantizer having s times u codebooks, one codebook for each of the u vocabulary words at each of s signal to noise ratios, wherein each of the codebooks has C codeword entries, and the single codebook of the first quantizer has a mixture of each corresponding signal to noise ratio codeword entry from each of the s times u codebooks.

5. The speech recognition system of claim 4 wherein the mixture of each corresponding signal to noise ratio codeword entry from each of the s times u codebooks is determined by clustering corresponding signal to noise ratio C codeword entries from each of the s times u codebooks and determining u times C codeword entries.

6. The speech recognition system of claim 4 wherein the first quantizer is a fuzzy matrix quantizer.

7. The speech recognition system of claim 1 wherein the quantization output data are observation sequences of indices representing the closeness of the speech input signal to each codeword in the single codebook.

8. The speech recognition system of claim 1 wherein the speech signal processor comprises:

a plurality of hidden Markov models each modeling a respective word in the vocabulary of u words, wherein the quantization output data representing the closeness of the speech input signal to the codewords is an observation sequence of indices produced by the speech signal processor; and

a probability module to determine respective probabilities of each hidden Markov model producing the observation sequence representing the speech input signal.

9. The speech recognition system of claim 8 wherein the probability module includes a fuzzy Viterbi algorithm.

10. The speech recognition system of claim 1 wherein the speech input signal is represented by one set of P order line spectral pairs and N successive speech input signals represent a word.

11. The speech recognition system of claim 1 further comprising:

a communication device having the first quantizer and speech signal processor.

12. The speech recognition system as in claim 1 wherein each word represented in the single codebook is designed with test speech input signals corrupted by s signal to noise ratios, where s is a non-negative integer and the speech signal processor includes u hidden Markov models.

13. The speech recognition system as in claim 12 wherein:

the speech classifier further includes a Viterbi algorithm;

the Viterbi algorithm and the u hidden Markov models are capable of generating probabilistic speech input signal classifying output data; and

the speech recognition system further comprising;

a neural network to utilize the probabilistic speech input signal classifying output data to;

(i) train the neural network to recognize the speech input signal as speech and (ii) recognize the speech input signal as speech.

14. A speech recognition system comprising:

a means for computing P order linear predictive code prediction coefficients to represent each a word input signal segmented into N frames, where P and N are a non-negative integers;

means for deriving line spectral pair coefficients from the prediction coefficients;

a fuzzy matrix quantizer for processing a P×

N matrix of the line spectral pair coefficients and providing fuzzy matrix quantization data from a single codebook having a group of C codewords for each of u vocabulary words, wherein each of the u groups of C codewords is derived from s groups of C codewords designed at s signal to noise ratios, wherein s and C are a non-negative integers;

a plurality of u hidden Markov models, respectively trained using the single codebook, for modeling respective speech processes and for producing respective output data in response to receipt of the quantization data;

a means for receiving the quantization data and the respective output data of the u hidden Markov models for determining a probability for each of the u hidden Markov models that the respective hidden Markov model produced the quantization data; and

means for receiving the probabilities for training to classify the word input signal and for classifying the word input signal as one of the u vocabulary words.

15. The speech recognition system as in claim 14 wherein u is greater than 1.

16. A method comprising the steps of:

designing a single codebook having a vocabulary of u words, each word being represented by codewords designed with test speech input signals corrupted by s signal to noise ratios, where u and s are non-negative integers;

generating a respective response by the designed single codebook to each of the test speech input signals corrupted by the s signal to noise ratios; and

training a hidden Markov model for each of the u words with each of the responses generated by the single codebook.

17. The method of claim 16 further comprising the steps of:

receiving a speech input signal representation;

generating a response by the designed single codebook to the speech input signal representation; and

computing, for each of the u hidden Markov models, respective probabilities of the hidden Markov model classifying the speech input signal representation as one of the u vocabulary words.

18. The method of claim 17 further comprising the step of:

providing the respective probability from each of the hidden Markov models to a multilevel perceptron neural network to classify the speech input signal as one of the u vocabulary words.

19. The method of claim 17 further comprising the step of:

training a speech classifier with each of the respective probabilities of each of the hidden Markov models.

20. The method as in claim 19 wherein the speech classifier is a neural network.

21. The method of claim 16 wherein the step of designing the single codebook comprises the steps of:

a) providing r times s word input signals to a feature extractor, wherein the r times s word input signals are respectively formed by an ith word of the u vocabulary words repeated r times at s signal to noise ratios, wherein r is a number of repetitions of the ith word;

b) dividing each word input signal into N frames;

c) computing linear predictive code (LPC) prediction coefficients for each of the N frames;

d) generating a spectral representation of each of the N frames using the computed LPC coefficients;

e) clustering the spectral representation of each of the N frames into C cells, wherein C is a non-negative integer;

f) generating a centroid for each of the C cells using the generated spectral representation of each of the N frames to form a codebook for the ith word;

g) repeating steps a) through f) for each of the remaining u vocabulary words;

h) combining each centroid in the kth cell in the ith codebook to form a single cell in a single codebook; and

i) repeating step h) for each of the remaining C cells to form the single codebook with C cells for each of the u vocabulary words.

22. The method of claim 21, wherein s equals 7, and u equals 100, and C equals 16.

23. The method of claim 16 wherein the single codebook is a fuzzy matrix quantized codebook, and the u hidden Markov models are trained with a fuzzy Viterbi algorithm.

24. The method of claim 16 wherein the step of generating a respective response by the designed single codebook comprises the steps of:

computing a relative closeness index between each codeword entry in the single codebook and a representation of the speech input signal; and

grouping each relative closeness index into an observation sequence response.

25. The method of claim 16 wherein each of the speech input signals is represented by P order line spectral pair coefficients.

26. The method of claim 16 wherein s equals nine, u equals 26, and C equals 32.

27. The method as in claim 16 wherein u is greater than one.

28. A method comprising the steps of:

designing a single codebook having a vocabulary of u words, each word being represented by codewords designed with test speech input signals corrupted by s signal to noise ratios, where u and s are non-negative integers;

generating a respective response by the designed single codebook to each of the test speech input signals corrupted by the s signal to noise ratios;

training a hidden Markov model for each of the u words with each of the responses generated by the single codebook;

for each test speech input signal, determining a respective probability for each of the hidden Markov models to classify the test speech input signal; and

training a neural network to recognize the speech input signal using each respective probability for each of the hidden Markov models.

29. The method as in claim 28 wherein s equals 7 and u is greater than one.