Method and apparatus for formant tracking using a residual model

• US 20040199382A1
• Filed: 04/01/2003
• Published: 10/07/2004
• Est. Priority Date: 04/01/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Deng, Li, Acero, Alejandro, Bazzi, Issam
• Granted Patent: US 7,424,423 B2
• Time in Patent Office: Days
• US Class Current: 704/209
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 25/15 the extracted parameters being formant information

Abstract

A method of tracking formants defines a formant search space comprising sets of formants to be searched. Formants are identified for a first frame in the speech utterance by searching the entirety of the formant search space using the codebook, and for the remaining frames by searching the same space using both the codebook and the continuity constraint across adjacent frames. Under one embodiment, the formants are identified by mapping sets of formants into feature vectors and applying the feature vectors to a model. Formants are also identified by applying dynamic programming to search for the best sequence that optimally satisfies the continuity constraint required by the model.

Claims

1. A method of tracking formants in a speech signal, the method comprising:

defining a formant search space comprising sets of formants;

identifying formants in a first frame of the speech signal using the entirety of the formant search space; and

identifying formants in a second frame of the speech signal using the entirety of the formant search space.

2. The method of claim 1 wherein identifying formants in a first frame of the speech signal comprises utilizing a mapping between a set of formants and a feature vector.

3. The method of claim 2 wherein identifying formants in a first frame of the speech signal further comprises utilizing a residual model that models a difference between an input feature vector representing a frame of the speech signal and a feature vector mapped from a set of formants.

4. The method of claim 3 wherein the residual model is trained using an Expectation Maximization algorithm.

5. The method of claim 3 wherein utilizing the residual model comprises sequentially applying feature vectors mapped from each of the sets of formants in the formant search space to the residual model to identify a probability for each set of formants.

6. The method of claim 5 wherein identifying formants in the first frame comprises selecting the set of formants with the highest probability as the set of formants for the first frame.

7. The method of claim 5 wherein identifying formants in the first frame comprises performing a minimum mean squared error calculation using each of the sets of formants in the formant search space and the probabilities for the sets of formants.

8. The method of claim 5 wherein identifying a probability for a set of formants further comprises determining a probability for transitioning from a set of formants identified in a previous frame to the set of formants in the first frame.

9. The method of claim 8 further comprising identifying a total probability for each of a plurality of sequences of sets of formants for a sequence of frames in the speech signal.

10. The method of claim 9 wherein identifying a set of formants for the first frame comprises selecting the sequence of sets of formants with the highest total probability.

11. The method of claim 9 wherein identifying a set of formants for the first frame comprises making a minimum mean square error calculation using the sets of formants aligned with the first frame in each of the sequences of sets of formants.

12. A computer-readable medium having computer-executable instructions for performing steps comprising:

receiving an input feature vector representing a frame of a speech signal;

mapping a set of formants into a simulated feature vector; and

applying the input feature vector and the simulated feature vector to a model to determine a probability that the set of formants is present in the frame of the speech signal.

13. The computer-readable medium of claim 12 wherein mapping a set of formants comprises determining a transfer function from the set of formants, generating a spectral distribution based on the transfer function, and creating a feature vector based on the spectral distribution.

14. The computer-readable medium of claim 12 wherein mapping a set of formants comprises mapping a plurality of sets of formants into a plurality of simulated feature vectors before receiving an input feature vector.

15. The computer-readable medium of claim 14 further comprising training the model using the plurality of simulated feature vectors and a plurality of training feature vectors generated from a training speech signal.

16. The computer-readable medium of claim 15 wherein training the model comprises performing Expectation Maximization training.

17. The computer-readable medium of claim 12 further comprising using the probability that a set of formants is present in the frame of the speech signal to identify a set of formants for the frame.

18. The computer-readable medium of claim 17 wherein identifying a set of formants comprises identifying the set of formants with the highest probability as the set of formants for the frame.

19. The computer-readable medium of claim 17 wherein identifying a set of formants comprises performing a minimum mean square error calculation using the probability for the set of formants.

20. The computer-readable medium of claim 19 wherein the minimum mean square error calculation uses a separate probability for each of a plurality of sets of formants.

21. The computer-readable medium of claim 12 wherein determining a probability that the set of formants is present in the frame further comprises determining a probability of transitioning from a set of formants in a previous frame to the set of formants.