Speech modeling and enhancement based on magnitude-normalized spectra

• US 7,930,178 B2
• Filed: 12/23/2005
• Issued: 04/19/2011
• Est. Priority Date: 12/23/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Subramanya, Amarnag, Zhang, Zhengyou, Acero, Alejandro, Liu, Zicheng
• Primary Examiner(s): Wozniak; James S
• Assistant Examiner(s): He; Jialong
• Application Number:
  US11/317,269
• Publication Number:
  US 20070150263A1
• Time in Patent Office: 1,943 Days
• Field of Search: 704/233, 704/266, 704/226, 704/234, 704/224
• US Class Current: 704/234
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 21/0208 Noise filtering

Abstract

A frame of a speech signal is converted into the spectral domain to identify a plurality of frequency components and an energy value for the frame is determined. The plurality of frequency components is divided by the energy value for the frame to form energy-normalized frequency components. A model is then constructed from the energy-normalized frequency components and can be used for speech recognition and speech enhancement.

Claims

1. A method comprising:

converting frames of a speech signal into the spectral domain to identify feature vectors each comprising a plurality of frequency components;

removing a D.C. frequency component from the plurality of frequency components to form a plurality of filtered frequency components;

determining an energy value for the filtered frequency components for each frame;

for each frame, dividing the plurality of filtered frequency components of the feature vector for the frame by the energy value for the filtered frequency components for the frame to form an energy-normalized feature vector comprising energy-normalized frequency components for the frame; and

a processor constructing a model from the energy-normalized feature vectors, wherein

constructing the model comprises;

clustering frames of energy-normalized frequency components into mixture components by determining a distance d({tilde over (X)}_i, {tilde over (X)}_j) between an energy-normalized feature vector {tilde over (X)}_i for a frame i and an energy-normalized feature vector {tilde over (X)}_j for a frame j as;

2. The method of claim 1 further comprising using the model to determine a likelihood for a frame of an input speech signal by:

converting the frame of the input speech signal to the spectral domain to produce a plurality of frequency components;

determining an energy value for the frame of the input signal;

dividing the plurality of frequency components of the frame of the input speech signal by the energy value for the frame of the input signal to form input energy-normalized frequency components; and

applying the input energy-normalized frequency components to the model to determine the likelihood.

3. The method of claim 1 further comprising using the model to estimate a clean speech value from a noisy speech signal.

4. A computer storage medium having computer-executable instructions for performing steps comprising:

receiving values representing a noisy air conduction microphone signal;

receiving values representing a noisy alternative sensor signal; and

using a model of energy-normalized clean-speech spectral values to estimate a vector of energy-normalized noise-reduced speech spectral values from the noisy air conduction microphone signal and the noisy alternative sensor signal by determining an expected value for each mixture component and state of the model as;

5. The computer storage medium of claim 4 wherein estimating a noise-reduced value further comprises estimating the gain value that describes the ratio of clean speech spectral values to energy-normalized clean speech spectral values.

6. The computer storage medium of claim 5 wherein estimating a noise-reduced value comprises estimating energy-normalized noise-reduced speech spectral values and multiplying the energy-normalized noise-reduced speech spectral values by the gain value to produce noise-reduced values.

7. The computer storage medium of claim 6 further comprising estimating a separate gain value for each of a plurality of frames of the noisy speech signal.

8. The computer storage medium of claim 4 wherein estimating a noise-reduced value further comprises utilizing a model of a speech state that provides the probability that a frame of the noisy speech signal contains speech.

9. A method comprising:

receiving an air conduction microphone signal;

receiving an alternative sensor signal;

using the air conduction microphone signal, the alternative sensor signal, and a model of energy-normalized clean speech spectral values to estimate a noise-reduced speech value, wherein estimating a noise-reduced speech value comprises estimating energy-normalized noise-reduced speech spectral values and converting the energy-normalized noise-reduced speech spectral values into noise-reduced speech values by multiplying the energy-normalized noise-reduced speech spectral values by a gain value determined as;

10. The method of claim 9 further comprising iterating between estimating an energy-normalized noise-reduced speech spectral value for frame t and estimating the gain value for frame t multiple times.

11. The method of claim 9 wherein estimating the gain value comprises estimating a separate gain value for each frame of the air conduction microphone signal.

12. The method of claim 9 wherein the model of energy-normalized clean speech spectral values comprises a mixture model.