Multi-sensory speech enhancement using a speech-state model

• US 20060293887A1
• Filed: 06/28/2005
• Published: 12/28/2006
• Est. Priority Date: 06/28/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Zhengyou, Droppo, James G., Acero, Alejandro, Liu, Zicheng, Subramanya, Amarnag
• Granted Patent: US 7,680,656 B2
• Time in Patent Office: Days
• US Class Current: 704/233
• CPC Class Codes: G10L 2021/02165 Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal
• CPC Class Codes: G10L 21/0208 Noise filtering

Abstract

A method and apparatus determine a likelihood of a speech state based on an alternative sensor signal and an air conduction microphone signal. The likelihood of the speech state is used, together with the alternative sensor signal and the air conduction microphone signal, to estimate a clean speech value for a clean speech signal.

Claims

1. A method of determining an estimate for a noise-reduced value representing a portion of a noise-reduced speech signal, the method comprising:

generating an alternative sensor signal using an alternative sensor;

generating an air conduction microphone signal;

using the alternative sensor signal and the air conduction microphone signal to estimate a likelihood of a speech state; and

using the likelihood of the speech state to estimate the noise-reduced value.

2. The method of claim 1 wherein using the likelihood of the speech state to estimate the noise-reduced value comprises using the likelihood of the speech state and an expectation value for the noise-reduced value to estimate the noise-reduced value.

3. The method of claim 2 wherein the expectation value for the noise-reduced value is based on the alternative sensor signal and the air conduction microphone signal.

4. The method of claim 1 wherein estimating a likelihood of a speech state comprises estimating a separate likelihood for each of a plurality of speech states and wherein using the likelihood of the speech state to estimate the noise-reduced value comprises using the likelihoods for each speech state in the plurality of speech states.

5. The method of claim 4 wherein using the likelihoods for each speech state in the plurality of speech states comprises using the likelihoods for each speech state to weight expectation values for the noise-reduced value.

6. The method of claim 1 wherein estimating a likelihood of a speech state comprises estimating a separate likelihood component for each of a set of frequency components and combining the separate likelihood components to form the estimate of the likelihood of the speech state.

7. The method of claim 1 further comprising using the estimate of the likelihood of a speech state to determine if a frame of the air conduction microphone signal contains speech.

8. The method of claim 7 further comprising using a frame of the air conduction microphone signal that is determined to not contain speech to determine a variance for a noise source and using the variance for the noise source to estimate the noise-reduced value.

9. The method of claim 1 further comprising modeling a probability of a noise-reduced value given a state as a distribution having a variance.

10. The method of claim 9 further comprising estimating the variance of the distribution as a linear combination of an estimate of a noise-reduced value for a preceding frame and a filtered version of the air conduction microphone signal for a current frame.

11. The method of claim 10 wherein the filtered version of the air conduction microphone signal is formed using a filter that is frequency dependent.

12. The method of claim 10 wherein the filtered version of the air conduction microphone signal is formed using a filter that is dependent on a signal-to-noise ratio.

13. The method of claim 1 further comprising performing an iteration by using the estimate of the noise-reduced value to form a new estimate of the noise-reduced value.

14. A computer-readable medium having computer-executable instructions for performing steps comprising:

determining a likelihood of a speech state based on an alternative sensor signal and an air conduction microphone signal; and

using the likelihood of the speech state to estimate a clean speech value.

15. The computer-readable medium of claim 14 wherein using the likelihood of the speech state to estimate a clean speech value comprises weighting an expectation value.

16. The computer-readable medium of claim 14 wherein using the likelihood of the speech state to estimate a clean speech value comprises:

using the likelihood of the speech state to identify a frame of a signal as a non-speech frame;

using the non-speech frame to estimate a variance for a noise; and

using the variance for the noise to estimate the clean speech value.

17. The computer-readable medium of claim 14 wherein estimating a likelihood for a speech state comprises estimating a separate likelihood for each of a plurality of frequency components, and combining the separate likelihoods to form the likelihood for the speech state.

18. A method of identifying a clean speech value for a clean speech signal, the method comprising:

forming a model wherein the clean speech signal is dependent upon a speech state, an alternative sensor signal is dependent upon the clean speech signal, and an air conduction microphone signal is dependent upon the clean speech signal; and

determining an estimate of the clean speech value based on the model, a value for the alternative sensor signal, and a value for the air conduction microphone signal.

19. The method of claim 18 further comprising determining a likelihood for a state and wherein determining an estimate of the clean speech value further comprises using the likelihood for the state.

20. The method of claim 18 wherein forming the model comprises forming a model wherein the alternative sensor signal and the air conduction microphone signal are dependent upon a noise source.