Method and apparatus for multi-sensory speech enhancement

• US 20060072767A1
• Filed: 09/17/2004
• Published: 04/06/2006
• Est. Priority Date: 09/17/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Zhengyou, Droppo, James G., Acero, Alejandro, Liu, Zicheng, Huang, Xuedong David
• Granted Patent: US 7,574,008 B2
• Time in Patent Office: Days
• US Class Current: 381/71.600
• CPC Class Codes: G10L 2021/02161 Number of inputs available containing the signal or the noise to be suppressed
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: H04R 2460/13 Hearing devices using bone conduction transducers
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

A method and apparatus determine a channel response for an alternative sensor using an alternative sensor signal and an air conduction microphone signal. The channel response is then used to estimate a clean speech value using at least a portion of the alternative sensor signal.

Claims

1. A method of determining an estimate for a noise-reduced value representing a portion of a noise-reduced speech signal, the method comprising:

generating an alternative sensor signal using an alternative sensor other than an air conduction microphone;

generating an air conduction microphone signal;

using the alternative sensor signal and the air conduction microphone signal to estimate a value for a channel response of the alternative sensor signal; and

using the channel response to estimate the noise-reduced value.

2. The method of claim 1 wherein estimating a value for a channel response comprises finding an extreme of an objective function.

3. The method of claim 1 wherein estimating a channel response comprises modeling the alternative sensor signal as a clean speech signal convolved with the channel response, with the result summed with a noise term.

4. The method of claim 1 wherein the channel response comprises a channel response to a clean speech signal.

5. The method of claim 4 further comprising determining a channel response of the alternative sensor to a background speech signal.

6. The method of claim 5 wherein using the channel response to estimate the noise-reduced value comprises using the channel response to the clean speech signal and the channel response to the background speech signal to estimate the noise-reduced value.

7. The method of claim 1 further comprising using the estimate of the noise-reduced value to estimate a value for a background speech signal.

8. The method of claim 1 wherein estimating a value for a channel response comprises using a sequence of frames of the alternative sensor signal and the air conduction microphone signal to estimate a single channel response value for the frames in the sequence of frames.

9. The method of claim 8 wherein using the channel response to estimate a noise-reduced value comprises estimating a separate noise-reduced value for each frame in the sequence of frames.

10. The method of claim 1 wherein estimating a value for a channel response comprises estimating the value for a current frame by weighting values for the alternative sensor signal and the air conduction microphone signal in the current frame more heavily than values for the alternative sensor signal and the air conduction microphone signal in a previous frame.

11. A computer-readable medium having computer-executable instructions for performing steps comprising:

determining a channel response for an alternative sensor using an alternative sensor signal and an air conduction microphone signal; and

using the channel response to estimate a clean speech value using at least a portion of the alternative sensor signal.

12. The computer-readable medium of claim 11 wherein determining a channel response comprises determining a single channel response for a sequence of frames of the alternative sensor signal and the air conduction microphone signal.

13. The computer-readable medium of claim 11 wherein the channel response comprises a channel response to a clean speech signal.

14. The computer-readable medium of claim 13 further comprising determining a channel response to a background speech signal.

15. The computer-readable medium of claim 14 further comprising using the channel response to the background speech signal with the channel response to the clean speech signal to estimate the clean speech value.

16. The computer-readable medium of claim 11 further comprising using the clean speech value to estimate a background speech value.

17. A method of identifying a clean speech signal, the method comprising:

estimating noise parameters that describe noise in an alternative sensor signal;

using the noise parameters to estimate a channel response for an alternative sensor; and

using the channel response to estimate a value for the clean speech signal.

18. The method of claim 17 wherein estimating noise parameters comprises using the alternative sensor signal to identify periods when a user is not speaking.

19. The method of claim 18 further comprising performing speech detection on portions of an air conduction microphone signal associated with the periods when the user is not speaking to identify no-speech periods and background speech periods.

20. The method of claim 19 further comprising using portions of the alternative sensor signal associated with the no-speech periods to estimate the noise parameters.

21. The method of claim 20 further comprising using the no-speech periods to estimate noise parameters that describe noise in the air conduction microphone signal.

22. The method of claim 20 further comprising using the portions of the alternative sensor signal associated with the background speech periods to estimate a channel response to background speech.

23. The method of claim 22 further comprising using the channel response to background speech to estimate clean speech.

24. The method of claim 17 further comprising determining an estimate of a background speech value.

25. The method of claim 24 wherein determining an estimate of a background speech value comprises using the estimate of the clean speech value to estimate the background speech value.

26. The method of claim 17 further comprising using a prior model of the channel response to estimate the clean speech value.