Multi-sensory speech enhancement using synthesized sensor signal

• US 7,406,303 B2
• Filed: 09/16/2005
• Issued: 07/29/2008
• Est. Priority Date: 07/05/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Deng, Li, Zhang, Zhengyou, Liu, Zicheng, Subramanya, Amarnag
• Primary Examiner(s): NGUYEN, THUAN T
• Application Number:
  US11/228,710
• Publication Number:
  US 20070010291A1
• Time in Patent Office: 1,047 Days
• Field of Search: 455/414.1, 455/501, 455 6711- 6715, 455/76, 455/550.1, 455/570, 455/260, 455/114.2, 455/226.1, 455/296, 381/312, 381151-153
• US Class Current: 455/260
• CPC Class Codes: G10L 2021/02165 Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal
• CPC Class Codes: G10L 21/0208 Noise filtering

Abstract

A synthesized alternative sensor signal is produced from an alternative sensor signal. The synthesized alternative sensor signal is computed using vocal tract resonances estimated based on the alternative sensor signal, and using a waveform synthesis technique that converts the estimated vocal tract resonance sequence into a spectral magnitude sequence. The synthesized alternative sensor signal and the alternative sensor signal are used to estimate a clean speech value.

Claims

1. A method of determining an estimate for a noise-reduced value representing a portion of a noise-reduced speech signal, the method comprising:

generating an alternative sensor signal using an alternative sensor;

forming a synthesized alternative sensor signal based on the alternative sensor signal; and

using the alternative sensor signal, and the synthesized alternative sensor signal to form an estimate of the noise-reduced value.

2. The method of claim 1 further comprising generating an air-conduction microphone signal and using the air-conduction microphone signal with the alternative sensor signal and the synthesized alternative sensor signal to form the estimate of the noise-reduced value.

3. The method of claim 1 wherein forming the synthesized alternative sensor signal comprises identifying vocal tract resonances in the alternative sensor signal and using the identified vocal tract resonances to construct the synthesized alternative sensor signal.

4. The method of claim 3 wherein identifying vocal tract resonances comprises identifying a sequence of vocal tract resonances and then applying temporal smoothing to the sequence of vocal tract resonances to from a final sequence of vocal tract resonances.

5. The method of claim 3 wherein constructing the synthesized alternative sensor signal from the vocal tract resonances comprises using phase information from the alternative sensor signal to construct the synthesized alternative sensor signal.

6. The method of claim 5 wherein constructing the synthesized alternative sensor signal comprises:

forming cepstral values from the vocal tract resonances;

determining cepstral values from the alternative sensor signal;

subtracting the cepstral values of the alternative sensor signal from the cepstral values formed from the vocal tract resonances to form a cepstral difference;

converting the cepstral difference to the spectral domain to form a spectral difference; and

using the spectral difference and a complex spectral domain value of the alternative sensor signal to form a complex spectral domain value for the synthesized alternative sensor signal.

7. The method of claim 1 wherein forming an estimate of the noise-reduced value further comprises utilizing the variance of a noise term associated with the synthesized alternative sensor signal.

8. The method of claim 1 wherein forming the synthesized alternative sensor signal comprises:

identifying vocal tract resonances in the alternative sensor signal;

identifying vocal tract resonances in an air conduction microphone signal; and

using vocal tract resonances identified in the alternative sensor signal and the vocal tract resonance identified in the air conduction microphone signal to construct the synthesized alternative sensor signal.

9. The method of claim 1 wherein forming an estimate of the noise-reduced value further comprises utilizing a channel distortion for the synthesized alternative sensor signal.

10. The method of claim 9 wherein the channel distortion for the synthesized alternative sensor signal is based on a channel distortion for the alternative sensor signal.

11. A computer-readable medium having computer-executable instructions for performing steps comprising:

receiving a sensor signal representing speech;

identifying vocal tract resonances in the sensor signal;

converting the identified vocal tract resonances into a synthesized sensor signal; and

using the synthesized sensor signal to identify a clean speech value.

12. The computer-readable medium of claim 11 wherein identifying a clean speech value further comprises using the sensor signal to identify the clean speech value.

13. The computer-readable medium of claim 12 wherein identifying the clean speech value further comprises using an additional sensor signal to identify the clean speech value.

14. The computer-readable medium of claim 11 wherein converting the identified vocal tract resonances into a synthesized sensor signal comprises:

forming cepstral values from the vocal tract resonances;

forming cepstral values from the sensor signal;

subtracting the cepstral values formed from sensor signal from the cepstral values formed from the vocal tract resonances to form a difference; and

using the difference to form the synthesized sensor signal.

15. The computer-readable medium of claim 11 wherein identifying vocal tract resonances comprises identifying an initial sequence of vocal tract resonances and then applying temporal smoothing to the initial sequence to form a final sequence of vocal tract resonances.

16. The computer-readable medium of claim 11 wherein identifying a clean speech value further comprises using a variance of a noise term associated with the synthesized sensor signal.

17. The computer-readable medium of claim 11 further comprising:

receiving a second sensor signal representing speech;

identifying vocal tract resonances in the second sensor signal; and

wherein converting the identified vocal tract resonances into a synthesized sensor signal comprises combining the vocal tract resonances identified in the sensor signal and the vocal tract resonances identified in the second sensor signal to form combined vocal tract resonances and converting the combined vocal tract resonances into the synthesized sensor signal.

18. A method of identifying a clean speech value for a clean speech signal, the method comprising:

receiving an air-conduction microphone signal;

receiving an alternative sensor signal;

forming a synthesized alternative sensor signal; and

using the air-conduction microphone signal, the alternative sensor signal and the synthesized alternative sensor signal to estimate the clean speech value.

19. The method of claim 18 wherein the synthesized alternative sensor signal is formed in part by identifying vocal tract resonances in the alternative sensor signal.

20. The method of claim 18 wherein forming the synthesized alternative sensor signal comprises converting identified vocal tract resonances into cepstral domain values, converting the alternative sensor signal into cepstral domain values, and subtracting the cepstral domain values of the alternative sensor signal from the cepstral domain values of the vocal tract resonances.