Adaptive phase difference based noise reduction for automatic speech recognition (ASR)

• US 9,449,594 B2
• Filed: 09/17/2013
• Issued: 09/20/2016
• Est. Priority Date: 09/17/2013
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Stemmer, Georg, Kurylo, Lukasz
• Primary Examiner(s): Elahee, MD S
• Application Number:
  US14/124,790
• Publication Number:
  US 20150078571A1
• Time in Patent Office: 1,099 Days
• Field of Search: 381/60, 381/92, 382/118, 375/346, 375/330, 455/205
• US Class Current: 1/1
• CPC Class Codes: G10L 2021/02087 the noise being separate speech, e.g. cocktail party
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Embodiments of a system and method for adapting a phase difference-based noise reduction system are generally described herein. In some embodiments, spatial information associated with a first and second audio signal are determined, wherein the first and second audio signals including a target audio inside a beam and noise from outside the beam. A signal-to-noise ratio (SNR) associated with the audio signals is estimated. A mapping of phase differences to gain factors is adapted for determination of attenuation factors for attenuating frequency bins associated with noise outside the beam. Spectral subtraction is performed to remove estimated noise from the single-channel signal based on a weighting that affects frequencies associated with a target signal less. Frequency dependent attenuation factors are applied to attenuate frequency bins outside the beam to produce a target signal having noise reduced.

Claims

1. An adaptive phase difference-based noise reduction system, comprising:

a first channel for receiving a first audio signal at a first microphone;

a second channel for receiving a second audio signal at a second microphone, the first and second audio signals including a target audio inside a beam and noise from outside the beam;

a processor, coupled to the first and second channel, the processor arranged to;

determine spatial information associated with the first audio signal and with the second audio signal;

adjust phase differences by applying an adjustment for a phase difference greater than π

equal to −

2π

+ the phase difference greater than π and

applying an adjustment for a phase difference less than −

π

equal to 2π

+the phase difference less than −

π

;

to

estimate a signal-to-noise ratio (SNR) on a single-channel signal derived from the first and second audio signals using the determined spatial information;

adapt a mapping of the adjusted phase differences to gain factors for determination of frequency dependent attenuation factors for attenuating frequency bins associated with noise outside the beam, wherein to adapt the mapping of the adjusted phase differences to gain factors includes;

calculating phase difference errors for the frequency bins to be respectively equal to an absolute value of a phase difference divided by a phase-threshold vector component and subtracting a beam factor, components of a phrase-threshold vector corresponding to the frequency bins; and

clamping negative phase difference error values to zero to prevent frequency bins inside an inner portion of the beam from being attenuated; and

apply the frequency dependent attenuation factors to attenuate frequency bins outside the beam to produce a target signal having noise reduced.

2. The adaptive phase difference-based noise reduction system of claim 1, wherein the processor is further arranged to perform spectral subtraction on the single-channel signal derived from the first and second audio signals to remove estimated noise from the single-channel signal based on one selected from a group consisting of a weighting that affects frequencies associated with a target signal less and the estimated SNR.

3. The adaptive phase difference-based noise reduction system of claim 1, wherein the processor determines spatial information by adjusting the phase difference with a computed offset to allow beam steering.

4. The adaptive phase difference-based noise reduction system of claim 1, wherein the processor calculates the phase difference error by scaling phase differences to match time differences of arrival between the first audio signal of the first channel and the second audio signal of the second channel for each frequency bin.

5. The adaptive phase difference-based noise reduction system of claim 1, wherein the processor estimates the signal-to-noise ratio (SNR) on the single-channel signal derived from the first and second audio signals using the determined spatial information by estimating a current signal-to-noise-ratio (SNR) based on the calculated phase difference error for differentiating frequency bins from inside the beam from frequency bins outside the beam, and the single-channel signal.

6. The adaptive phase difference-based noise reduction system of claim 1, wherein the processor is further arranged to downmix the first and second audio signals of the first and second channels to derive the single-channel signal and to estimates the SNR by computing a ratio of noise energy outside of the beam and speech energy inside the beam.

7. The adaptive phase difference-based noise reduction system of claim 6, wherein the processor computes the ratio of noise energy outside of the beam and speech energy inside the beam by determining magnitudes of frequencies outside of the beam when a phase difference error is determined to be greater than zero, calculating a weighted time average of the magnitudes of frequencies outside of the beam, determining magnitudes of frequencies inside of the beam, estimating an instantaneous signal energy inside the beam and calculating a weighted time average of the estimation of the instantaneous signal energy inside the beam.

8. The adaptive phase difference-based noise reduction system of claim 1, wherein the processor adapts the mapping of phase differences to gain factors for determination of attenuation factors by computing attenuation factors based on a predetermined parameter, the estimated SNR and a calculated phase difference error for each of the frequency bins.

9. The adaptive phase difference-based noise reduction system of claim 1, wherein the processor is further arranged to calculate phase difference errors between more than two channels and combining attenuation factors obtained for each channel pair.

10. The adaptive phase difference-based noise reduction system of claim 1, wherein the processor derives the single-channel signal by processing more than two input audio signals of more than two channels to generate the single-channel signal.

11. A method for adapting a phase difference-based noise reduction system, comprising

determining spatial information associated with a first audio signal received in a first channel from a first microphone and with a second audio signal received in a second channel from a second microphone, the first and second audio signals including a target audio inside a beam and noise from outside the beam;

adjusting phase differences by applying an adjustment for a phase difference greater than π

equal to −

2π

+ the phase difference greater than π and

applying an adjustment for a phase difference less than π

equal to 2π

+ the phase difference less than −

π

;

estimating a signal-to-noise ratio (SNR) on a single-channel signal derived from the first and second audio signals using the determined spatial information;

adapting a mapping of the adjusted phase differences to gain factors for determination of frequency dependent attenuation factors for attenuating frequency bins associated with noise outside the beam, wherein adapting the mapping from the phase differences to gain factors includes;

calculating phase difference errors for the frequency bins to be respectively equal to an absolute value of a phase difference divided by a phase-threshold vector component and subtracting a beam factor, components of a phase-threshold vector corresponding to the frequency bins; and

clamping negative phase difference error values to zero to prevent frequency bins inside an inner portion of the beam from being attenuated; and

applying the frequency dependent attenuation factors to attenuate frequency bins outside the beam to produce a target signal having noise reduced.

12. The method of claim 11 further comprises performing spectral subtraction on a single-channel signal derived from the first and second audio signals to remove estimated noise from the single-channel signal based on a weighting that affects frequencies associated with a target signal less, wherein the performing spectral subtraction on a single-channel signal further comprises subtracting noise from the single channel signal based on the estimated SNR.

13. The method of claim 11, wherein the determining spatial information further comprises adjusting the phase difference with a computed offset to allow beam steering.

14. The method of claim 11, wherein the determining the phase difference errors further comprises scaling phase differences to match time differences of arrival between the first audio signal of the first channel and the second audio signal of the second channel for each frequency bin and wherein the estimating the signal-to-noise ratio (SNR) on the single-channel signal derived from the first and second audio signals using the determined spatial information further comprises estimating a current signal-to-noise-ratio (SNR) based on a phase difference error for differentiating frequency bins from inside the beam from frequency bins outside the beam, and the single-channel signal.

15. The method of claim 11 further comprising downmixing the first and second audio signals of the first and second channels to generate the single-channel signal.

16. The method of claim 11 further comprising using beamforming to generate the single-channel signal.

17. The method of claim 11, wherein estimating the SNR further comprises computing a ratio of noise energy outside of the beam and speech energy inside the beam.

18. The method of claim 17, wherein the computing the ratio of noise energy outside of the beam and speech energy inside the beam further comprises:

determining magnitudes of frequencies outside of the beam when a phase difference error is determined to be greater than zero;

calculating a weighted time average of the magnitudes of frequencies outside of the beam;

determining magnitudes of frequencies inside of the beam;

estimating an instantaneous signal energy inside the beam; and

calculating a weighted time average of the estimation of the instantaneous signal energy inside the beam.

19. The method of claim 11 further comprising calculating phase difference errors between more than two channels and combining attenuation factors obtained for each channel pair, wherein the adapting a mapping of phase differences to gain factors for determination of attenuation factors further comprises computing attenuation factors based on a predetermined parameter, the estimated SNR and the calculated phase difference error for each of the frequency bins.

20. The method of claim 11, further comprising generating the single-channel signal by processing more than two input audio signals of more than two channels to generate the single-channel signal.

21. At least one non-transitory machine readable medium comprising instructions that, when executed by the machine, cause the machine to perform operations for adapting a phase difference-based noise reduction system, the operations comprising:

determining spatial information associated with a first audio signal received in a first channel from a first microphone and with a second audio signal received in a second channel from a second microphone, the first and second audio signals including a target audio inside a beam and noise from outside the beam;

adjusting phase differences by applying an adjustment for a phase difference greater than π

equal to −

2π

+ the phase difference greater than π and

applying an adjustment for a phase difference less than π

equal to 2π

+ the phase difference less than −

π

;

estimating a signal-to-noise ratio (SNR) on a single-channel signal derived from the first and second audio signals using the determined spatial information;

adapting a mapping of the adjusted phase differences to gain factors for determination of frequency dependent attenuation factors for attenuating frequency bins associated with noise outside the beam, wherein adapting the mapping from the phase differences to gain factors includes;

calculating phase difference errors for the frequency bins to be respectively equal to an absolute value of a phase difference divided by a phase-threshold vector component and subtracting a beam factor, components of a phase-threshold vector corresponding to the frequency bins; and

clamping negative phase difference error values to zero to prevent frequency bins inside an inner portion of the beam from being attenuated; and

applying the frequency dependent attenuation factors to attenuate frequency bins outside the beam to produce a target signal having noise reduced.

22. The at least one non-transitory machine readable medium of claim 21 further comprises calculating phase difference errors between more than two channels and combining attenuation factors obtained for each channel pair and performing spectral subtraction on a single-channel signal derived from the first and second audio signals to remove estimated noise from the single-channel signal based on a weighting that affects frequencies associated with a target signal less, wherein the performing spectral subtraction on a single-channel signal further comprises subtracting noise from the single channel signal based on the estimated SNR and wherein the adapting a mapping of phase differences to gain factors for determination of attenuation factors further comprises computing attenuation factors based on a predetermined parameter, the estimated SNR and the calculated phase difference error for each of the frequency bins.

23. The at least one non-transitory machine readable medium of claim 21, wherein the determining spatial information further comprises adjusting the phase difference with a computed offset to allow beam steering.

24. The at least one non-transitory machine readable medium of claim 21, wherein the determining the phase difference errors further comprises scaling phase differences to match time differences of arrival between the first audio signal of the first channel and the second audio signal of the second channel for each frequency bin and wherein the estimating the signal-to-noise ratio (SNR) on the single-channel signal derived from the first and second audio signals using the determined spatial information further comprises estimating a current signal-to-noise-ratio (SNR) based on a phase difference error for differentiating frequency bins from inside the beam from frequency bins outside the beam, and the single-channel signal.