Online dereverberation algorithm based on weighted prediction error for noisy time-varying environments

• US 10,446,171 B2
• Filed: 12/22/2017
• Issued: 10/15/2019
• Est. Priority Date: 12/23/2016
• Status: Active Grant

• Current Assignee: Synaptics Incorporated
• Original Assignee: Synaptics Incorporated
• Inventors: Kaskari, Saeed Mosayyebpour, Nesta, Francesco, Thormundsson, Trausti
• Primary Examiner(s): Goins, Davetta W
• Assistant Examiner(s): Ganmavo, Kuassi A
• Application Number:
  US15/853,693
• Publication Number:
  US 20180182410A1
• Time in Patent Office: 662 Days
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band

Abstract

Systems and methods for processing multichannel audio signals include receiving a multichannel time-domain audio input, transforming the input signal to plurality of multi-channel frequency domain, k-spaced under-sampled subband signals, buffering and delaying each channel, saving a subset of spectral frames for prediction filter estimation at each of the spectral frames, estimating a variance of the frequency domain signal at each of the spectral frames, adaptively estimating the prediction filter in an online manner using a recursive least squares (RLS) algorithm, linearly filtering each channel using the estimated prediction filter, nonlinearly filtering the linearly filtered output signal to reduce residual reverberation and the estimated variances, producing a nonlinearly filtered output signal, and synthesizing the nonlinearly filtered output signal to reconstruct a dereverberated time-domain multi-channel audio signal.

Claims

1. A method for processing multichannel audio signals comprising:

receiving an input signal comprising a time-domain, multi-channel audio signal;

transforming the input signal to a frequency domain input signal comprising a plurality of multi-channel frequency domain, k-spaced under-sampled subband signals;

buffering and delaying each channel of the frequency domain input signal;

saving a subset of spectral frames for prediction filter estimation at each of the spectral frames;

estimating a variance of the frequency domain input signal at each of the spectral frames;

adaptively estimating a prediction filter in an online manner by using a recursive least squares (RLS) algorithm and a cost function based at least in part on the estimated variance;

linearly filtering each channel of the frequency domain input signal to reduce reverberation using the estimated prediction filter to produce a linearly filtered output signal;

nonlinearly filtering the linearly filtered output signal to reduce residual reverberation using the estimated variances, producing a nonlinearly filtered output signal; and

synthesizing the nonlinearly filtered output signal to reconstruct a dereverberated time-domain, multi-channel audio signal, wherein a number of output channels is equal to a number of input channels.

2. The method of claim 1, wherein estimating the variance of the frequency domain input signal further comprises estimating a clean speech variance.

3. The method of claim 2, wherein estimating the variance of the frequency domain input signal further comprises estimating a noise variance.

4. The method of claim 3, wherein estimating the variance of the frequency domain input signal further comprises estimating a residual speech variance.

5. The method of claim 1, wherein adaptively estimating further comprises using an adaptive RLS algorithm to estimate the prediction filter at each frame independently for each frequency bin of the frequency domain input signal by imposing sparsity to a correlation matrix.

6. The method of claim 5 further comprising detecting changes in speaker movement and resetting the prediction filter and the correlation matrix in response to a sudden change in speaker movement.

7. The method of claim 1, wherein the input signal comprises at least one target signal;

and wherein the nonlinear filtering computes an enhanced speech signal for each target signal.

8. The method of claim 7, wherein the nonlinear filtering reduces residual reverberation and background noise.

9. The method of claim 1, wherein estimating the variance of the frequency domain input signal further comprises:

estimating a new clean speech variance based on a previous estimated prediction filter;

estimating a new residual reverberation variance using a fixed exponentially decaying weighting function with a tuning parameter to customize an audio solution; and

estimating a noise variance using a single-microphone noise variance estimation method to estimate the noise variance for each channel and then computing an average.

10. The method of claim 1, wherein buffering and delaying each channel of the frequency domain input signal further comprises saving a plurality of spectral frames for each subband of each channel, wherein a number of spectral frames saved differs for at least two subbands.

11. The method of claim 10, wherein at least one subband has a buffer length that is longer than a number of frames saved for a higher frequency subband.

12. An audio processing system comprising:

an audio input operable to receive a time-domain, multi-channel audio signal;

a subband decomposition module operable to transform the input signal to a frequency domain input signal comprising a plurality of multi-channel frequency domain, k-spaced under-sampled subband signals;

a buffer operable to buffer and delay each channel of the frequency domain input signal, saving a subset of spectral frames for prediction filter estimation at each of the spectral frames;

a variance estimator operable to estimate a variance of the frequency domain input signal at each of the spectral frames;

a prediction filter estimator operable to adaptively estimate the prediction filter in an online manner by using a recursive least squares (RLS) algorithm having a cost function based at least in part on the estimated variance;

a linear filter operable to linearly filter each channel of the frequency domain input signal to reduce reverberation using the estimated prediction filter to produce a linearly filtered output signal;

a non-linear filter operable to nonlinearly filter the linearly filtered output signal to reduce residual reverberation using the estimated variances, producing a nonlinearly filtered output signal; and

a synthesizer operable to synthesize the nonlinearly filtered output signal to reconstruct a dereverberated time-domain, multi-channel audio signal, wherein a number of output channels is equal to a number of input channels.

13. The audio processing system of claim 12, wherein the variance estimator is further operable to estimate a clean speech variance.

14. The audio processing system of claim 13, wherein the variance estimator is further operable to estimate a noise variance.

15. The audio processing system of claim 14, wherein the variance estimator is further operable to estimate a residual speech variance.

16. The audio processing system of claim 12, wherein the prediction filter estimator is further operable to use an adaptive RLS algorithm to estimate the prediction filter at each frame independently for each frequency bin of the frequency domain input signal by imposing sparsity to a correlation matrix.

17. The audio processing system of claim 16 wherein the variance estimator is further operable to detect changes due to speaker movement and to reset the prediction filter and the correlation matrix.

18. The audio processing system of claim 12, wherein the time-domain, multi-channel audio signal comprises at least one target signal;

and

wherein the nonlinear filter is further operable to compute an enhanced speech signal for each target signal.

19. The audio processing system of claim 18, wherein the nonlinear filter is operable to reduce residual reverberation and background noise.

20. The audio processing system of claim 12, wherein the variance estimator is further operable to:

estimate a new clean speech variance based on a previous estimated prediction filter;

estimate a new residual reverberation variance using a fixed exponentially decaying weighting function with a tuning parameter to customize an audio solution; and

estimate a noise variance using a single-microphone noise variance estimation method to estimate the noise variance for each channel and then computing an average.