System and method for multi-channel multi-feature speech/noise classification for noise suppression

• US 8,239,196 B1
• Filed: 07/28/2011
• Issued: 08/07/2012
• Est. Priority Date: 07/28/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Paniconi, Marco
• Primary Examiner(s): Smits, Talivaldis Ivars
• Assistant Examiner(s): ROBERTS, SHAUN A
• Application Number:
  US13/193,297
• Time in Patent Office: 376 Days
• Field of Search: 704/226, 704/202, 704/232, 704/233, 381/71.1, 381/94.1
• US Class Current: 704/233
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: G10L 25/84 for discriminating voice from noise

Abstract

An architecture and framework for speech/noise classification of an audio signal using multiple features with multiple input channels (e.g., microphones) are provided. The architecture may be implemented with noise suppression in a multi-channel environment where noise suppression is based on an estimation of the noise spectrum. The noise spectrum is estimated using a model that classifies each time/frame and frequency component of a signal as speech or noise by applying a speech/noise probability function. The speech/noise probability function estimates a speech/noise probability for each frequency and time bin. A speech/noise classification estimate is obtained by fusing (e.g., combining) data across different input channels using a layered network model. Individual feature data acquired at each channel and/or from a beam-formed signal is mapped to a speech probability, which is combined through layers of the model into a final speech/noise classification for use in noise estimation and filtering processes for noise suppression.

Claims

1. A method for noise estimation and filtering based on classifying an audio signal received at a noise suppression module via a plurality of input channels as speech or noise, the method comprising:

measuring signal classification features for a frame of the audio signal input from each of the plurality of input channels;

generating a feature-based speech probability for each of the measured signal classification features of each of the plurality of input channels;

generating, for each of the plurality of input channels, a speech probability for the input channel by combining the feature-based speech probabilities of the input channel using an additive model for a middle layer of a probabilistic layered network model;

generating a combined speech probability over the plurality of input channels using the speech probabilities of the input channels;

classifying the audio signal as speech or noise based on the combined speech probability; and

updating an initial noise estimate for each of the plurality of input channels using the combined speech probability.

2. The method of claim 1, wherein the generating of the combined speech probability is performed using an additive model for a top layer of the probabilistic layered network model.

3. The method of claim 1, wherein the measured signal classification features from the plurality of input channels are input data to the probabilistic layered network model.

4. The method of claim 1, wherein the combined speech probability over the plurality of input channels is an output of the probabilistic layered network model.

5. The method of claim 1, wherein the probabilistic layered network model includes a set of intermediate states each denoting a class state of speech or noise for one or more layers of the probabilistic layered network model.

6. The method of claim 5, wherein the probabilistic layered network model further includes a set of state-conditioned transition probabilities.

7. The method of claim 6, wherein the speech probability for the intermediate state of the layer of the probabilistic layered network model is determined using one or both of an additive model and a multiplicative model.

8. The method of claim 7, further comprising applying the additive model or the multiplicative model to one of the state-conditioned transition probabilities to combine data from a lower layer.

9. The method of claim 5, wherein the feature-based speech probability for each of the measured signal classification features denotes a probability of a class state of speech or noise for a layer of the one or more layers of the probabilistic layered network model.

10. The method of claim 5, further comprising determining a speech probability for an intermediate state of a layer of the probabilistic layered network model using data from a lower layer of the probabilistic layered network model.

11. The method of claim 5, wherein at any layer and for any intermediate state, an additive model is used to generate a speech probability for the intermediate state, conditioned on the lower layer state.

12. The method of claim 5, wherein at any layer and for any intermediate state, a multiplicative model is used to generate a speech probability for the intermediate state, conditioned on the lower layer state.

13. The method of claim 1, wherein the speech probability for each of the input channels denotes a probability of a class state of speech or noise for a layer of the one or more layers of the probabilistic layered network model.

14. The method of claim 1, wherein the combined speech probability is generated as a weighted sum of the speech probabilities for the plurality of input channels.

15. The method of claim 14, wherein the weighted sum of the speech probabilities includes one or more weighting terms, the one or more weighting terms being based on one or more conditions.

16. The method of claim 1, wherein the probabilistic layered network model is a Bayesian network model.

17. The method of claim 1, wherein the probabilistic layered network model includes three layers.

18. The method of claim 1, wherein classifying the audio signal as speech or noise based on the combined speech probability includes applying a threshold to the combined speech probability.

19. The method of claim 1, further comprising determining an initial noise estimate for each of the plurality of input channels.

20. The method of claim 1, further comprising:

combining the frames of the audio signal input from the plurality of input channels;

measuring at least one signal classification feature of the combined frames of the audio signal;

calculating a feature-based speech probability for the combined frames using the measured at least one signal classification feature; and

combining the feature-based speech probability for the combined frames with the speech probabilities generated for each of the plurality of input channels.

21. The method of claim 20, wherein the combined frames of the audio signal is a time-aligned superposition of the frames of the audio signal received at each of the plurality of input channels.

22. The method of claim 20, wherein the combined frames of the audio signal is a signal generated using beam-forming on signals from the plurality of input channels.

23. The method of claim 20, wherein the combined frames of the audio signal is used as an additional input channel to the plurality of input channels.

24. The method of claim 1, wherein the initial noise estimate is updated with a recursive time average using a combined speech probability function.

25. The method of claim 24, wherein updating the initial noise estimate with the recursive time average includes using an input magnitude spectrum quantity to weight the speech probability, the input magnitude spectrum quantity being a magnitude spectrum of one of the plurality of input channels, a magnitude spectrum of the combined frames, or a combination of the magnitude spectrums of one of the plurality of input channels and the combined frames.

26. The method of claim 1, wherein the feature-based speech probability is a function of the measured signal classification feature, and wherein the speech probability for each of the plurality of input channels is a function of the feature-based speech probabilities for the input channel.

27. The method of claim 26, wherein the speech probability for each of the plurality of input channels is obtained by combining the feature-based speech probabilities of the input channel using one or both of an additive model and a multiplicative model for a state-conditioned transition probability.

28. The method of claim 1, wherein the feature-based speech probability is generated for each of the signal classification features by mapping each of the signal classification features to a probability value using a map function.

29. The method of claim 28, wherein the map function is a model with a set of width and threshold parameters.

30. The method of claim 28, wherein the feature-based speech probability is updated with a time-recursive average.

31. The method of claim 1, wherein the signal classification features include at least:

average likelihood ratio over time, spectral flatness measure, and spectral template difference measure.

32. The method of claim 1, wherein for a single input channel an additive model is used for a middle layer of the probabilistic layered network model to generate a speech probability for the single input channel.

33. The method of claim 1, wherein for a single input channel a multiplicative model is used for a middle layer of the probabilistic layered network model to generate a speech probability for the single input channel.

34. The method of claim 1, wherein a state-conditioned transition probability for an intermediate state at any intermediate layer of the probabilistic layered network model is fixed off-line or determined adaptively on-line.

35. The method of claim 1, wherein for a set of two input channels an additive model is used for a top layer of the probabilistic layered network model to generate a speech probability for the two input channels.

36. The method of claim 1, wherein a beam-formed signal is another input to the probabilistic layered network model, and wherein an additive model is used for a top layer to generate a speech probability for the two plurality of input channels and the beam-formed signal.

37. The method of claim 36, wherein for the beam-formed signal, a speech probability conditioned on signal classification features of the beam-formed signal is obtained by mapping the signal classification features to a probability value using a map function.

38. The method of claim 37, wherein a time-recursive average is used to update the speech probability of the beam-formed signal.

39. The method of claim 1, wherein a multiplicative model is used for the middle layer of the probabilistic layered network model to generate the speech probability for each of the plurality of input channels.