Adaptive spatial classifier for multi-microphone systems

• US 9,378,754 B1
• Filed: 07/21/2010
• Issued: 06/28/2016
• Est. Priority Date: 04/28/2010
• Status: Expired due to Fees

• Current Assignee: Knowles Electronics Llc (Knowles Corporation)
• Original Assignee: Knowles Electronics Llc (Knowles Corporation)
• Inventors: Every, Mark, Solbach, Ludger
• Primary Examiner(s): He, Jialong
• Application Number:
  US12/841,061
• Time in Patent Office: 2,169 Days
• Field of Search: 704/226, 704/233, 308/94
• US Class Current: 1/1
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: G10L 25/81 for discriminating voice from music
• CPC Class Codes: H04B 3/20 Reducing echo effects or singing; Opening or closing transmitting path; Conditioning for transmission in one direction or the other

Abstract

A robust noise suppression system may concurrently reduce noise and echo components in an acoustic signal while limiting the level of speech distortion. The system may receive acoustic signals from two or more microphones in a close-talk, hand-held or other configuration. The received acoustic signals are transformed to cochlea domain sub-band signals and echo and noise components may be subtracted from the sub-band signals. Features in the acoustic sub-band signals are identified and used to generate a multiplicative mask. The multiplicative mask is applied to the noise subtracted sub-band signals and the sub-band signals are reconstructed in the time domain.

Claims

1. A method for performing noise reduction in an audio signal, the method comprising:

executing a frequency analysis module by a processor to transform a primary acoustic signal and one or more secondary acoustic signals to frequency domain sub-band signals, each of the primary acoustic signal and the one or more secondary acoustic signals representing at least one captured sound from a primary microphone and at least one secondary microphone, respectively;

forming a noise cancelled sub-band signal from at least one of the frequency domain sub-band signals by performing at least one of subtractive noise cancellation and echo cancellation;

deriving a null-processing inter-aural level difference (NP-ILD) feature from the noise cancelled sub-band signal and a raw microphone signal obtained from the at least one secondary microphone;

deriving an inter-aural level difference (ILD) feature based on energy outputs of the primary microphone and the at least one secondary microphone;

classifying each of a plurality of the frequency domain sub-band signals as speech or noise, based on a set of derived features including the NP-ILD feature and the ILD feature; and

performing noise reduction on the plurality of the frequency domain sub-band signals based on the classification.

2. The method of claim 1, wherein performing noise reduction includes performing the subtractive noise cancellation on the frequency domain sub-band signals associated with the primary acoustic signal.

3. The method of claim 1, wherein performing noise reduction includes applying a multiplicative mask to the frequency domain sub-band signals associated with the primary acoustic signal.

4. The method of claim 1, wherein the set of derived features includes a further spatial feature computed based on a difference between two of the frequency domain sub-band signals.

5. The method of claim 4, wherein the further spatial feature includes an inter-aural time difference derived from the primary acoustic signal and one of the one or more secondary acoustic signals.

6. The method of claim 4, wherein the further spatial feature includes a phase difference derived from the primary acoustic signal and one of the one or more secondary acoustic signals.

7. The method of claim 1, wherein at least one of the set of derived features is based on a ratio of a first energy of the noise cancelled sub-band signal and a second energy of one of the frequency domain sub-band signals associated with the one or more secondary acoustic signals.

8. The method of claim 1, wherein noise includes echo.

9. The method of claim 1, wherein the classifying is performed for each time frame and sub-band.

10. The method of claim 1, wherein the noise reduction is performed to optimize performance of an automatic speech recognition system.

11. A system for performing noise reduction in an audio signal, the system comprising:

a memory;

a frequency analysis module stored in the memory and executed by a processor to transform a primary acoustic signal and one or more secondary acoustic signals to frequency domain sub-band signals, each of the primary acoustic signal and the one or more secondary acoustic signals representing at least one captured sound from a primary microphone and at least one secondary microphone, respectively;

a noise canceller to form a noise cancelled sub-band signal from at least one of the frequency domain sub-band signals by performing at least one of subtractive noise cancellation and echo cancellation;

a feature extractor module executable to derive;

a null-processing inter-aural level difference (NP-ILD) feature from the noise cancelled sub-band signal and a raw microphone signal obtained from the at least one secondary microphone; and

an inter-aural level difference (ILD) feature based on energy outputs of the primary microphone and the at least one secondary microphone;

a source inference module executable to classify each frequency domain sub-band signal as speech or noise based on a set of derived features including the NP-ILD feature and the ILD feature; and

a noise reduction module executable to perform noise reduction on the frequency domain sub-band signals based on the classification.

12. The system of claim 11, the noise reduction module including the noise canceller configured to perform talk subtractive noise cancellation on the frequency domain sub-band signals associated with the primary acoustic signal.

13. The system of claim 11, the noise reduction module including a modifier configured to apply a multiplicative mask to the frequency domain sub-band signals associated with the primary acoustic signal.

14. The system of claim 11, wherein the feature extractor module is further executable to derive a further spatial feature, the set of derived features including the further spatial feature and the further spatial feature being computed based on a difference between two of the frequency domain sub-band signals.

15. The system of claim 14, wherein the further spatial feature includes one or more of an, inter-aural phase difference and an inter-aural time difference derived from the primary acoustic signal and one of the one or more secondary acoustic signals.

16. The system of claim 11, wherein at least one of the set of derived features is based on a ratio of a first energy of the noise cancelled sub-band signal and a second energy of one of the frequency domain sub-band signals associated with the one or more secondary acoustic signals.

17. The system of claim 11, the source inference module being executable to perform the classification for each time frame and sub-band.

18. A non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for reducing noise in an audio signal, the method comprising:

transforming a primary acoustic signal and one or more secondary acoustic signals to frequency domain sub-band signals, each of the primary acoustic signal and the one or more secondary acoustic signals representing at least one captured sound from a primary microphone and at least one secondary microphone, respectively;

forming a noise cancelled sub-band signal from at least one of the frequency domain sub-band signals by performing at least one of subtractive noise cancellation and echo cancellation;

deriving a null-processing inter-aural level difference (NP-ILD) feature from the noise cancelled sub-band signal and a raw microphone signal obtained from the at least one secondary microphone;

deriving an inter-aural level difference (ILD) feature based on energy outputs of the primary microphone and the at least one secondary microphone;

classifying each of a plurality of the frequency domain sub-band signals as speech or noise, based on a set of derived features including the NP-ILD feature and the ILD feature; and

performing noise reduction on the plurality of the frequency domain sub-band signals based on the classification.