Microphone array speech enhancement

• US 10,186,277 B2
• Filed: 03/19/2015
• Issued: 01/22/2019
• Est. Priority Date: 03/19/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Salishev, Sergey I.
• Primary Examiner(s): Suthers, Douglas
• Application Number:
  US15/545,286
• Publication Number:
  US 20180012616A1
• Time in Patent Office: 1,405 Days
• Field of Search: 381 92
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: G10L 25/21 the extracted parameters being power information
• CPC Class Codes: H04R 1/04 Structural association of microphone with electric circuitry therefor in deaf-aid sets H04R25/00
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/04 for correcting frequency response

Abstract

Speech received from a microphone array is enhanced. In one example, a noise filtering system receives audio from the plurality of microphones, determines a beamformer output from the received audio, applies a first auto-regressive moving average smoothing filter to the beamformer output, determines noise estimates from the received audio, applies a second auto-regressive moving average smoothing filter to the noise estimates, and combines the first and second smoothing filter outputs to produce a power spectral density output of the received audio with reduced noise.

Claims

1. A method of filtering audio from a microphone array comprising:

receiving audio from a plurality of microphones;

determining a beamformer output from the received audio;

applying a first auto-regressive moving average smoothing filter to the beamformer output, including determining a harmonic noise model using the first smoothing filter, wherein the harmonic noise model is determined by determining an estimate for a log spectral power of harmonic voice components of a gain from the first smoothing filter;

determining noise estimates from the received audio;

applying a second auto-regressive moving average smoothing filter to the noise estimates; and

combining the first and second smoothing filter outputs, which includes the harmonic noise model, to produce a power spectral density output of the received audio with reduced noise.

2. The method of claim 1, wherein determining the estimate for a log spectral power comprises combining a log of the power spectral density of the beamformer output with a log of the gain from the first smoothing filter.

3. The method of claim 1, further comprising determining a comfort noise using the second smoothing filter and wherein combining comprises combining the comfort noise as the second smoothing filter output, wherein the comfort noise is determined by applying a function of the second smoothing filter output with a function of breath noise.

4. The method of claim 3, wherein the function of the second smoothing filter is a logarithmic function and wherein the function of breath noise is a logarithmic function.

5. The method of claim 4, wherein the function of the smoothing filter is factored by a weight, α

, and the function of the breath noise is factored by 1−

α

.

6. The method of claim 1, wherein combining comprises combining in accordance with a classifier that scales a difference between the first and second smoothing filter outputs.

7. The method of claim 6, wherein the first smoothing filter output is converted to a harmonic noise and the second smoothing filter output is converted to a comfort noise and wherein the classifier determines whether the harmonic noise or the comfort noise prevails in the received audio and combines the harmonic noise and the comfort noise with the received audio based on the determination.

8. The method of claim 7, wherein determining comprises applying a logistic regression to a signal to noise ratio.

9. The method of claim 1, wherein determining a beamformer output comprises converting the received audio to short term Fourier transform audio frames and taking a weighted sum of each frame over each microphone.

10. The method of claim 9, wherein weight of the weighted sum differs for each microphone.

11. A non-transitory machine-readable medium having instructions stored thereon that, when operated on by the machine, cause the machine to perform operations comprising:

receiving audio from a plurality of microphones;

determining a beamformer output from the received audio;

applying a first auto-regressive moving average smoothing filter to the beamformer output, including determining a harmonic noise model using the first smoothing filter, wherein the harmonic noise model is determined by determining an estimate for a log spectral power of harmonic voice components of a gain from the first smoothing filter;

determining noise estimates from the received audio;

applying a second auto-regressive moving average smoothing filter to the noise estimates; and

combining the first and second smoothing filter outputs, which includes the harmonic noise model, to produce a power spectral density output of the received audio with reduced noise.

12. The medium of claim 11, wherein combining comprises combining in accordance with a classifier that scales a difference between the first and second smoothing filter outputs.

13. The medium of claim 12, wherein the first smoothing filter output is converted to a harmonic noise and the second smoothing filter output is converted to a comfort noise and wherein the classifier determines whether the harmonic noise or the comfort noise prevails in the received audio and combines the harmonic noise and the comfort noise with the received audio based on the determination.

14. An apparatus comprising:

a microphone array comprising a plurality of microphones; and

a noise filtering system to receive audio from the plurality of microphones, determine a beamformer output from the received audio, apply a first auto-regressive moving average smoothing filter to the beamformer output, determine noise estimates from the received audio, apply a second auto-regressive moving average smoothing filter to the noise estimates, and combine the first and second smoothing filter outputs to produce a power spectral density output of the received audio with reduced noise,

wherein the noise filtering system is operable to apply a first auto-regressive moving average smoothing filter to the beamformer output to determine a harmonic noise model by determining an estimate for a log spectral power of harmonic voice components of a gain from the first smoothing filter.

15. The apparatus of claim 14, further comprising a speech recognition system to receive the power spectral density output and to recognize a statement in the received audio.

16. The apparatus of claim 14, further comprising a speech conversion system to combine the power spectral density output with phase data to generate an audio signal containing speech with reduced noise and a speech transmitter to transmit the audio signal to a remote device.

17. A wearable device comprising:

a frame configured to be worn by a user;

a microphone array connected to the frame and comprising a plurality of microphones; and

a noise filtering system connected to the frame to receive audio from the plurality of microphones, determine a beamformer output from the received audio, apply a first auto-regressive moving average smoothing filter to the beamformer output, determine noise estimates from the received audio, apply a second auto-regressive moving average smoothing filter to the noise estimates, and combine the first and second smoothing filter outputs to produce a power spectral density output of the received audio with reduced noise,

wherein the noise filtering system is operable to apply a first auto-regressive moving average smoothing filter to the beamformer output to determine a harmonic noise model by determining an estimate for a log spectral power of harmonic voice components of a gain from the first smoothing filter.

18. The device claim 17, wherein combining comprises combining in accordance with a classifier that scales a difference between the first and second smoothing filter outputs.

19. The device of claim 18, wherein the first smoothing filter output is converted to a harmonic noise and the second smoothing filter output is converted to a comfort noise and wherein the classifier determines whether the harmonic noise or the comfort noise prevails in the received audio and combines the harmonic noise and the comfort noise with the received audio based on the determination by applying a logistic regression to a signal to noise ratio.