Processing audio signals during a communication event

• US 9,269,367 B2
• Filed: 08/18/2011
• Issued: 02/23/2016
• Est. Priority Date: 07/05/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Skype Limited (Microsoft Corporation)
• Inventors: Strmmer, Stefan, Srensen, Karsten Vandborg
• Primary Examiner(s): Saunders, Jr., Joseph
• Assistant Examiner(s): Mooney, James
• Application Number:
  US13/212,688
• Publication Number:
  US 20130013303A1
• Time in Patent Office: 1,650 Days
• Field of Search: 381/94.7
• US Class Current: 1/1
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: G10L 21/0208 Noise filtering

Abstract

A method of processing audio signals during a communication session between a user device and a remote node, includes receiving a plurality of audio signals at audio input means at the user device including at least one primary audio signal and unwanted signals and receiving direction of arrival information of the audio signals at a noise suppression means. Known direction of arrival information representative of at least some of said unwanted signals is provided to the noise suppression means and the audio signals are processed at the noise suppression means to treat as noise, portions of the signal identified as unwanted dependent on a comparison between the direction of arrival information of the audio signals and the known direction of arrival information.

Claims

1. A method of processing audio signals during a communication session between a user device and a remote node, the method comprising:

receiving a plurality of audio signals at the user device, the plurality of audio signals including at least one primary audio signal and unwanted signals;

receiving direction of arrival information of the audio signals at a noise reduction stage;

querying the user device for stored known direction of arrival information stored from one or more prior communication sessions;

providing to the noise reduction stage known direction of arrival information representative of at least some of said unwanted signals;

estimating at least one principal direction from which the at least one primary audio signal is received at a beamformer at the user device;

processing the plurality of audio signals to generate a single channel audio output signal comprising a sequence of frames, the noise reduction stage processing each of said frames in sequence;

comparing the direction of arrival information for a principal signal component of the current frame being processed with the known direction of arrival information;

determining whether the principal signal component of the current frame is an unwanted signal based on said comparison; and

responsive to determining that the principal signal component of the current frame is an unwanted signal based on direction of arrival information, applying maximum attenuation to the entire current frame.

2. The method according to claim 1, wherein the known direction of arrival information includes at least one direction from which far-end signals are received at the beamformer.

3. The method according to claim 1, wherein the known direction of arrival information includes at least one classified direction, the at least one classified direction being a direction from which at least one unwanted audio signal arrives at the beamformer and is identified based on the signal characteristics of the at least one unwanted audio signal.

4. The method according to claim 1, wherein the known direction of arrival information includes at least one principal direction from which the at least one primary audio signal is received at the beamformer.

5. The method according to claim 1, wherein the beamformer processes the plurality of audio signals to generate the single channel audio output signal, and the known direction of arrival information further includes the beam pattern of the beamformer.

6. The method according to claim 1, further comprising determining that the principal signal component of the current frame is an unwanted signal if:

the principal signal component is received at the beamformer from at least one direction from which far-end signals are received at the beamformer;

the principal signal component is received at the beamformer from at least one classified direction;

or

the principal signal component is not received at the beamformer from at least one principal direction.

7. The method according to claim 1, further comprising:

receiving the plurality of audio signals and information on the at least one principal direction at signal processing circuitry;

processing the plurality of audio signals at the signal processing circuitry using said information on the at least one principal direction to provide additional information to the noise reduction stage; and

applying a level of attenuation to the current frame being processed at the noise reduction stage in dependence on said additional information and said comparison.

8. The method according to claim 7, wherein the additional information includes an indication on the desirability of the principal signal component of the current frame.

9. The method according to claim 7, wherein the additional information includes a power level of the principal signal component of the current frame relative to an average power level of the at least one primary audio signal.

10. The method according to claim 7, wherein the additional information includes a signal classification of the principal signal component of the current frame.

11. The method according to claim 7, wherein the additional information includes at least one direction from which the principal signal component of the current frame is received at the beamformer.

12. The method according to claim 1, further comprising:

receiving the single channel audio output signal and information on the at least one principal direction at signal processing circuitry;

processing the single channel audio output signal at the signal processing circuitry using said information on the at least one principal direction to provide additional information to the noise reduction stage; and

applying a level of attenuation to the current frame being processed at the noise reduction stage in dependence on said additional information and said comparison.

13. A user device for processing audio signals during a communication session between the user device and a remote node, the user device comprising:

a beamformer configured to;

receive a plurality of audio signals including at least one primary audio signal and unwanted signals; and

generate, from the plurality of audio signals, a single channel audio output signal including a plurality of frames; and

a noise reduction stage configured to;

receive direction of arrival information for the plurality of audio signals and known direction of arrival information representative of at least some of said unwanted signals in the single channel audio output signal;

process the single channel audio output signal by treating as noise, portions of the signal identified as unwanted dependent on a comparison between the direction of arrival information of the plurality of audio signals in the single channel audio output signal and the known direction of arrival information; and

process the single channel audio output signal by applying varying levels of attenuation to respective different signals in a single frame of the plurality of frames.

14. The user device according to claim 13, wherein the beamformer is further configured to:

estimate at least one principal direction from which the at least one primary audio signal arrives; and

process the plurality of audio signals to generate a single channel audio output signal by forming a beam in the at least one principal direction and substantially suppressing audio signals from any direction other than the principal direction.

15. The user device according to claim 14, wherein the at least one principal direction is determined by:

determining a time delay that maximizes the cross-correlation between the audio signals being received at the beamformer; and

detecting speech characteristics in the audio signals received at the beamformer with said time delay of maximum cross-correlation.

16. The user device according to claim 13, wherein the noise reduction stage is configured to output audio data received at the user device from the remote node in the communication session.

17. The user device according to claim 13, wherein the unwanted signals are generated by a source at the user device, said source comprising at least one of:

audio output means of the user device;

a source of activity at the user device wherein said activity includes clicking activity comprising button clicking activity, keyboard clicking activity, and mouse clicking activity.

18. The user device according to claim 13, wherein the unwanted signals are generated by a source external to the user device.

19. The user device according to claim 13, wherein the at least one primary audio signal is a speech signal received at the beamformer.

20. A computer program product comprising computer readable instructions stored on a computer readable medium, the instructions executable for execution by one or more computer processors at a user device to perform operations comprising:

processing a plurality of audio signals including at least one primary audio signal and unwanted signals during a communication session between the user device and a remote node;

receiving direction of arrival information of the plurality of audio signals;

detecting one or more principal directions from the received direction of arrival information;

informing a user of the user device of the detected one or more principal directions;

responsive to said informing, prompting the user of the user device to verify that the one or more detected principal directions from the received direction of arrival information are correct principal directions;

providing known direction of arrival information representative of at least some of said unwanted signals; and

processing the audio signals to treat as noise, portions of the signal identified as unwanted dependent on a comparison between the direction of arrival information of the audio signals and the known direction of arrival information.

21. The method according to claim 20, wherein the known direction of arrival information includes at least one direction from which far-end signals are received at a beamformer of the user device.

22. A method of processing audio signals during a communication session between a user device and a remote node, the method comprising:

receiving a plurality of audio signals at the user device including at least one primary audio signal and unwanted signals;

receiving direction of arrival information of the plurality of audio signals;

providing known direction of arrival information representative of at least some of said unwanted signals;

detecting one or more principal directions from the received direction of arrival information;

informing a user of the user device of the detected one or more principal directions;

responsive to said informing, prompting the user of the user device to verify that the one or more detected principal directions from the received direction of arrival information are correct principal directions; and

processing the audio signals to treat as noise, portions of the signal identified as unwanted dependent on the known direction of arrival information and the verified one or more detected principal directions.

23. The method according to claim 22, wherein the known direction of arrival information includes at least one direction from which far-end signals are received at a beamformer of the user device.

24. The method according to claim 23, wherein the known direction of arrival information includes at least one principal direction from which the at least one primary audio signal is received at the beamformer.

25. The method according to claim 23, wherein the known direction of arrival information includes at least one classified direction being a direction from which at least one unwanted audio signal arrives at the beamformer and identified based on signal characteristics of at least one unwanted audio signal.