Processing audio signals

• US 9,042,574 B2
• Filed: 11/30/2011
• Issued: 05/26/2015
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Skype S.Ã R.L. (Microsoft Corporation)
• Inventors: Sorensen, Karsten Vandborg
• Primary Examiner(s): Chin, Vivian
• Assistant Examiner(s): FAHNERT, FRIEDRICH
• Application Number:
  US13/308,210
• Publication Number:
  US 20130083936A1
• Time in Patent Office: 1,273 Days
• Field of Search: 381/91, 381/92, 381/122, 381/66, 381/94.1, 381/71.1, 379/406.08, 700/94, 455/570, 455/426.1, 369/24.01
• US Class Current: 381/92
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: H04M 9/082 using echo cancellers echo cancellers per se H04B3/23

Abstract

Audio signals are processed for use in a communication event. A data store may be queried to obtain an indication of an echo direction, which relates to a direction from which audio signals output from the audio output are likely to be received at a microphone array (plurality of microphones) of a device. Beamformer coefficients of an adaptive beamformer of the device are determined in dependence upon the received indication of the echo direction. Audio signals are received at the microphone array. The adaptive beamformer applies the determined beamformer coefficients to the received audio signals, thereby generating a beamformer output for use in the communication event. The beamformer coefficients are determined such that echo suppression is applied to audio signals received at the microphone array from the indicated echo direction.

Claims

1. A method of processing audio signals at a device for use in a communication event, the device comprising an audio output for outputting audio signals of the communication event, the method comprising:

the device querying a data store to obtain an indication of an echo direction from which audio signals output from the audio output are likely to be received at a microphone array of the device, said microphone array comprising a plurality of microphones;

determining beamformer coefficients of an adaptive beamformer of the device in dependence upon the obtained indication of the echo direction;

receiving audio signals at the plurality of microphones of the microphone array; and

applying, using the adaptive beamformer, the determined beamformer coefficients to the received audio signals, thereby generating a beamformer output for use in the communication event, wherein the beamformer coefficients are determined such that the adaptive beamformer applies echo suppression to audio signals received at the microphone array from the indicated echo direction.

2. The method of claim 1 further comprising:

outputting audio signals of the communication event from the audio output;

determining an actual echo direction from which the outputted audio signals are received at the microphone array; and

responsive to determining the actual echo direction is different from the indicated echo direction, adapting the beamformer coefficients such that the adaptive beamformer applies echo suppression to audio signals received at the microphone array from the actual echo direction.

3. The method of claim 1 wherein querying the data store further comprises querying the data store with an identifier of the device.

4. The method of claim 3 further comprising, responsive to querying the data store:

using the identifier of the device to retrieve, from the data store, echo direction data received from at least one other device having an identifier which corresponds to the identifier of the device; and

using the retrieved echo direction data to determine the echo direction for the device.

5. The method of claim 4 wherein the identifier of the at least one other device corresponds to the identifier of the device if the device is at least one of:

the same type,

the same model, or

the same category, as the at least one other device.

6. The method of claim 4 wherein there are a plurality of said other devices, and wherein the echo direction for the device is determined based on averaged statistics of the echo direction data received from the plurality of other devices.

7. The method of claim 6 further comprising determining that the echo direction data received from the plurality of other devices indicates that a particular echo direction is most common for the plurality of other devices and setting the echo direction for the device to be the particular echo direction.

8. The method of claim 4 further comprising:

receiving echo direction data from the at least one other device, said echo direction data received from each of the at least one other device comprising;

data relating to an echo direction from which echoes are received during a communication event at that other device, and

an identifier of that other device; and

storing the received echo direction data in the data store.

9. The method of claim 1 wherein the data store is at the device.

10. The method of claim 1 wherein the data store is at a server, and wherein querying a data store further comprises:

transmitting a query from the device to the server; and

receiving, at the device and from the server, said indication of the echo direction.

11. The method of claim 10 wherein the indication of the echo direction is broadcast from the server to the device.

12. The method of claim 1 further comprising:

determining, at the device, echo direction data relating to an echo direction from which echoes are received during a communication event at the device; and

storing, in the data store, the determined echo direction data,

wherein the determined echo direction data stored in the data store is used to determine the indication of the echo direction which is obtained, at least in part, by querying the data store.

13. A device for processing audio signals for use in a communication event, the device comprising:

an audio output for outputting audio signals of the communication event;

a microphone array comprising a plurality of microphones for receiving audio signals, the device further configured to query a data store to obtain an indication of an echo direction from which audio signals output from the audio output are likely to be received at the microphone array; and

an adaptive beamformer for applying beamformer coefficients to the audio signals received at the plurality of microphones, thereby generating a beamformer output for use in the communication event;

the device further configured to determine said beamformer coefficients in dependence upon the obtained indication of the echo direction, including beamformer coefficients configured to apply echo suppression to audio signals received at the microphone array from the indicated echo direction.

14. The device of claim 13 further configured to:

determine an actual echo direction from which outputted audio signals outputted from the audio output are received at the microphone array; and

adapt the beamformer coefficients, if the actual echo direction is different from the indicated echo direction, such that the adaptive beamformer applies echo suppression to audio signals received at the microphone array from the actual echo direction.

15. The device of claim 13 further configured to query the data store with an identifier of the device.

16. The device of claim 13 further comprising said data store.

17. The device of claim 13 wherein the data store is at a server, and wherein to query the data store, the device is further configured to:

transmit a query to the server; and

receive, from the server, said indication of the echo direction.

18. A computer program product for processing audio signals at a device for use in a communication event, the device comprising an audio output for outputting audio signals of the communication event, the computer program product being embodied on one or more computer-readable storage memory hardware devices and configured, responsive to execution by at least one processor of the device, to perform the operations of:

querying a data store to obtain an indication of an echo direction from which audio signals output from the audio output are likely to be received at a microphone array of the device, said microphone array comprising a plurality of microphones;

determining beamformer coefficients of an adaptive beamformer of the device in dependence upon the obtained indication of the echo direction; and

implementing the adaptive beamformer to apply the determined beamformer coefficients to audio signals received at the plurality of microphones of the microphone array, thereby generating a beamformer output for use in the communication event, wherein the beamformer coefficients are determined such that the adaptive beamformer applies echo suppression to audio signals received at the microphone array from the indicated echo direction.

19. A system for processing audio signals for use in a communication event, the system comprising:

a device comprising;

an audio output for outputting audio signals of the communication event;

a microphone array comprising a plurality of microphones for receiving audio signals, the device further configured to query a data store to obtain an indication of an echo direction from which audio signals output from the audio output are likely to be received at the microphone array; and

an adaptive beamformer for applying beamformer coefficients to the received audio signals, thereby generating a beamformer output for use in the communication event, the device further configured to determine said beamformer coefficients in dependence upon the obtained indication of the echo direction, including beamformer coefficients configured to apply echo suppression to audio signals received at the microphone array from the indicated echo direction; and

a server comprising the data store and configured to;

determining determine the echo direction; and

provide the indication of the echo direction to the device in response to a query from the device.

20. A method of processing audio signals in a communication system, the method comprising:

determining echo direction data at a plurality of devices of the communication system, said echo direction data determined at each one of the plurality of devices comprising;

data relating to an echo direction from which echoes are received during a communication event at that one of the plurality of devices, and

an identifier of that one of the plurality of devices;

transmitting the determined echo direction data from the plurality of devices to a data store of the communication system;

storing the echo direction data received from the plurality of devices at the data store;

querying the data store with an identifier of a particular device of the communication system;

using the identifier of the particular device to retrieve, from the data store, echo direction data received from at least one of said plurality of devices having an identifier which corresponds to the identifier of the particular device;

using the retrieved echo direction data to determine an echo direction for the particular device;

determining beamformer coefficients of an adaptive beamformer of the particular device in dependence upon the indication of the echo direction;

receiving audio signals at a plurality of microphones of a microphone array of the particular device; and

applying, using the adaptive beamformer, the determined beamformer coefficients to the received audio signals, thereby generating a beamformer output for use in a communication event, wherein the beamformer coefficients are determined such that echo suppression is applied to audio signals received at the microphone array from the indicated echo direction.

21. A communication system for processing audio signals, the communication system comprising:

a plurality of devices configured to;

determine echo direction data, said echo direction data determined at each one of the plurality of devices comprising;

data relating to an echo direction from which echoes are received during a communication event at that one of the plurality of devices, and

an identifier of that one of the plurality of devices; and

transmit the determined echo direction data from the plurality of devices to a server;

a server configured to;

receive the transmitted echo direction data from the plurality of devices;

store the received echo direction data in a data store associated with the server;

use an identifier of a respective device of the communication system to retrieve, from the data store, echo direction data received from at least one of said plurality of devices having an identifier which corresponds to the identifier of the respective device;

use the retrieved echo direction data to determine an echo direction for the respective device; and

provide an indication of the determined echo direction to the respective device; and

a particular device comprising;

an audio output for outputting audio signals of a communication event;

a microphone array comprising a plurality of microphones for receiving audio signals, the particular device configured to query the data store to obtain the indication of the determined echo direction, provided by the server, associated with the received audio signals; and

an adaptive beamformer for applying beamformer coefficients to the received audio signals, thereby generating a beamformer output for use in the communication event, the particular device further configured to determine said beamformer coefficients in dependence upon the obtained indication of the determined echo direction, including beamformer coefficients associated with applying echo suppression to audio signals received at the microphone array from the indicated echo direction.

22. A method of processing data at a server of a communication system, the method comprising:

receiving, at the server, echo direction data from a plurality of devices of the communication system, said echo direction data received from each respective device of the plurality of devices comprising;

data relating to an echo direction from which echoes are received during a communication event at the respective device of the plurality of devices, and

an identifier of the respective device of the plurality of devices;

storing the received echo direction data in a data store;

determining an identifier of a particular device of the communication system;

using the identifier of the particular device to retrieve, at the server and from the data store, echo direction data received from at least one device of said plurality of devices having an identifier which corresponds to the identifier of the particular device;

using the retrieved echo direction data to determine, at the server, an echo direction for the particular device; and

providing an indication of the determined echo direction to the particular device.

23. The method of claim 22 wherein determining an identifier of the particular device further comprises receiving, at the server from the device, a query for echo direction data, said query including the identifier of the particular device.

24. The method of claim 22 wherein providing an indication of the determined echo direction further comprises transmitting the indication of the determined echo direction from the server to the particular device.

25. The method of claim 24 wherein transmitting an indication of the determined echo direction from the server to the particular device further comprises broadcasting the indication of the determined echo direction.

26. The method of claim 22 wherein the identifier of the at least one of said plurality of devices corresponds to the identifier of the particular device if the particular device is at least one of:

the same type,

the same mod; and

the same category, as the at least one of said plurality of devices.

27. The method of claim 22 wherein there are a plurality of said devices having an identifier which corresponds to the identifier of the particular device, and wherein the echo direction for the particular device is determined based on averaged statistics of the echo direction data received from the plurality of said devices having an identifier which corresponds to the identifier of the particular device.

28. The method of claim 22 further comprising:

determining that the retrieved echo direction data indicates that a particular echo direction is the most common for the plurality of said devices having an identifier which corresponds to the identifier of the particular device; and

setting the echo direction for the particular device to be the particular echo direction.

29. A server for processing data, the server being part of a communication system, the server comprising:

at least one processor; and

one or more computer-readable storage memory devices embodying processor executable instructions which, responsive to execution by the at least one processor, are configured to enable the server to;

receive echo direction data from a plurality of devices of the communication system, said echo direction data for each the respective device of the plurality of devices comprising;

data relating to an echo direction from which echoes are received during a communication event at the respective device of the plurality of devices, and

an identifier of the respective device of the plurality of devices;

store the received echo direction data in an associated data store;

determine an identifier of a particular device of the communication system;

use the identifier of the particular device to retrieve, from the data store, echo direction data received from at least one device of said plurality of devices having an identifier which corresponds to the identifier of the particular device;

use the retrieved echo direction data to determine an echo direction for the particular device; and

provide an indication of the determined echo direction to the particular device.

30. A computer program product for processing data at a server of a communication system, the computer program product being embodied on one or more computer-readable memory hardware devices and configured, responsive to execution by at least one processor of the server, to perform the operations of:

receiving echo direction data from a plurality of devices of the communication system, said echo direction data received from each respective device of the plurality of devices comprising;

data relating to an echo direction from which echoes are received during a communication event at the respective device of the plurality of devices, and

an identifier of the respective device of the

plurality of devices;

storing the received echo direction data in a data store;

determining an identifier of a particular device of the communication system;

using the identifier of the particular device to retrieve, from the data store, echo direction data received from at least one of said plurality of devices having an identifier which corresponds to the identifier of the particular device;

using the retrieved echo direction data to determine an echo direction for the particular device; and

providing an indication of the determined echo direction to the particular device.

31. A method of processing data in a communication system, the method comprising:

determining echo direction data at a plurality of devices of the communication system, said echo direction data determined at each respective device of the plurality of devices comprising;

data relating to an echo direction from which echoes are received during a communication event at the respective device of the plurality of devices, and

an identifier of the respective device of the plurality of devices;

transmitting the determined echo direction data from the plurality of devices to a server of the communication system;

storing the echo direction data received at the server from the plurality of devices in a data store;

determining an identifier of a particular device of the communication system;

using the identifier of the particular device to retrieve, at the server from the data store, echo direction data received from at least one of said plurality of devices having an identifier which corresponds to the identifier of the particular device;

using the retrieved echo direction data to determine, at the server, an echo direction for the particular device; and

providing an indication of the determined echo direction for the particular device to the particular device.

32. The method of claim 31 wherein determining echo direction data at a plurality of devices further comprises, for each respective device of the plurality of devices, gathering information relating to the echo direction from which echoes are received throughout the communication event at the respective device of the plurality of devices.

33. The method of claim 32 wherein said gathering information includes updating a direction of arrival estimate during echo activity of the communication event at the respective device of the plurality of devices.

34. The method of claim 33 further comprising, for each one of the plurality of devices, using the direction of arrival estimate at the end of the communication event at that one of the plurality of devices to determine the echo direction data that is transmitted to the server.

35. A communication system for processing data, the communication system comprising:

a plurality of devices implemented, at least in part, in hardware and configured to;

determine echo direction data, said echo direction data determined at each respective device of the plurality of devices comprising;

data relating to an echo direction from which echoes are received during a communication event at the respective device of the plurality of devices, and

an identifier of the respective device of the plurality of devices; and

transmit the determined echo direction data from the plurality of devices to a server; and

a server implemented, at least in part in hardware, and configured to;

receive the transmitted echo direction data from the plurality of devices;

store the received echo direction data at a data store;

determine an identifier of a particular device of the communication system;

use the identifier of the particular device to retrieve, from the data store, echo direction data received from at least one of said plurality of devices having an identifier which corresponds to the identifier of the particular device;

use the retrieved echo direction data to determine an echo direction for the particular device; and

provide an indication of the determined echo direction to the particular device.

36. A device configured to process audio signals for use in a communication event, the device comprising:

an audio output block configured to output audio signals of the communication event;

a microphone array comprising a plurality of microphones configured to receive audio signals;

a querying block configured to query a data store to obtain an indication of an echo direction from which audio signals output from the audio output block are likely to be received at the microphone array;

an adaptive beamformer configured to apply beamformer coefficients to the audio signals received at the plurality of microphones, thereby generating a beamformer output for use in the communication event; and

a determining block configured to determine said beamformer coefficients in dependence upon the obtained indication of the echo direction, wherein the determining block is configured to determine the beamformer coefficients such that the adaptive beamformer applies echo suppression to audio signals received at the microphone array from the indicated echo direction.