Null-forming techniques to improve acoustic echo cancellation

• US 8,885,815 B1
• Filed: 06/25/2012
• Issued: 11/11/2014
• Est. Priority Date: 06/25/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Rawles, LLC (Amazon.com, Inc.)
• Inventors: Velusamy, Kavitha, Chhetri, Amit S., Gopalan, Ramya, Chu, Wai C., Li, Wei
• Primary Examiner(s): Jamal, Alexander
• Application Number:
  US13/532,570
• Time in Patent Office: 869 Days
• Field of Search: 379/406.03
• US Class Current: 379/406.03
• CPC Class Codes: G01S 3/80 using ultrasonic, sonic or infrasonic waves
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: H04M 9/08 Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic
• CPC Class Codes: H04M 9/082 using echo cancellers echo cancellers per se H04B3/23
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/02 for preventing acoustic reaction , i.e. acoustic oscillatory feedback specially adapted for hearing aids H04R25/453

Abstract

A plurality of microphones of a communication device is grouped into multiple microphone groups, such that each microphone group includes two or more microphones. For each microphone group, output of the corresponding microphones is processed to form an acoustic null in a corresponding spatial direction, such that sound from the corresponding spatial direction is attenuated in the processed output. One of the microphone groups is selected based on various factors leading to maximal echo attenuation and rejection of reverberant components of the room. The selected microphone group is then used to detect sound from a near end talker of the communication device.

Claims

1. A method comprising:

grouping a plurality of microphones of a communication device in a plurality of microphone groups, each microphone group including two or more microphones of the plurality of microphones and the communication device including a speaker that outputs sound from a first sound source;

for each microphone group, processing output of the microphones of the microphone group to (i) attenuate sound captured by the microphones of the microphone group from substantially a direction of the speaker and (ii) amplify sound captured by the microphones of the microphone group from a direction other than the direction of the speaker;

for each microphone group, determining a respective sound signal from a second sound source based at least in part on (i) the processed output of the microphones of the microphone group and (ii) the sound from the first sound source output by the speaker;

for each microphone group, determining a correlation value that is indicative of a correlation between (i) the processed output of the microphones of the microphone group and (ii) the determined sound signal for the microphone group, the correlation value being indicative of double talk in the processed output of the microphones of the microphone group, wherein a plurality of correlation values is generated corresponding to the plurality of microphone groups;

selecting a correlation value that has a highest value among the plurality of correlation values, wherein the selected correlation value corresponds to a first microphone group of the plurality of microphone groups; and

detecting sound from the second sound source based at least in part on the processed output of the microphones of the first microphone group.

2. The method of claim 1, further comprising:

selecting one or more other correlation values that are higher than a threshold value, wherein the one or more other correlation values correspond to one or more other microphone groups of the plurality of microphone groups; and

detecting sound from the second sound source based at least in part on the processed outputs of the microphones of the one or more other microphone groups.

3. The method of claim 1, wherein processing the output of the microphones further comprises:

processing the output of the microphones of the first microphone group to amplify sound captured by the microphones of the first microphone group from a first direction; and

processing the output of the microphones of a second microphone group of the plurality of microphone groups to amplify sound captured by the microphones of the second microphone group from a second direction, where the first direction is different from the second direction.

4. The method of claim 3, wherein processing the output of the microphones further comprises:

processing the output of the microphones of both the first and second microphone groups to attenuate sound captured by the microphones of the first and second microphone groups from substantially the direction of the speaker.

5. The method of claim 1, further comprising:

based at least in part on selecting the correlation value, estimating a direction of the second sound source relative to the communication device, wherein the estimated direction of the second sound source relative to the communication device is substantially a direction in which sound captured by the microphones of the first microphone group is amplified.

6. The method of claim 5, further comprising:

for each microphone group, further processing output of the microphones of the microphone group to amplify sound captured by the microphones of the microphone group from the estimated direction of the second sound source relative to the communication device.

7. The method of claim 1, wherein determining the respective sound signal further comprises:

for each microphone group, determining the respective sound signal from the second sound source based at least in part on removing (i) the sound from the first sound source output by the speaker from (ii) the processed output of the microphones of the microphone group.

8. A communication device comprising:

a speaker;

a plurality of microphones, wherein the plurality of microphones are grouped in a plurality of microphone groups such that each microphone group of the plurality of groups comprises two or more corresponding microphones of the plurality of microphones;

a plurality of null forming modules, wherein each of the plurality of null forming modules is associated with a respective microphone group of the plurality of microphone groups, and wherein each of the plurality of null forming modules is configured to process output of the microphones of the corresponding microphone group to (i) attenuate sound captured by the microphones of the microphone group from substantially a direction of the speaker and (ii) amplify sound captured by the microphones of the microphone group from a respective direction other than the direction of the speaker; and

one or more processing modules including at least one module that selects at least a first microphone group from the plurality of microphone groups based at least in part on the processing of the output of the microphones of the corresponding microphone groups.

9. The communication device of claim 8, the one or more processing modules further comprising:

a plurality of acoustic echo cancellation (AEC) modules, wherein each of the plurality of AEC modules is associated with a respective microphone group of the plurality of microphone groups, wherein one of more of the plurality of AEC modules are configured to detect sound from a sound source that is different from the speaker.

10. The communication device of claim 9, wherein each of the plurality of AEC modules is configured to:

determine a respective sound signal from the sound source, based at least in part on (i) an output of the corresponding null forming module and (ii) the sound output by the speaker; and

generate a corresponding correlation value indicating a level of double talk in the output of the corresponding null forming module, such that a plurality of correlation values are generated corresponding to the plurality of microphone groups.

11. The communication device of claim 10, wherein the at least one module that selects at least the first microphone group comprises:

a double talk factor processing module configured to process the plurality of correlation values by selecting a first correlation value from the plurality of correlation values, wherein the first correlation value corresponds to the first microphone group of the plurality of microphone groups and the selecting of the first microphone group is based at least in part on the selecting of the first correlation value,

wherein the communication device is configured to detect sound from the sound source using signals output from the microphones of the first microphone group.

12. The communication device of claim 11, wherein the double talk factor processing module is further configured to select the first correlation value based on determining that the first correlation value has a highest value among the plurality of correlation values.

13. The communication device of claim 10, wherein each of the plurality of AEC modules is configured to generate, based at least in part on an output of a corresponding null forming module, a corresponding correlation value, such that a plurality of correlation values are generated corresponding to the plurality of microphone groups, the communication device further comprising:

a double talk factor processing module configured to select a first correlation value from the plurality of correlation values, wherein the first correlation value corresponds to a first microphone group and the selecting of the first microphone group is based at least in part on the selecting of the first correlation value; and

the communication device is configured to estimate a direction of the sound source based at least in part on the first correlation value.

14. The communication device of claim 13, wherein the estimated direction of the sound source is substantially a direction in which sound captured by the microphones of the first microphone group is amplified.

15. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

grouping a plurality of microphones of a communication device in a plurality of microphone groups, such that each microphone group comprises two or more microphones;

for each microphone group, processing output of microphones of the microphone group to amplify sound captured by the microphones of the microphone group from a respective particular direction;

based at least in part on the processing the output of the microphones of each of the microphone groups, selecting a first microphone group from the plurality of microphone groups; and

detecting sound from a sound source based at least in part on selecting the first microphone group.

16. The one or more non-transitory computer-readable media of claim 15, wherein the acts further comprise:

for each microphone group, determining a near-to-far ratio for the processed output of the microphones of the microphone group, wherein the near-to-far ratio is an indication of a ratio of (i) a strength of sound from the sound source in the processed output of the microphones of the microphone group and (ii) a strength of sound from a speaker in the processed output of the microphones of the microphone group, the sound from the speaker being received by the microphones of the microphone group via echo from the speaker, the sound source being different from the speaker,

wherein selecting the first microphone group further comprises;

selecting the first microphone group based at least in part on a corresponding first near-to-far ratio having a highest value among a plurality of near-to-far ratios corresponding to the plurality of microphone groups.

17. The one or more non-transitory computer-readable media of claim 16, wherein the acts further comprise estimating, based at least in part on selecting the first microphone group, a direction of the sound source relative to the communication device.

18. The one or more non-transitory computer-readable media of claim 17, wherein the estimated direction of the sound source is substantially a direction in which sound captured by the microphones of the first microphone group is amplified.

19. The one or more non-transitory computer-readable media of claim 17, wherein detecting sound from the sound source further comprises:

based at least in part on estimating the direction of the sound source, detecting sound from the sound source using a first microphone of the plurality of microphones, wherein the first microphone is nearest to the sound source among the plurality of microphones.

20. The one or more non-transitory computer-readable media of claim 17, wherein the acts further comprise:

for each microphone group, further processing output of microphones of the microphone group to amplify sound captured by the microphones of the microphone group from the estimated direction of the sound source.

21. The one or more non-transitory computer-readable media of claim 16, wherein the acts further comprise:

for each microphone group, processing output of microphones of the microphone group to attenuate sound captured by the microphones of the microphone group from substantially a direction of a speaker associated with the communication device.