Multi-channel acoustic echo cancellation

• US 10,482,868 B2
• Filed: 09/28/2017
• Issued: 11/19/2019
• Est. Priority Date: 09/28/2017
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Sereshki, Saeed Bagheri, Kadri, Romi
• Primary Examiner(s): Chin, Vivian C
• Assistant Examiner(s): Odunukwe, Ubachukwu A
• Application Number:
  US15/718,911
• Publication Number:
  US 20190096384A1
• Time in Patent Office: 782 Days
• Field of Search: 381 66
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10K 11/178 by electro-acoustically regenerating the original acoustic waves in anti-phase
• CPC Class Codes: G10K 2210/3012 Algorithms
• CPC Class Codes: G10K 2210/505 Echo cancellation, e.g. multipath-, ghost- or reverberation-cancellation
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: H04B 17/336 Signal-to-interference ratio [SIR] or carrier-to-interference ratio [CIR]
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04M 9/082 using echo cancellers echo cancellers per se H04B3/23
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F

Abstract

A method of operating a playback device includes receiving source audio content that includes a first and second channel stream of audio. The method also includes playing back, via a first and second speaker driver of the playback device, the first and second channel streams of audio, thereby producing a first and second channel audio output. A captured stream of audio is received by a microphone of the playback device, and portions of the captured stream of audio correspond to the first and second channel audio outputs. The first and second channel streams of audio are combined into a compound audio signal, and acoustic echo cancellation is performed on the compound audio signal to produce an acoustic echo cancellation output, which is then applied to the captured stream of audio to increase the signal-to noise ratio of the captured stream of audio.

Claims

1. A method of operating a playback device having a first speaker driver, at least a second speaker driver, and one or more microphones, the method comprising:

receiving, via a network interface of the playback device, a source stream of audio comprising source audio content to be played back by the playback device, wherein the source audio content comprises a first channel stream of audio and a second channel stream of audio;

producing a first channel audio output by playing back, via the first speaker driver, the first channel stream of audio;

producing a second channel audio output by playing back, via the second speaker driver, the second channel stream of audio;

receiving, via the one or more microphones, a captured stream of audio comprising a first portion corresponding to the first channel audio output and a second portion corresponding to the second channel audio output, wherein the captured stream of audio has a first signal-to-noise ratio;

performing a singular value decomposition on the first channel stream of audio and the second channel stream of audio to result in a combined set of signal components;

selecting a subset of the combined set of signal components based on one or more parameters;

performing acoustic echo cancellation on the subset of the combined set of signal components, wherein performing acoustic echo cancellation on the subset of the combined set of signal components produces a first acoustic echo cancellation output; and

applying the first acoustic echo cancellation output to the captured stream of audio, thereby increasing the signal-to noise ratio of the captured stream of audio from the first signal-to-noise ratio to a second signal-to-noise ratio, wherein the second signal-to-noise ratio is greater than the first signal-to-noise ratio.

2. The method of claim 1, wherein the captured stream of audio comprises a third portion corresponding to a vocal command issued by a user, and wherein applying the first acoustic echo cancellation output to the captured stream of audio, thereby increasing the signal-to-noise ratio of the captured stream of audio from the first signal-to-noise ratio to the second signal-to-noise ratio, results in the first portion and second portion being eliminated or minimized in the captured stream of audio.

3. The method of claim 1, further comprising:

transforming both of the subset of the combined set of signal components and the captured stream of audio into a Short-Time Fourier Transform domain.

4. The method of claim 1, wherein selecting the subset of the combined set of signal components based on the one or more parameters comprises selecting a subset of the combined set of signal components having at least one of (a) an energy content above a first threshold energy content or (b) a calculated variance above a first threshold variance.

5. The method of claim 1,

wherein the playback device comprises a third speaker driver,

wherein the source audio content comprises the first channel stream of audio, the second channel stream of audio, and a third channel stream of audio, and

wherein the method further comprises producing a third channel audio output by playing back, via the third speaker driver, the third channel stream of audio.

6. The method of claim 5, wherein performing the singular value decomposition on the first channel stream of audio and the second channel stream of audio to result in the combined set of signal components comprises:

performing the singular value decomposition on the first channel stream of audio, the second channel stream of audio, and the third channel stream of audio to result in the combined set of signal components.

7. The method of claim 1, further comprising:

detecting a trigger to perform acoustic echo cancellation on the subset of the combined set of signal components, wherein detecting the trigger comprises detecting that (a) a playback function is initiated by the playback device or (b) an unmute command is received by the playback device after the playback function is initiated.

8. The method of claim 1, wherein performing the singular value decomposition on the first channel stream of audio is performed by one or more processors of the playback device,

wherein performing the singular value decomposition on the second channel stream of audio is performed by the one or more processors of the playback device,

wherein selecting the subset of the combined set of signal components based on the one or more parameters is performed by the one or more processors of the playback device,

wherein performing acoustic echo cancellation on the subset of the combined set of signal components is performed by the one or more processors of the playback device, and

wherein applying the first acoustic echo cancellation output to the captured stream of audio is performed by the one or more processors of the playback device.

9. A tangible, non-transitory computer-readable medium storing instructions that when executed by one or more processors cause a playback device to perform functions comprising:

receiving, via a network interface of the playback device, a source stream of audio comprising source audio content to be played back by the playback device, wherein the playback device comprises a first speaker driver and at least a second speaker driver and further comprises one or more microphones, and wherein the source audio content comprises a first channel stream of audio and a second channel stream of audio;

producing a first channel audio output by playing back, via the first speaker driver, the first channel stream of audio;

producing a second channel audio output by playing back, via the second speaker driver, the second channel stream of audio;

receiving, via the one or more microphones, a captured stream of audio comprising a first portion corresponding to the first channel audio output, and further comprising a second portion corresponding to the second channel audio output, wherein the captured stream of audio has a first signal-to-noise ratio;

performing a singular value decomposition on the first channel stream of audio and the second channel stream of audio to result in a combined set of signal components;

selecting a subset of the combined set of signal components based on one or more parameters;

performing acoustic echo cancellation on the subset of the combined set of signal components, wherein performing acoustic echo cancellation on the subset of the combined set of signal components produces a first acoustic echo cancellation output; and

applying the first acoustic echo cancellation output to the captured stream of audio, thereby increasing the signal-to noise ratio of the captured stream of audio from the first signal-to-noise ratio to a second signal-to-noise ratio, wherein the second signal-to-noise ratio is greater than the first signal-to-noise ratio.

10. The computer-readable medium of claim 9, wherein the captured stream of audio comprises a third portion corresponding to a vocal command issued by a user, and wherein applying the first acoustic echo cancellation output to the captured stream of audio, thereby increasing the signal-to-noise ratio of the captured stream of audio from the first signal-to-noise ratio to the second signal-to-noise ratio, results in the first portion and second portion being eliminated or minimized in the captured stream of audio.

11. The computer-readable medium of claim 9, further comprising instructions that when executed by the one or more processors cause the playback device to perform functions comprising:

transforming both of the subset of the combined set of signal components and the captured stream of audio into a Short-Time Fourier Transform domain.

12. The computer-readable medium of claim 9, wherein selecting the subset of the combined set of signal components based on the one or more parameters comprises selecting a subset of the combined set of signal components having at least one of (a) an energy content above a first threshold energy content or (b) a calculated variance above a first threshold variance.

13. The computer-readable medium of claim 9,

wherein the playback device comprises a third speaker driver,

wherein the source audio content comprises the first channel stream of audio, the second channel stream of audio, and a third channel stream of audio, and

wherein the method further comprises producing a third channel audio output by playing back, via the third speaker driver, the third channel stream of audio.

14. The computer-readable medium of claim 13, wherein performing the singular value decomposition on the first channel stream of audio and the second channel stream of audio to result in the combined set of signal components comprises:

performing the singular value decomposition on the first channel stream of audio, the second channel stream of audio, and the third channel stream of audio to result in the combined set of signal components.

15. The computer-readable medium of claim 9, further comprising instructions that when executed by the one or more processors cause the playback device to perform functions comprising:

detecting a trigger to perform acoustic echo cancellation on the subset of the first combined set of signal components, wherein detecting the trigger comprises detecting that (a) a playback function is initiated by the playback device or (b) an unmute command is received by the playback device after the playback function is initiated.

16. The computer-readable medium of claim 9,

wherein performing the singular value decomposition on the first channel stream of audio is performed by the one or more processors,

wherein performing the singular value decomposition on the second channel stream of audio is performed by the one or more processors,

wherein selecting the subset of the combined set of signal components based on the one or more parameters is performed by the one or more processors,

wherein performing acoustic echo cancellation on the subset of the combined set of signal components is performed by the one or more processors, and

wherein applying the first acoustic echo cancellation output to the captured stream of audio is performed by the one or more processors.