Adaption-based reduction of echo and noise

• US 9,595,997 B1
• Filed: 01/02/2013
• Issued: 03/14/2017
• Est. Priority Date: 01/02/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Yang, Jun
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US13/733,033
• Time in Patent Office: 1,532 Days
• Field of Search: 704/200, 704/226, 704/227, 704/228, 704/233, 381/71.1, 381 941- 949
• US Class Current: 1/1
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: H04B 3/23 using a replica of transmitted signal in the time domain, e.g. echo cancellers
• CPC Class Codes: H04M 9/082 using echo cancellers echo cancellers per se H04B3/23

Abstract

The systems, devices, and processes described herein may generate a signal, such as an audio signal or an electrical signal, that may include echo and/or noise. The echo may be at least partially canceled based at least in part on acoustic echo cancellation or local echo cancellation, while the noise may be at least partially reduced based at least in part on noise reduction. A residual amount of echo and/or noise may then be determined. The residual echo and/or noise may be suppressed based at least in part on adaptive filtering of the signal, which may include modifying weights of an algorithm associated with an adaptive filter.

Claims

1. A system comprising:

memory;

one or more processors;

one or more computer-executable instructions stored in the memory and

executable by the one or more processors to;

perform spatial filtering on an audio signal generated by multiple microphones,

the audio signal being associated with noise and an acoustic echo;

reduce at least a portion of the acoustic echo associated with the audio signal and reduce at least a portion of the noise associated with the audio signal to generate an output signal;

determine residual acoustic echo associated with the output signal, the residual acoustic echo representing acoustic echo that has not been removed from the audio signal;

determine residual noise within the output signal, the residual noise representing noise that has not been removed from the audio signal; and

reduce the residual acoustic echo or the residual noise associated with the output signal by;

synchronizing the audio signal and the output signal by delaying the audio signal to account for an amount of time to reduce the residual acoustic echo or the residual noise from the output signal, wherein delaying the audio signal results in a delayed audio signal;

removing, by an adaptive filter, the acoustic echo and the noise to generate an adapted signal;

determining an error signal based at least in part on a difference between the delayed audio signal and the adapted signal;

determining, based at least in part on the error signal, one or more weight modifications to the adaptive filter; and

reducing, by a second filter, at least one of the residual acoustic echo or the residual noise from the output signal based at least in part on respective values of the one or more weight modifications.

2. The system as recited in claim 1, wherein the one or more computer-executable instructions are further executable by the one or more processors to reduce the at least one of the residual acoustic echo or the residual noise from the output signal based at least in part on dynamically modifying one or more weights of an algorithm associated with the adaptive filter to generate modified one or more weights, the modified one or more weights of the algorithm being used to identify a voice represented within a subsequent audio signal.

3. The system as recited in claim 2, wherein the one or more computer-executable instructions are further executable by the one or more processors to:

detect speech represented within the audio signal; and

in response to detecting the speech represented within the audio signal, determine one or more words or phrases associated with the speech.

4. The system of claim 1, wherein the system is associated with a voice-controlled device that includes a speaker, and wherein the acoustic echo is based at least in part on sound that is output by the speaker.

5. A method comprising:

under control of one or more processors of a computing device,

generating an audio signal from sound, the audio signal being associated with an acoustic echo and noise;

reducing at least a portion of the acoustic echo associated with the audio signal and reducing at least a portion of the noise associated with the audio signal to generate an output signal;

determining that the output signal is associated with one or more of residual acoustic echo or residual noise after the reducing of the acoustic echo and the reducing of the noise from the audio signal; and

reducing the residual acoustic echo or the residual noise from the output signal by;

synchronizing the audio signal and the output signal by delaying the audio signal to account for an amount of time to reduce the residual acoustic echo or the residual noise from the output signal, wherein delaying the audio signal results in a delayed audio signal;

removing, by an adaptive filter, the acoustic echo and the noise to generate an adapted signal;

determining an error signal based at least in part on a difference between the delayed audio signal and the adapted signal;

determining, based at least in part on the error signal, one or more weight modifications to the adaptive filter; and

reducing, by a second filter, at least one of the residual acoustic echo or the residual noise from the output signal based at least in part on respective values of the one or more weight modifications.

6. The method as recited in claim 5, wherein the acoustic echo is based at least in part on a microphone capturing the sound and a speaker subsequently outputting the sound.

7. The method as recited in claim 5, further comprising:

determining that a voice is represented within the audio signal based at least in part on an algorithm associated with the adaptive filter;

in response to determining that the voice is represented within the audio signal,

modifying one or more weights of the algorithm to generate one or more modified weights; and

transmitting respective values of the one or more modified weights to the second filter, the second filter reducing the at least one of the residual acoustic echo or the residual noise based at least in part on the respective values of the one or more modified weights.

8. The method as recited in claim 7, further comprising:

determining a subsequent audio signal; and

modifying the one or more modified weights based at least in part on the subsequent audio signal and in response to transmitting the respective values of the one or more modified weights to the second filter.

9. The method as recited in claim 5, further comprising:

determining that a voice is represented within the audio signal; and

in response to determining that the voice is represented within the audio signal, bypassing the audio signal with respect to the adaptive filter.

10. The method as recited in claim 9, further comprising:

determining that one or more of the acoustic echo or the noise is included within the audio signal; and

in response to determining that one or more of the acoustic echo or the noise is included within the audio signal, reducing the acoustic echo or the noise associated with the audio signal.

11. The method as recited in claim 5, further comprising:

detecting speech represented within the audio signal; and

in response to detecting the speech represented within the audio signal, determining one or more words or phrases of the speech.

12. A system comprising:

memory;

one or more processors;

one or more computer-executable instructions stored in the memory and executable by the one or more processors to;

reduce acoustic echo associated with an audio signal and reduce noise associated with the audio signal to generate an output signal;

determine residual acoustic echo associated with the output signal;

determine residual noise associated with the output signal; and

reduce at least one of the residual acoustic echo or the residual noise from the output signal by;

synchronizing the audio signal and the output signal by delaying the audio signal to account for an amount of time to reduce the residual acoustic echo or the residual noise from the output signal, wherein delaying the audio signal results in a delayed audio signal;

removing, by an adaptive filter, the acoustic echo and the noise to generate an adapted signal;

determining an error signal based at least in part on a difference between the delayed audio signal and the adapted signal;

determining, based at least in part on the error signal, one or more weight modifications to the adaptive filter; and

reducing, by a second filter, at least one of the residual acoustic echo or the residual noise from the output signal based at least in part on respective values of the one or more weight modifications.

13. The system as recited in claim 12, wherein the one or more computer-executable instructions are further executable by the one or more processors to determine the audio signal based at least in part on audio captured by multiple microphones.

14. The system as recited in claim 12, wherein the one or more computer-executable instructions are further executable by the one or more processors to:

determine that the audio signal is generated by multiple microphones; and

perform spatial filtering on the audio signal to determine that the audio signal is associated with the noise and the acoustic echo.

15. The system as recited in claim 12, wherein the one or more computer-executable instructions are further executable by the one or more processors to:

determine that the acoustic echo and the noise are associated with the audio signal; and

in response to determining that the acoustic echo and the noise are associated with the audio signal, filter the audio signal by reducing at least a portion of the acoustic echo and the noise.

16. The system as recited in claim 15, wherein the one or more computer-executable instructions are further executable by the one or more processors to:

determine that a target voice is represented within the audio signal; and

in response to determining that the target voice is represented within the audio signal, bypass the audio signal.

17. The system as recited in claim 12, wherein the one or more computer-executable instructions are further executable by the one or more processors to:

reduce the acoustic echo and the noise associated with the audio signal in response to determining that the acoustic echo and the noise is at or above a predetermined threshold; and

bypass the audio signal in response to determining that the acoustic echo and the noise is below the predetermined threshold.

18. The system as recited in claim 12, wherein the system is associated with a voice-controlled device that includes multiple microphones and a speaker, and wherein the acoustic echo is based at least in part on one of the multiple microphones capturing previously captured audio that is output by the speaker.

19. A method comprising:

reduce acoustic echo associated with an audio signal and reduce noise associated with the audio signal to generate an output signal;

determine residual acoustic echo associated with the output signal;

determine residual noise associated with the output signal; and

reduce at least one of the residual acoustic echo or the residual noise from the output signal by;

synchronizing the audio signal and the output signal by delaying the audio signal to account for an amount of time to reduce the residual acoustic echo or the residual noise from the output signal, wherein delaying the audio signal results in a delayed audio signal;

removing, by an adaptive filter, the acoustic echo and the noise to generate an adapted signal;

determining an error signal based at least in part on a difference between the delayed audio signal and the adapted signal;

determining, based at least in part on the error signal, one or more weight modifications to the adaptive filter; and

reducing, by a second filter, at least one of the residual acoustic echo or the residual noise from the output signal based at least in part on respective values of the one or more weight modifications.

20. The method as recited in claim 19, further comprising:

determining that a target voice is represented within the audio signal; and

in response to determining that the target voice is represented within the audio signal, bypassing the audio signal.

21. The method as recited in claim 19, further comprising:

reducing the acoustic echo and the noise associated with the audio signal in response to determining that the acoustic echo and the noise is at or above a predetermined threshold; and

bypassing the audio signal in response to determining that the acoustic echo and the noise is below the predetermined threshold.

22. The method as recited in claim 19, further comprising:

detecting speech represented within the audio signal; and

in response to detecting the speech represented within the audio signal, determining one or more words or phrases of the speech.

23. The method as recited in claim 19, further comprising:

determining that the audio signal is generated by multiple microphones; and

performing spatial filtering on the audio signal to determine that the audio signal is associated with the noise and the acoustic echo.