Identifying directions of acoustically reflective surfaces

• US 9,294,860 B1
• Filed: 03/10/2014
• Issued: 03/22/2016
• Est. Priority Date: 03/10/2014
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Carlson, Ty Loren
• Primary Examiner(s): Hamid, Ammar
• Application Number:
  US14/203,108
• Time in Patent Office: 743 Days
• Field of Search: 381/66, 381/61
• US Class Current: 1/1
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: H04R 29/008 Visual indication of individual signal levels visual indication of stereophonic sound image H04S7/40
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

An audio device may be configured to produce output audio and to also capture input audio for speech recognition. Beamforming may be used to produce directional audio signals that emphasize sounds received from different directions. Acoustic echo cancellation (AEC) may be used to remove components of output audio from the directional input signals. AEC may be implemented by an adaptive filter based on dynamically optimized filter coefficients. The filter coefficients may be analyzed to detect the presence of potentially problematic reflective surfaces near the audio device. When such a reflective surface is detected, the user may be notified by illuminating a directional indicator on a housing of the audio device, where the directional indicator indicates the direction of the potentially problematic surface relative to the audio device.

Claims

1. An audio device, comprising:

a housing;

a speaker to generate output audio;

multiple microphones to produce input audio signals;

one or more audio beamformers configured to produce multiple directional audio signals from the input audio signals, wherein each of the directional audio signals potentially contains one or more acoustic echoes of the output audio, and wherein each of the directional audio signals emphasizes sound received from a corresponding direction relative to the audio device;

one or more visual indicators, the one or more visual indicators being controllable to visually indicate different directions relative to the audio device;

one or more acoustic echo cancellers configured to cancel the one or more acoustic echoes from the directional audio signals based at least in part on sequences of dynamically updated filter coefficients;

a proximity detector configured to perform acts comprising;

analyzing the sequences of dynamically updated filter coefficients to identify one of the directional audio signals that contains an acoustic echo originating from a nearest of one or more acoustically reflective surfaces; and

controlling the one or more visual indicators to indicate the direction of the nearest of the one or more acoustically reflective surfaces.

2. The audio device of claim 1, wherein the coefficients of each sequence correspond to echo strengths at respectively corresponding acoustic path lengths, the analyzing further comprising, for an individual sequence:

identifying a peak coefficient in the sequence, wherein the peak coefficient corresponds to a first reflection of the output audio and to a particular one of the acoustic path lengths;

comparing the particular one of the acoustic path lengths to a length threshold; and

comparing the peak coefficient to an echo strength threshold.

3. The audio device of claim 1, wherein the proximity detector is configured to perform the acts in response to a detected impairment of speech recognition.

4. The audio device of claim 1, wherein:

the housing is cylindrical and has a circular top surface;

the one or more visual indicators comprise a ring-shaped display element on the top surface of the housing; and

controlling the one or more visual indicators comprises illuminating a portion of the display element that corresponds to the direction of the nearest of the one or more acoustically reflective surfaces.

5. The audio device of claim 1, wherein each of the one or more acoustic echo cancellers comprises a finite impulse response filter that is responsive to a corresponding one of the sequences of coefficients.

6. A device comprising:

a speaker configured to generate output audio;

a microphone array configured to produce multiple input audio signals;

one or more audio beamformers configured to produce multiple directional audio signals from the input audio signals, wherein each directional audio signal potentially contains one or more acoustic echoes of the output audio;

one or more acoustic echo cancellers configured to cancel the one or more acoustic echoes from each of the directional audio signals; and

a proximity detector configured to perform acts comprising;

identifying at least one of the directional audio signals that contains an acoustic echo of the output audio caused by a nearby acoustically reflective surface that is within a threshold proximity of the device; and

providing a proximity indication that is based at least in part on the direction corresponding to the identified one of the directional audio signals.

7. The device of claim 6, wherein the proximity indication specifies the direction corresponding to the identified one of the directional audio signals to identify the direction of the nearby acoustically reflective surface.

8. The device of claim 6, wherein the proximity indication specifies a direction opposite to the direction corresponding to the identified one of the directional audio signals to identify a direction in which the device should be moved.

9. The device of claim 6, further comprising one or more visual indicators that are controllable to visually indicate different directions relative to the device, wherein providing the proximity indication comprises controlling the one or more visual indicators.

10. The device of claim 6, wherein the acoustic echo has a strength that exceeds a strength threshold.

11. The device of claim 6, wherein the identifying and providing are performed by the proximity detector in response to detected impairment of recognizing speech in one or more of the directional audio signals.

12. The device of claim 6, wherein:

each of the one or more acoustic echo cancellers comprises an adaptive filter that dynamically generates a plurality of filter parameters indicating (a) strengths of the acoustic echoes and (b) proximities of acoustically reflective surfaces that cause the acoustic echoes; and

the proximity detector is configured analyze the filter parameters to identify the at least one of the directional audio signals that contains the acoustic echo of the output audio.

13. The device of claim 12, wherein the adaptive filter comprises an adaptive finite impulse response (FIR) filter.

14. The device of claim 12, wherein analyzing the filter parameters comprises:

comparing the strength of the acoustic echo of the output audio caused by the nearby acoustically reflective surface to a strength threshold; and

comparing the proximity of the nearby acoustically reflective surface to a proximity threshold.

15. A method comprising:

receiving multiple directional audio signals based at least in part on input audio signals produced by multiple microphones;

cancelling acoustic echoes that are potentially present in the directional audio signals due to one or more acoustically reflective surfaces, wherein the cancelling is based on dynamically calculated parameters that indicate proximities of the one or more acoustically reflective surfaces to the multiple microphones;

analyzing the dynamically calculated parameters to identify at least one of the directional audio signals having an acoustic echo caused by one of the one or more acoustically reflective surfaces within at least a threshold proximity of the multiple microphones;

providing an indication that is based at least in part on the direction corresponding to the identified at least one of the directional audio signals.

16. The method of claim 15, further comprising analyzing the dynamically calculated parameters to identify at least one of the directional audio signals having an acoustic echo exceeding a threshold magnitude.

17. The method of claim 15, wherein the proximity indication specifies the direction corresponding to the identified one of the directional audio signals.

18. The method of claim 15, wherein the proximity indication specifies a direction opposite to the direction corresponding to the identified one of the directional audio signals.

19. The method of claim 15, wherein the cancelling uses an adaptive filter and the dynamically calculated parameters comprise filter coefficients that are dynamically updated by the adaptive filter.

20. The method of claim 15, wherein the analyzing and providing are performed in response to a detected impairment of recognizing speech in one or more of the directional audio signals.

21. The method of claim 15, further comprising performing acoustic beamforming to produce the multiple directional audio signals based at least in part on the input audio signals.