Beam rejection in multi-beam microphone systems

• US 10,746,840 B1
• Filed: 05/22/2017
• Issued: 08/18/2020
• Est. Priority Date: 04/04/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Barton, William Folwell, Hild, II, Kenneth Edgar, Gopalan, Ramya, Velusamy, Kavitha, Chhetri, Amit Singh
• Primary Examiner(s): Hamid, Ammar T
• Application Number:
  US15/601,981
• Time in Patent Office: 1,184 Days
• Field of Search: 381 92, 381 66
• CPC Class Codes: G01S 3/805 using adjustment of real or effective orientation of directivity characteristics of a transducer or transducer system to give a desired condition of signal derived from that transducer or transducer system, e.g. to give a maximum or minimum signal
• CPC Class Codes: G01S 3/8083 determining direction of source
• CPC Class Codes: G01S 3/86 with means for eliminating undesired waves, e.g. disturbing noises
• CPC Class Codes: G01S 5/22 Position of source determined by co-ordinating a plurality of position lines defined by path-difference measurements G01S5/28 takes precedence
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/401 2D or 3D arrays of transducers
• CPC Class Codes: H04R 2201/403 Linear arrays of transducers
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 25/407 Circuits for combining signals of a plurality of transducers
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

The systems, devices, and processes described herein may identify a beam of a voice-controlled device that is directed toward a reflective surface, such as a wall. The beams may be created by a beamformer. An acoustic echo canceller (AEC) may create filter coefficients for a reference sound. The filter coefficients may be analyzed to identify beams that include multiple peaks. The multiple peaks may indicate presence of one or more reflective surfaces. Using the amplitude and the time delay between the peaks, the device may determine that it is close to a reflective surface in a direction of the beam.

Claims

1. A computing device comprising:

memory;

one or more processors coupled to the memory;

a loudspeaker coupled to the one or more processors;

a plurality of microphones coupled to the one or more processors;

a beamforming component stored in the memory and executable by the one or more processors;

an acoustic echo cancellation (AEC) component stored in the memory and executable by the one or more processors; and

a speech processing component stored in the memory and executable by the one or more processors,

wherein the one or more processors are configured to;

cause the plurality of microphones to generate a first audio signal and a second audio signal based on speech input;

cause the beamforming component to process the first audio signal and the second audio signal to generate a first beamformed signal and a second beamformed signal, respectively;

cause the AEC component to perform echo cancellation on the first beamformed signal and the second beamformed signal to generate first echo-canceled data and second echo-canceled data, respectively;

determine that the first beamformed signal is directed towards a reflective surface; and

cause the speech processing component to;

(i) receive the second echo-canceled data;

(ii) send at least a portion of the second echo-canceled data to a remote computing resource;

(iii) receive response data from the remote computing resource, the response data being generated based on the portion of the second echo-canceled data; and

(iv) output audible sound representing the response data.

2. The computing device as recited in claim 1, wherein the one or more processors are further configured to cause the AEC component to create a first filter coefficient associated with the first beamformed signal, wherein determining that the first beamformed signal is directed towards the reflective surface is based at least in part on the first filter coefficient.

3. The computing device as recited in claim 1, wherein the one or more processors are further configured to cause the AEC component to cancel at least some acoustic echo generated by sound emitted from the loudspeaker.

4. The computing device as recited in claim 1, wherein the one or more processors are further configured to determine a user profile associated with the speech input, and wherein the output of the audible sound is performed is based at least in part on the user profile.

5. The computing device as recited in claim 1, further comprising a noise reduction component, and wherein the one or more processors are further configured to cause the noise reduction component to remove at least some background noise in at least one of the first beamformed signal or the second beamformed signal.

6. The computing device as recited in claim 1, wherein the first audio signal is generated by a different microphone of the plurality of microphones than the second audio signal.

7. A computer-implemented method comprising:

generating a first audio signal and a second audio signal based on speech input received by a plurality of microphones;

processing the first and the second audio signals by a beamforming component to generate a first beamformed signal and a second beamformed signal, respectively;

performing acoustic echo cancellation (AEC) on the first beamformed signal and the second beamformed signal to generate first echo-canceled data and second echo-canceled data, respectively;

determining that the first beamformed signal is directed towards a reflective surface;

sending at least a portion of the second echo-canceled data to a remote computing resource;

receiving response data from the remote computing resource as a result of at least processing the second echo-canceled data by the remote computing resource; and

causing a loudspeaker to output audible sound representing the response data.

8. The computer-implemented method as recited in claim 7, further comprising determine a user profile associated with the speech input, and wherein the response data is based at least in part on the user profile.

9. The computer-implemented method as recited in claim 7, further comprising generating the first audio signal by a first microphone and generating the second audio signal by a second microphone of the plurality of microphones.

10. The computer-implemented method as recited in claim 7, further comprising generating a first filter coefficient associated with the first beamformed signal and a second filter coefficient associated with the second beamformed signal, the first filter coefficient and the second filter coefficient to cancel sound associated with at least one of the first audio signal or the second audio signal, and wherein determining that the first beamformed signal is directed towards the reflective surface is based at least in part on the first filter coefficient.

11. The computer-implemented method as recited in claim 7, wherein the AEC cancels at least some background noise captured by the plurality of microphones to reduce presence of the background noise in the echo-canceled data.

12. The computer-implemented method as recited in claim 7, wherein the AEC cancels at least some ambient noise captured by the plurality of microphones.

13. The computer-implemented method as recited in claim 7, wherein generating the first beamformed signal and the second beamformed signal includes processing by a fixed beamformer.

14. The computer-implemented method as recited in claim 7, further comprising causing the loudspeaker to generate the sound, and wherein the AEC removes the sound from the at least one of the first beamformed signal or the second beamformed signal to generate the echo-canceled data.

15. A system comprising:

memory;

one or more processors coupled to the memory;

a plurality of microphones coupled to the one or more processors;

a beamforming component stored in the memory and executable by the one or more processors;

an acoustic echo cancellation (AEC) component stored in the memory and executable by the one or more processors; and

a speech processing component stored in the memory and executable by the one or more processors,

wherein the one or more processors are configured to;

cause the plurality of microphones to generate a first audio signal and a second audio signal based on speech input;

cause the beamforming component to process the first audio signal and the second audio signal to generate a first beamformed signal and a second beamformed signal;

cause the AEC component to perform echo cancellation on the first beamformed signal and the second beamformed signal to generate first echo-canceled data and second echo-canceled data, respectively;

determine that the first beamformed signal is directed towards a reflective surface; and

cause the speech processing component to process the second echo-canceled data to determine a command and in response to processing the command, send at least a portion of the second echo-canceled data to a remote computing resource and receive response data from the remote computing resource, the response data being generated based on the portion of the second echo-canceled data.

16. The system as recited in claim 15, cause a loudspeaker to output audible sound representing the response data.

17. The system as recited in claim 15, further comprising a detection component, and wherein the one or more processors are further configured to cause the detection component to determine that sound received by the plurality of microphones includes the speech input.

18. The system as recited in claim 15, wherein the one or more processors are further configured to cause the AEC component to generate a first filter coefficient associated with the first beamformed signal and a second filter coefficient associated with the second beamformed signal, the first filter coefficient and the second filter coefficient to facilitate processing beamformed signals to create the echo-canceled data, and wherein determining that the first beamformed signal is directed towards the reflective surface is based at least in part on the first filter coefficient.

19. The system as recited in claim 15, wherein the one or more processors are further configured to determine a user profile associated with the speech input, and wherein the response data is based at least in part on the user profile.

20. The system as recited in claim 15, wherein the one or more processors are further configured to cause the AEC component to cancel at least some acoustic echo generated by sound emitted from a loudspeaker.