Speech processing of reflected sound

• US 9,820,036 B1
• Filed: 12/30/2015
• Issued: 11/14/2017
• Est. Priority Date: 12/30/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Tritschler, Charles, Chhetri, Amit Singh, Hilmes, Philip Ryan
• Primary Examiner(s): Edun, Muhammad N
• Application Number:
  US14/984,010
• Time in Patent Office: 685 Days
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: H04R 1/326 for microphones H04R1/34 and H04R1/40 take precedence
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407

Abstract

Sound is banked laterally over an array of microphones arranged on a rear surface of a device. Sound enters a duct behind the device from different directions via inlets along the sides of the device. The duct directs the sound waves across the microphone array. An effective direction from which the banked sounds originated is determined, relative to a front of the device. Based on the determined effective direction, the device applies spatial filtering to isolate the received sound waves, selectively increasing a signal-to-noise ratio of sound from the selected source and at least partially occluding sounds from other sources.

Claims

1. A device comprising:

a front surface;

a rear surface;

spacers that create a space filled with air for sound waves to pass over the rear surface when the device is physically coupled to a wall via the spacers;

an array of microphones arranged to generate audio data representing the sound waves as the sound waves pass over the rear surface, the array of microphones comprising a first microphone and a second microphone;

a processor; and

a memory including instructions operable to be executed by the processor to determine an effective direction of a source of the sound waves relative to the front surface of the device by configuring the processor to;

determine a first product by multiplying a first electrical signal from the first microphone by a first spatial filter transform associated with a first direction;

determine a second product by multiplying a second electrical signal from the second microphone by a second spatial filter transform associated with the first direction;

determine a first magnitude of a first sum of at least the first product and the second product;

determine a third product by multiplying the first electrical signal by a third spatial filter transform associated with a second direction;

determine a fourth product by multiplying the second electrical signal by a fourth spatial filter transform associated with the second direction;

determine a second magnitude of a second sum of at least the third product and the fourth product;

determine that the second magnitude is greater than the first magnitude; and

select the second direction as the effective direction.

2. The device of claim 1, wherein the instructions further configure the processor to:

determine a fifth product by multiplying a third electrical signal from the first microphone by the third spatial filter transform;

determine a sixth product by multiplying a fourth electrical signal from the second microphone by the fourth spatial filter transform;

determine a third sum of the fifth product and the sixth product; and

perform an inverse Fourier transform on the third sum for a plurality of acoustic frequencies, to produce a time-variant signal.

3. The device of claim 1, wherein the array of microphones comprises at least four omnidirectional microphones and the device further comprises a plurality of foam cones arranged in an asymmetric pattern across the rear surface.

4. A method, comprising:

detecting a first soundwave at a first microphone of a microphone array arranged on a rear surface of a device, the first soundwave propagating in a first direction substantially parallel to the rear surface;

detecting a second soundwave at the first microphone, the second soundwave propagating in a second direction substantially parallel to the rear surface;

determining first data representing the first soundwave and second soundwave as detected by the first microphone;

detecting the first soundwave at a second microphone of the microphone array;

detecting the second soundwave at the second microphone;

determining second data representing the first soundwave and the second soundwave as detected at the second microphone; and

determining, using the first data and the second data, an effective direction of a source of the first soundwave and the second soundwave, the effective direction relative to a front surface of the device, opposite the rear surface.

5. The method of claim 4, wherein determining the effective direction comprises:

applying a first set of weights associated with a first direction to the first data and the second data;

applying a second set of weights associated with a second direction to the first data and the second data;

determining that applying the second set of weights produces a larger magnitude than applying the first set of weights; and

selecting the effective direction to be the second direction.

6. The method of claim 5, wherein applying the first set of weights to the first data and the second data comprises:

determining a first product of the first data multiplied by a first weight of the first set of weights;

determining a second product of the second data multiplied by a second weight of the first set of weights;

determining a sum of at least the first product and the second product; and

determining a magnitude of the sum.

7. The method of claim 4, further comprising:

determining a first product of the first data multiplied by a first weight associated with the effective direction;

determining a second product of the second data multiplied by a second weight associated with the effective direction;

determining a sum of at least the first product and the second product; and

performing an inverse Fourier transform on the sum for a plurality of acoustic frequencies, to produce a time-variant signal.

8. The method of claim 4, further comprising:

providing an air-filled space between the rear surface and a barrier that is parallel to the rear surface, there being a fixed distance between the rear surface and the barrier, the air-filled space being open along at least one edge of the rear surface, and the first microphone and the second microphone arranged on the rear surface to face the barrier.

9. The method of claim 8, further comprising:

providing cones of acoustic foam in an asymmetric pattern on the rear surface to dampen the first sound wave after the first sound wave has propagated across at least one of the first microphone and the second microphone.

10. A device comprising:

a front surface;

a rear surface forming an air-filled space with an adjacent acoustic barrier;

a plurality of structures separating the front surface from the rear surface by a fixed distance, the plurality of structures defining at least one inlet; and

an array of microphones arranged on the rear surface to face the acoustic barrier, wherein sound entering the air-filled space through the at least one inlet propagates substantially parallel to the rear surface to cross the array of microphones.

11. The device of claim 10, the device further comprising:

a processor configured to process data from the array of microphones in a frequency range extending from a lower frequency to an upper frequency,

wherein the fixed distance is less than or equal to approximately one-half a wavelength corresponding to the upper frequency.

12. The device of claim 10, further comprising:

an acoustic reflector comprising the acoustic barrier to redirect sound waves incident along at least one edge of a front surface of the device into the air-filled space.

13. The device of claim 10, wherein the array of microphones comprises at least four omnidirectional microphones, center axes of the microphones of the array of microphones being to the rear surface.

14. The device of claim 10, further comprising cones of acoustic foam arranged on the rear surface of the device in an asymmetric pattern to dampen reflections of the sound within the air-filled space.

15. A device comprising:

a front surface;

a rear surface forming an air-filled space with an adjacent acoustic barrier, the air-filled space opening along at least one edge of the rear surface;

an array of microphones arranged on the rear surface to face the acoustic barrier, wherein sound entering the air-filled space propagates substantially parallel to the rear surface to cross the array of microphones;

a processor; and

a memory including instructions operable to be executed by the processor, the instructions configuring the processor to;

determine an effective direction of a source of the sound relative to the front surface of the device based on data generated by the array of microphones in response to the sound propagating in the air-filled space.

16. The device of claim 15, wherein the array of microphones comprises a first microphone and a second microphone, the instructions to determine the effective direction further configuring the processor to:

determine a first product of a first weight associated with a first direction multiplied by first data from the first microphone, the first data representing sound incident on the first microphone;

determine a second product of a second weight associated with the first direction multiplied by second data from the second microphone, the second data representing sound incident on the second microphone;

determine a first magnitude based on a sum of the first product and the second product;

determine a third product of a third weight associated with a second direction multiplied by the first data;

determine a fourth product of a fourth weight associated with the second direction multiplied by the second data;

determine a second magnitude based on a sum of the first product and the second product; and

determine that the second magnitude is larger than the first magnitude; and

select the second direction as the effective direction.

17. The device of claim 16, wherein the instructions further configure the processor to:

apply a set of weights associated with the effective direction to the data generated by the array of microphones, producing a plurality of weighted signals;

determine a sum of the plurality of weighted signals; and

perform an inverse Fourier transform on the sum, to produce output data representing a time-variant audio signal.

18. The device of claim 17, the device further comprising a tactile interface on the front surface, and a speech recognition processor, the speech recognition processor configured to receive the output data representing the time-variant audio signal to identify a speech-based command,

wherein the device is configured to initiate a same action in response to an input received by the tactile interface and in response to the speech-based command.

19. The device of claim 18, wherein the tactile interface is a dimmer configured to control an intensity level of lighting.