Adaptive microphone array compensation

• US 9,363,598 B1
• Filed: 02/10/2014
• Issued: 06/07/2016
• Est. Priority Date: 02/10/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Yang, Jun
• Primary Examiner(s): Bernardi, Brenda
• Application Number:
  US14/176,797
• Time in Patent Office: 848 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04R 2430/03 Synergistic effects of band splitting and sub-band processing
• CPC Class Codes: H04R 29/006 Microphone matching
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/04 for correcting frequency response

Abstract

An audio-based system may perform audio beamforming and/or sound source localization based on multiple input microphone signals. Each input microphone signal can be calibrated to a reference based on the energy of the microphone signal in comparison to an energy indicated by the reference. Specifically, respective gains may be applied to each input microphone signal, wherein each gain is calculated as a ratio of a energy reference to the energy of the input microphone signal.

Claims

1. A device, comprising:

a microphone array comprising a plurality of microphones configured to produce a respective plurality of microphone signals;

one or more microphone compensators corresponding to one or more of the plurality of microphone signals, the one or more microphone compensators configured to receive an energy reference signal and a corresponding microphone signal, and configured to;

for each of a plurality of frequencies;

determine an energy of the received microphone signal;

determine a gain associated with the received microphone signal, wherein the gain is based on a ratio of an energy of the energy reference signal and the energy of the received microphone signal; and

produce a compensated microphone signal by applying the gain to the received microphone signal; and

a sound processor comprising one or more of the following;

an audio beamformer configured to process each compensated microphone signal to produce one or more directional audio signals respectively representing sound received from one or more directions relative to the microphone array;

or

a sound localizer configured to analyze the compensated microphone signals to determine one or more positional coordinates of a location of origin of sound received by the microphone array.

2. The device of claim 1, wherein the one or more microphone compensators is further configured to determine the energy of the received microphone signal by averaging squared amplitude values of the received microphone signal.

3. The device of claim 1, wherein the one or more microphone compensators is further configured to determine the energy of the received microphone signal by averaging absolute amplitude values of the received microphone signal.

4. The device of claim 1, further comprising a reference generator that is responsive to one of the microphone signals to produce the energy reference signal by estimating an energy of said one of the microphone signals.

5. The device of claim 1, further comprising:

a reference generator configured to;

decompose the energy reference signal into a first reference sub-signal corresponding to a first frequency;

decompose the energy reference signal into a second reference sub-signal corresponding to a second frequency;

estimate a first energy value for the first reference sub-signal; and

estimate a second energy value for the second reference sub-signal;

the one or more microphone compensators further configured to;

decompose the received microphone signal into a first microphone sub-signal corresponding to the first frequency;

decompose the received microphone signal into a second microphone sub-signal corresponding to the second frequency;

estimate a third energy value for the first microphone sub-signal;

estimate a fourth energy value for the second microphone sub-signal;

calculate a first gain corresponding to the first frequency as a ratio of the first energy value and the third energy value;

calculate a second gain corresponding to the second frequency as a ratio of the second energy value and the fourth energy value;

apply the first gain to the first microphone sub-signal to generate a modified first microphone sub-signal;

apply the second gain to the second microphone sub-signal to generate a modified second microphone sub-signal; and

combine the modified first and second microphone sub-signals to create the compensated microphone signal.

6. A method, comprising:

receiving a plurality of microphone signals;

receiving a reference signal;

estimating an energy of each microphone signal at each of a plurality of frequencies;

estimating an energy of the reference signal at each of the plurality of frequencies; and

for each microphone signal, at each frequency, modifying the microphone signal based at least in part on (a) the estimated energy of the microphone signal at the frequency and (b) the estimated energy of the reference signal at the frequency.

7. The method of claim 6, further comprising providing the microphone signals to at least one of an audio beamformer or a sound source localizer.

8. The method of claim 6, wherein estimating the energy of a particular one of the microphone signals comprises averaging squared amplitude values of the particular microphone signal.

9. The method of claim 6, wherein the reference signal is received from a reference microphone.

10. The method of claim 6, wherein modifying the microphone signal comprises:

calculating a gain as a ratio of (a) the estimated energy of the reference signal at the frequency and (b) the estimated energy of the microphone signal at the frequency; and

modifying the microphone signal as a function of the gain.

11. The method of claim 6, further comprising:

decomposing each microphone signal into a plurality of microphone sub-signals corresponding respectively to each of the plurality of frequencies; and

decomposing the reference signal into a plurality of reference sub-signals corresponding respectively to each of the plurality of frequencies.

12. A method, comprising:

receiving a plurality of microphone signals;

obtaining an energy reference signal;

for each of a plurality of frequencies;

determining an energy of one or more microphone signals of the plurality of microphone signals;

determining a gain for the one or more microphone signals based at least in part on (a) the determined energy of the one or more microphone signals and (b) an energy of the energy reference signal; and

modifying the one or more microphone signals as a function of the determined gain to produce corresponding one or more modified microphone signals.

13. The method of claim 12, further comprising providing the one or more modified microphone signals to at least one of an audio beamformer or a sound source localizer.

14. The method of claim 12, wherein obtaining the energy reference signal comprises:

receiving a reference signal from a reference microphone; and

estimating an energy of the reference signal.

15. The method of claim 12, wherein obtaining the energy reference signal comprises:

receiving a reference signal from a reference microphone; and

estimating energies of the reference signal at different frequencies.

16. The method of claim 12, wherein obtaining the energy reference signal comprises receiving an energy reference value.

17. The method of claim 12, wherein determining the energy of the one or more microphone signals comprises averaging squared amplitude values of the one or more microphone signals.

18. The method of claim 12, wherein the one or more microphone signals has multiple frequency components, the method further comprises:

for each of the multiple frequency components;

obtaining an energy reference signal;

determining an energy of the respective frequency component; and

determining a gain for the respective frequency component, wherein the gain is based at least in part on the energy reference signal corresponding to the respective frequency component and the determined energy of the respective frequency component; and

modifying the one or more microphone signals as a function of the gain calculated for each of the multiple frequency components.

19. The method of claim 18, wherein obtaining the energy reference signal corresponding to the respective frequency component comprises:

receiving a reference microphone signal having multiple frequency components; and

determining an energy of each frequency component of the multiple frequency components of the reference microphone signal.