System and method for beamforming using a microphone array

• US 7,415,117 B2
• Filed: 03/02/2004
• Issued: 08/19/2008
• Est. Priority Date: 03/02/2004
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Tashev, Ivan, Malvar, Henrique
• Primary Examiner(s): Chin; Vivian
• Assistant Examiner(s): Monikang; George C
• Application Number:
  US10/792,313
• Publication Number:
  US 20050195988A1
• Time in Patent Office: 1,631 Days
• Field of Search: 381/92, 381/91, 381/93, 381/66, 367/119, 367/901, 702/191, 702/194, 704/225, 704/226
• US Class Current: 381/92
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

The ability to combine multiple audio signals captured from the microphones in a microphone array is frequently used in beamforming systems. Typically, beamforming involves processing the output audio signals of the microphone array in such a way as to make the microphone array act as a highly directional microphone. In other words, beamforming provides a “listening beam” which points to a particular sound source while often filtering out other sounds. A “generic beamformer,” as described herein automatically designs a set of beams (i.e., beamforming) that cover a desired angular space range within a prescribed search area. Beam design is a function of microphone geometry and operational characteristics, and also of noise models of the environment around the microphone array. One advantage of the generic beamformer is that it is applicable to any microphone array geometry and microphone type.

Claims

1. A method for real-time design of beam sets for a microphone array from a set of pre-computed noise models, comprising using a computing device to:

compute a set of complex-valued gains for each subband of a frequency-domain decomposition of microphone array signal inputs for each of a plurality of beam widths within a range of beam widths, said sets of complex-valued gains being computed from the pre-computed noise models in combination with known geometry and directivity of microphones comprising the microphone array;

search the sets of complex-valued gains to identify a single set of complex-valued gains for each frequency-domain subband and for each of a plurality of target focus points around the microphone array; and

wherein each said set of complex-valued gains is individually selected as the set of complex-valued gains having a lowest total noise energy relative to corresponding sets of complex-valued gains for each frequency-domain subband for each target focus point around the microphone array, and wherein each selected set of complex-valued gains is then provided as an entry in said beam set for the microphone array.

2. The method of claim 1 wherein the frequency-domain decomposition is a Modulated Complex Lapped Transform (MCLT).

3. The method of claim 1 wherein the frequency-domain decomposition is a Fast Fourier Transform (FFT).

4. The method of claim 1 wherein the pre-computed noise models include at least one of ambient noise models, instrumental noise models, and point source noise models.

5. The method of claim 4 wherein the ambient noise models are computed by direct sampling and averaging of isotropic noise in a workspace around the microphone array.

6. The method of claim 4 wherein the instrumental noise models are computed by direct sampling and averaging of the output of the microphones in the microphone array in a workspace without noise and reverberation, so that only those noises originating from the circuitry of the microphone array is sampled.

7. The method of claim 1 wherein the total noise energy is computed as a function of the pre-computed noise models and the beam widths in combination with the corresponding sets of complex-valued gains.

8. The method of claim 1 wherein at least one member of the set of pre-computed noise models is recomputed in real-time in response to changes in noise levels around the microphone array.

9. The method of claim 1 wherein the sets of complex-valued gains are normalized to ensure unit gain and zero phase shift for signals originating from each target focus point.

10. The method of claim 1 wherein the range of beam widths is defined by a pre-determined minimum beam width, a pre-determined maximum beam width, and a pre-determined beam width step size.

11. The method of claim 1 wherein the range of beam widths is defined by a user adjustable minimum beam width, a user adjustable maximum beam width, and a user adjustable beam width step size.

12. The method of claim 1 wherein the known geometry and directivity of the microphones comprising the microphone array are provided from a device description file which defines operational characteristics of the microphone array.

13. The method of claim 12 wherein the device description file is internal to the microphone array, and wherein the known geometry and directivity of the microphones comprising the microphone array are automatically reported to the computing device for use in the real-time design of beam sets.

14. The method of claim 1 further comprising a beamforming processor for applying the beam set for real-time processing of incoming microphone signals from the microphone array.

15. A system for automatically designing beam sets for a sensor array, comprising:

monitoring all sensor signal outputs of a sensor array having a plurality of sensors, each sensor having a known geometry and directivity pattern;

generating at least one noise model from the sensor signal outputs;

defining a set of target beam shapes as a function of a set of target beam focus points and a range of target beam widths, said target beam focus points being spatially distributed within a workspace around the sensor array;

defining a set of target weight functions to provide a gain for weighting each target focus point depending upon the position of each target focus point relative to a particular target beam shape;

computing a set of potential beams by computing a set of normalized weights for fitting the directivity pattern of each microphone into each target beam shape throughout the range of target beam widths across a frequency range of interest for each weighted target focus point;

identifying a set of beams by computing a total noise energy for each potential beam across a frequency range of interest, and selecting each potential beam having a lowest total noise energy for each of a set of frequency bands across the frequency range of interest.

16. The system of claim 15 wherein the normalized weights represent sets of complex-valued gains for each subband of a frequency-domain decomposition of sensor array signal inputs.

17. The system of claim 16 wherein the frequency-domain decomposition is a Modulated Complex Lapped Transform (MCLT).

18. The system of claim 16 wherein the frequency-domain decomposition is a Fast Fourier Transform (FFT).

19. The system of claim 15 wherein generating the at least one noise model from the sensor signal outputs comprises computing at least one of an ambient noise model, an instrumental noise model, and a point source noise model through direct sampling and analysis of noise in a workspace around the sensor array.

20. The system of claim 15 wherein computing the total noise energy for each potential beam across a frequency range of interest comprises determining noise energy levels as a function of the at least one noise model and the normalized weights associated with each potential beam.

21. The system of claim 15 wherein at least one of the noise models is automatically recomputed in real-time in response to changes in noise levels around the sensor array.

22. The system of claim 15 wherein the normalized weights for each potential beam ensure unit gain and zero phase shift for signals originating from each corresponding target focus point.

23. The system of claim 15 wherein the range of target beam widths is limited by minimum and maximum beam widths in combination with a beam width angle step size for selecting specific target beam widths across the range of target beam widths.

24. The system of claim 15 wherein the known geometry and directivity of each sensor is automatically provided from a device description file residing within the sensor array.

25. The system of claim 15 further comprising a beamforming processor for real-time steerable beam-based processing of sensor array inputs by applying the set of beams to the sensor array inputs for particular target focus points.

26. A computer-readable medium having computer executable instructions for automatically designing a set of steerable beams for processing output signals of a microphone array, said computer executable instructions comprising:

computing sets of complex-valued gains for each of a plurality of beams through a range of beam widths for each of a plurality of target focus points around the microphone array from a set of parameters, said parameters including one or more models of noise of an environment within range of microphones in the microphone array and known geometry and directivity patterns of each microphone in the microphone array;

wherein each beam is automatically selected throughout the range of beam widths using a beam width angle step size for selecting specific beam widths across the range of beam widths;

computing a lowest total noise energy for each set of complex-valued gains for each target focus point for each beam width; and

identifying the sets of complex-valued gains and corresponding beam width having the lowest total noise energy for each target focus point, and selecting each such set as a member of the set of steerable beams for processing the output signals of a microphone array.

27. The computer readable medium of claim 26 wherein the complex-valued gains are normalized to ensure unit gain and zero phase shift for signals originating from corresponding target focus points.

28. The computer readable medium of claim 26 wherein the complex-valued gains are separately computed for each subband of a frequency-domain decomposition of microphone array input signals.

29. The computer readable medium of claim 28 wherein the frequency-domain decomposition is any of a Modulated Complex Lapped Transform (MCLT)-based decomposition, and a Fast Fourier Transform (FFT)-based decomposition.

30. The computer readable medium of claim 26 further comprising a beamforming processor for applying the set of steerable beams for processing output signals of the microphone array.

31. The computer readable medium of claim 30 wherein the beamforming processor comprises a sound source localization (SSL) system for using the optimized set of steerable beams for localizing audio signal sources within an environment around the microphone array.

32. The computer readable medium of claim 31 wherein the beamforming processor comprises an acoustic echo cancellation (AEC) system for using the optimized set of steerable beams for canceling echoes outside of a particular steered beam.

33. The computer readable medium of claim 31 wherein the beamforming processor comprises a directional filtering system for selectively filtering audio signal sources relative to the target focus point of one or more steerable beams.

34. The computer readable medium of claim 31 wherein the beamforming processor comprises a selective signal capture system for selectively capturing audio signal sources relative to the target focus point of one or more steerable beams.

35. The computer readable medium of claim 31 wherein the beamforming processor comprises a combination of two or more of:

a sound source localization (SSL) system for using the optimized set of steerable beams for localizing audio signal sources within an environment around the microphone array;

an acoustic echo cancellation (AEC) system for using the optimized set of steerable beams for canceling echoes outside of a particular steered beam;

a directional filtering system for selectively filtering audio signal sources relative to the target focus point of one or more steerable beams; and

a selective signal capture system for selectively capturing audio signal sources relative to the target focus point of one or more steerable beams.