Ultra small microphone array

• US 7,809,145 B2
• Filed: 05/04/2006
• Issued: 10/05/2010
• Est. Priority Date: 05/04/2006
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Mao, Xiadong
• Primary Examiner(s): Chin, Vivian
• Assistant Examiner(s): BLAIR, KILE O
• Application Number:
  US11/381,729
• Publication Number:
  US 20070260340A1
• Time in Patent Office: 1,615 Days
• Field of Search: 702190-195, 702/196, 704/233, 704/236, 381/94.1, 381/66, 381/92, 381/122, 381/91, 381/95, 381/97, 381/98, 381/356, 700/94
• US Class Current: 381/92
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/401 2D or 3D arrays of transducers
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Methods and apparatus for signal processing are disclosed. A discrete time domain input signal x_m(t) may be produced from an array of microphones M₀ . . . M_M. A listening direction may be determined for the microphone array. The listening direction is used in a semi-blind source separation to select the finite impulse response filter coefficients b₀, b₁ . . . , b_N to separate out different sound sources from input signal x_m(t). One or more fractional delays may optionally be applied to selected input signals x_m(t) other than an input signal x₀(t) from a reference microphone M₀. Each fractional delay may be selected to optimize a signal to noise ratio of a discrete time domain output signal y(t) from the microphone array. The fractional delays may be selected to such that a signal from the reference microphone M₀ is first in time relative to signals from the other microphone(s) of the array. A fractional time delay Δ may optionally be introduced into an output signal y(t) so that: y(t+Δ)=x(t+Δ)*b₀+x(t−1+Δ)*b₁+x(t−2+Δ)*b₂+ . . . +x(t−N+Δ)b_N, where Δ is between zero and ±1.

Claims

1. A method for digitally processing a signal from an array of two or more microphones M₀ . . . M_M, the method comprising:

producing a discrete time domain input signal x_m(t) at a runtime from each of the two or more microphones M₀ . . . M_M, where M is greater than or equal to 1;

determining a listening direction of the microphone array with a digital signal processing system having a digital processor coupled to a memory by

forming analysis frames of a pre-recorded signal stored in the memory from a source located in a preferred known listening direction with respect to the microphone array for a predetermined period of time at predetermined intervals using the processor,

transforming the analysis frames into the frequency domain using the processor,

estimating a calibration covariance matrix from vectors formed from the analysis frames that have been transformed into the frequency domain using the processor,

computing an eigenmatrix of the calibration covariance matrix, and

computing an inverse of the eigenmatrix;

using the known listening direction in a semi-blind source separation implemented by the processor to select a set of N finite impulse response filter coefficients b_i, where N is a positive integer.

2. The method of claim 1 wherein using the listening direction in a semi-blind source separation includes:

transforming each input signal x_m(t) to a frequency domain to produce a frequency domain input signal vector for each of k=0;

N frequency bins;

generating a runtime covariance matrix from each frequency domain input signal vector;

multiplying the runtime covariance matrix by the inverse of the eigenmatrix to produce a mixing matrix;

generating a mixing vector from a diagonal of the mixing matrix;

multiplying an inverse of the mixing vector by the frequency domain input signal vector to produce a vector containing independent components of the frequency domain input signal vector.

3. The method of claim 1, further comprising applying one or more fractional delays to one or more of the time domain input signals x_m(t) other than an input signal x₀(t) from a reference microphone M₀, wherein each fractional delay is selected to optimize a signal to noise ratio of a discrete time domain output signal y(t) from the microphone array and wherein the fractional delays are selected to such that a signal from the reference microphone M₀ is first in time relative to signals from the other microphone(s) of the array.

4. The method of claim 3 wherein the fractional delay is greater than a minimum delay, wherein the minimum delay is long enough to capture reverberation from the signal.

5. The method of claim 1, further comprising introducing a fractional time delay Δ

into the output signal y(t) so that;

y(t+Δ

)=x(t+Δ

)*b₀+x(t−

1+Δ

)*b₁+x(t−

2+Δ

)*b₂+ . . . +x(t−

N+Δ

)*b_N, where Δ

is between zero and ±

1, and where b₀, b₁, b₂ . . . , b_N are the finite impulse response filter coefficients b_i, where the symbol “

*”

represents the convolution operation.

6. The method of claim 5 further comprising determining values of the impulse response functions b_i that best separate two or more sources of sound from the input signals x_m(t).

7. The method of claim 5 wherein neighboring microphones in the microphone array are separated from each other by a distance of less than about 4 centimeters.

8. The method of claim 7 wherein neighboring microphones in the microphone array are separated from each other by a distance of between about 1 centimeter and about 2 centimeters.

9. The method of claim 5 wherein the microphones M₀ . . . M_M are characterized by a maximum response frequency of less than about 16 kilohertz.

10. The method of claim 5 wherein the microphones M₀ . . . M_M are characterized by a maximum response frequency of less than about 16 kilohertz and wherein neighboring microphones in the microphone array are separated from each other by a distance of less than about 4 centimeters.

11. The method of claim 5 wherein the microphones M₀ . . . M_M are characterized by a maximum response frequency of less than about 16 kilohertz and wherein neighboring microphones in the microphone array are separated from each other by a distance of between about 0.5 centimeter and about 2 centimeters.

12. The method of claim 5, wherein introducing a fractional time delay Δ

into the output signal y(t) includes;

delaying each time domain input signal x_m(t) by j+1 frames, where j is greater than or equal to 1; and

transforming each input signal x_m(t) to a frequency domain to produce a frequency domain input signal vector X_jk for each of k=0;

N frequency bins, such that there are N+1 frequency bins.

13. The method of claim 12, further comprising determining values of filter coefficients for each microphone m, each frame j and each frequency bin k, b_jk=[b_0j(k), b_1j(k), b_2j(k), b_3j(k)] that best separate out two or more sources of sound from the input signals x_m(t).

14. The method of claim 13 wherein determining the listening direction includes:

recording a signal from a source located in a preferred listening direction with respect to the microphone for a predetermined period of time;

forming analysis frames of the signal at predetermined intervals;

transforming the analysis frames into the frequency domain;

estimating a calibration covariance matrix from a vector of the analysis frames that have been transformed into the frequency domain;

computing an eigenmatrix of the calibration covariance matrix; and

computing an inverse of the eigenmatrix and wherein determining the values of filter coefficients for each microphone m, each frame j and each frequency bin k, b_jk includes;

generating a runtime covariance matrix from each frequency domain input signal vector X_jk;

multiplying the runtime covariance matrix by the inverse of the eigenmatrix to produce a mixing matrix;

generating a mixing vector from a diagonal of the mixing matrix; and

determining the values of b_jk from one or more components of the mixing vector.

15. The method of claim 1 wherein the two or more microphones M₀ . . . M_M are omni-directional microphones.

16. A signal processing apparatus, comprising:

an array of two or more microphones M₀ . . . M_M wherein each of the two or more microphones is adapted to produce a discrete time domain input signal x_m(t) at a runtime;

one or more processors coupled to the array of two or more microphones; and

a memory coupled to the array of two or more microphones and the processor, the memory having embodied therein a set of processor readable instructions configured to implement a method for digitally processing a signal, the processor readable instructions including;

one or more instructions for determining a listening direction of the microphone array from the discrete time domain input signals x_m(t) by

forming analysis frames of a pre-recorded a signal from a source located in a preferred known listening direction with respect to the microphone array for a predetermined period of time at predetermined intervals,

transforming the analysis frames into the frequency domain,

estimating a calibration covariance matrix from vectors formed from the analysis frames that have been transformed into the frequency domain,

computing an eigenmatrix of the calibration covariance matrix, and

computing an inverse of the eigenmatrix; and

one or more instructions for using the known listening direction in a semi-blind source separation to select filtering functions to separate out two or more sources of sound from the discrete time domain input signals x_m(t).

17. The apparatus of claim 16, wherein the processor readable instructions further include

one or more instructions for applying one or more fractional delays to one or more of the time domain input signals x_m(t) other than an input signal x₀(t) from a reference microphone M₀, wherein each fractional delay is selected to optimize a signal to noise ratio of a discrete time domain output signal y(t) from the microphone array and wherein the fractional delays are selected to such that a signal from the reference microphone M₀ is first in time relative to signals from the other microphone(s) of the array.

18. The apparatus of claim 16 wherein the processor readable instructions further include one or more instructions for introducing a fractional time delay Δ

into the output signal y(t) so that;

y(t)=x(t)*b_0+x(t−

1+Δ

)*b₁+x(t−

2+Δ

)*b₂Δ

. . . +x(t−

N+Δ

)*b_N, where Δ

is between zero and ±

1, and where b₀, b₁, b₂ . . . , b_N are finite impulse response filter coefficients, where the symbol “

*”

represents the convolution operation.

19. The apparatus of claim 18 wherein the one or more instructions for introducing a fractional time delay Δ

into the output signal y(t) include;

one or more instructions for delaying each time domain input signal x_m(t) by j+1 frames, where j is greater than or equal to 1; and

transforming each input signal x_m(t) to a frequency domain to produce a frequency domain input signal vector X_jk for each of k=0;

N frequency bins, such that there are N+1 frequency bins.

20. The apparatus of claim 18 wherein neighboring microphones in the microphone array are separated from each other by a distance of less than about 4 centimeters.

21. The apparatus of claim 20 wherein neighboring microphones in the microphone array are separated from each other by a distance of between about 1 centimeter and about 2 centimeters.

22. The apparatus of claim 18 wherein the microphones M₀ . . . M_M array are characterized by a maximum response frequency of less than about 16 kilohertz.

23. The apparatus of claim 18 wherein the microphones M₀ . . . M_M array are characterized by a maximum response frequency of less than about 16 kilohertz and wherein neighboring microphones in the microphone array are separated from each other by a distance of less than about 4 centimeters.

24. The apparatus of claim 18 wherein the microphones M₀ . . . M_M array are characterized by a maximum response frequency of less than about 16 kilohertz and wherein neighboring microphones in the microphone array are separated from each other by a distance of between about 1 centimeter and about 2 centimeters.

25. The apparatus of claim 16 wherein the two or more microphones M₀ . . . M_M are omni-directional microphones.

26. The apparatus of claim 16 wherein the one or more processors include a power processor element (PPE) and one or more synergistic processor elements (SPE) of a cell processor.

27. A method for digitally processing a signal from an array of two or more microphones M₀ . . . M_M, the method comprising:

receiving an audio signal at each of the two or more microphones M₀ . . . M_M;

producing a discrete time domain input signal x_m(t) at a runtime from each of the two or more microphones M₀ . . . M_M;

determining a listening direction of the microphone array with a digital signal processing system having a digital processor by

forming analysis frames of a pre-recorded a signal from a source located in a preferred known listening direction with respect to the microphone array for a predetermined period of time at predetermined intervals using the processor,

transforming the analysis frames into the frequency domain using the processor,

estimating a calibration covariance matrix from vectors formed from the analysis frames that have been transformed into the frequency domain using the processor,

computing an eigenmatrix of the calibration covariance matrix using the processor, and

computing an inverse of the eigenmatrix using the processor applying one or more fractional delays to one or more of the time domain input signals x_m(t) other than an input signal x₀(t) from a reference microphone M₀ using the processor, wherein each fractional delay is selected to optimize a signal to noise ratio of an output signal from the microphone array and wherein the fractional delays are selected to such that a signal from the reference microphone M₀ is first in time relative to signals from the other microphone(s) of the array.

28. The method of claim 27 wherein the fractional delay is greater than a minimum delay, wherein the minimum delay is long enough to capture reverberation from the signal.

29. The method of claim 27 wherein the two or more microphones M₀ . . . M_M are omni-directional microphones.