Speech separation with microphone arrays

• US 8,144,896 B2
• Filed: 02/22/2008
• Issued: 03/27/2012
• Est. Priority Date: 02/22/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Dmochowski, Jacek, Liu, Zicheng, Chou, Philip Andrew
• Primary Examiner(s): HA, NATHAN W
• Application Number:
  US12/035,439
• Publication Number:
  US 20090214052A1
• Time in Patent Office: 1,495 Days
• Field of Search: 381 92- 94, 381/94.3
• US Class Current: 381/94.3
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F

Abstract

A system that facilitates blind source separation in a distributed microphone meeting environment for improved teleconferencing. Input sensor (e.g., microphone) signals are transformed to the frequency-domain and independent component analysis is applied to compute estimates of frequency-domain processing matrices. Modified permutations of the processing matrices are obtained based upon a maximum magnitude based de-permutation scheme. Estimates of the plurality of source signals are provided based upon the modified frequency-domain processing matrices and input sensor signals.

Optionally, segments during which the set of active sources is a subset of the set of all sources can be exploited to compute more accurate estimates of frequency-domain mixing matrices. Source activity detection can be applied to determine which speaker(s), if any, are active. Thereafter, a least squares post-processing of the frequency-domain independent components analysis outputs can be employed to adjust the estimates of the source signals based on source inactivity.

Claims

1. A computer-implemented audio blind source separation system, comprising:

a frequency transform component for transforming a plurality of sensor signals to a corresponding plurality of frequency domain sensor signals, the plurality of sensor signals received from a plurality of input sensors; and

,

a frequency domain blind source separation component for estimating a plurality of source signals for each of a plurality of frequency bands based on the plurality of frequency domain sensor signals and processing matrices computed independently for each of the plurality of frequency bands; and

a maximum attenuation based de-permutation component for obtaining modified permutations of the processing matrices based upon a maximum-magnitude based de-permutation scheme,

wherein the system provides estimates of the plurality of source signals based on the plurality of frequency domain sensor signals and the modified permutations of the processing matrices.

2. The system of claim 1, wherein the frequency domain blind source separation component further employs independent component analysis to compute the processing matrices.

3. The system of claim 1, wherein the processing matrices comprise mixing matrices.

4. The system of claim 1, wherein the processing matrices comprise separation matrices.

5. The system of claim 1, wherein the system further employs source activity detection.

6. The system of claim 5, wherein the system further modifies the processing matrices based upon the source activity detection and a least squares estimation of the plurality of source signals.

7. The system of claim 6, wherein the system modifies the processing matrices more than once based upon the source activity detection and the least squares estimation of the plurality of source signals.

8. The system of claim 1, wherein the frequency transform component employs a short-time Fourier transform for transforming the plurality of sensor signals to the corresponding plurality of frequency domain sensor signals.

9. The system of claim 1, wherein a quantity of sources is less than or equal to a quantity of input sensors.

10. The system of claim 1, wherein at least one of the plurality of input sensors is an embedded microphone.

11. A computer-implemented method of blindly separating a plurality of source signals, comprising:

receiving a plurality of input sensor signals;

transforming the input sensor signals to a corresponding plurality of frequency-domain sensor signals using a short-time Fourier transform; and

computing estimates of the plurality of source signals for each of a plurality of frequency bands based upon the plurality of frequency-domain sensor signals and processing matrices computed independently for each of the plurality of frequency bands; and

obtaining modified permutations of the processing matrices based upon a maximum magnitude based de-permutation scheme.

12. The method of claim 11, wherein the processing matrices comprise separation matrices.

13. The method of claim 11, wherein the processing matrices comprise mixing matrices.

14. The method of claim 11, further comprising providing estimates of the plurality of source signals based on the plurality of frequency domain sensor signals and the modified permutations of the processing matrices.

15. A computer-implemented method of blindly separating a plurality of source signals, comprising:

determining source activity information specifying which two or more sources are active at a plurality of times; and

,

modifying processing matrices based upon a least squares estimation of the processing matrices and the source activity information.

16. The method of claim 15, further comprising providing an estimate the source signals based upon the modified processing matrices.

17. The method of claim 15, wherein the processing matrices comprise separation matrices.

18. The method of claim 15, wherein the processing matrices comprise mixing matrices.

19. The method of claim 15, wherein modifying the processing matrices based on source activity information is performed more than once.

20. The method of claim 15, wherein the processing matrices are received from an audio blind source separation system.