Blind source separation systems

• US 9,668,066 B1
• Filed: 06/22/2015
• Issued: 05/30/2017
• Est. Priority Date: 04/03/2015
• Status: Active Grant

• Current Assignee: AudioTelligence Limited
• Original Assignee: Cedar Audio Limited
• Inventors: Betts, David Anthony, Dmour, Mohammad A.
• Primary Examiner(s): He, Jialong
• Application Number:
  US14/746,262
• Time in Patent Office: 708 Days
• Field of Search: None
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2225/43 Signal processing in hearing aids to enhance the speech intelligibility
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 25/40 Arrangements for obtaining a desired directivity characteristic
• CPC Class Codes: H04R 25/405 by combining a plurality of transducers
• CPC Class Codes: H04R 25/554 using a wireless connection, e.g. between microphone and amplifier or using Tcoils
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

We describe a method of blind source separation for use, for example, in a listening or hearing aid. The method processes input data from multiple microphones each receiving a mixed signal from multiple audio sources, performing independent component analysis (ICA) on the data in the time-frequency domain based on an estimation of a spectrogram of each acoustic source. The spectrograms of the sources are determined from non-negative matrix factorization (NMF) models of each source, the NMF model representing time-frequency variations in the output of an acoustic source in the time-frequency domain. The NMF and ICA models are jointly optimized, thus automatically resolving an inter-frequency permutation ambiguity.

Claims

1. A method of processing acoustic data representing audio from a plurality of different acoustic sources mixed together to extract the audio from an individual one of the acoustic sources so that it can be listened to separately, the method comprising performing blind source separation by:

inputting acoustic data from a plurality of acoustic sensors, said acoustic data comprising acoustic signals combined from said plurality of acoustic sources;

converting said input acoustic data to combined source time-frequency domain data representing said acoustic signals combined from said plurality of acoustic sources, wherein said time-frequency domain data is represented by an observation matrix X_ƒ

for each of a plurality of frequencies ƒ

;

performing an independent component analysis (ICA) on said observation matrix X_ƒ

to determine a demixing matrix W_ƒ

for each said frequency such that an estimate Y_ƒ

of the acoustic signals from said plurality of acoustic sources at said frequencies ƒ

is determined by X_ƒ

W_ƒ

;

wherein said ICA is performed based on an estimation of an individual source spectrogram of each individual said acoustic source; and

wherein said estimation of said individual source spectrogram of each individual said acoustic source is determined from a model of said individual acoustic source, the model representing individual source time-frequency variations in a signal output of said individual acoustic source;

using said demixing matrix W_ƒ

to process said acoustic data comprising acoustic signals combined from said plurality of acoustic sources and demix individual acoustic data for an individual one of said plurality of acoustic sources; and

providing the acoustic data for the individual one of said plurality of acoustic sources to an output device for transmission to a user.

2. A method as claimed in claim 1 comprising iteratively improving said ICA and said model by performing said ICA to estimate said acoustic signals from said plurality of acoustic sources, then updating said model using said estimated acoustic signals to provide an updated estimation of said individual source spectrogram of each individual said acoustic source, then updating said ICA using said updated estimations of said individual source spectrograms.

3. A method as claimed in claim 2 wherein updating said ICA comprises determining a permutation of elements of said demixing matrix W_ƒ

over said acoustic sources prior to determining said updated estimations of said individual source spectrograms for said plurality of acoustic sources.

4. A method as claimed in claim 2 wherein said updating of said ICA comprises adjusting said demixing matrix W_ƒ

by a value dependent upon a gradient of said demixing matrix, wherein said gradient of said demixing matrix is dependent upon both said estimate Y_ƒ

of said acoustic signals from said plurality of acoustic sources and said estimation of said individual source spectrogram of each individual said acoustic source.

5. A method as claimed in claim 1 wherein said model for each acoustic source comprises a time-frequency dependent non-negative matrix factorisation (NMF) model.

6. A method as claimed in claim 5 wherein said NMF model comprises, for each of said plurality of acoustic sources, a spectral dictionary and set of dictionary activations;

and wherein the method further comprises updating said spectral dictionary and said set of dictionary activations for the acoustic sources responsive to said estimate of the acoustic signals from the sources (Y_ƒ

).

7. A method as claimed in claim 6 wherein said spectral dictionary and said set of dictionary activations are jointly optimised with the demixing matrix W_ƒ

for each said frequency.

8. A method as claimed in claim 7 wherein said joint optimisation comprises performing, jointly, the following operations:

Y_ƒ

←

X_ƒ

W_ƒ

for all ƒ

after updating W_ƒ

; and

σ

_k<sup>•

λ</sup>

←

V_k^TU_k for all k after updating U or V

where ←

denotes updating, U_k and V_k denote dictionaries and activations of said NMF model for each of said acoustic sources k, σ

_k denotes said estimation of the spectrogram of acoustic source k, and λ

is a parameter greater than zero.

9. A method as claimed in claim 8 wherein λ

=1.

11. A method as claimed in claim 1 further comprising compensating for a scaling ambiguity in W_ƒ

using said individual acoustic data as predicted to be received at one or more of said acoustic sensors.

12. A method as claimed in claim 1 wherein said converting of said acoustic data to the time-frequency domain is performed blockwise for successive blocks of time series acoustic data, the method further comprising ensuring that said individual acoustic data for an individual one of said plurality of acoustic sources represents the same individual one of said plurality of acoustic sources from one of said blocks to a next of said blocks to at least partially remove a source permutation ambiguity.

13. A method as claimed in claim 1 comprising using said demixing matrix W_ƒ

in a time domain to process said acoustic data comprising acoustic signals combined from a plurality of acoustic sources and demix individual acoustic data for an individual one of said plurality of acoustic sources.

14. A non-transitory data carrier carrying processor control code to, when running, implement the method of claim 1.

10. A method as claimed in 1 further comprising pre-processing said acoustic data to reduce a number of said acoustic signals from said plurality of acoustic sensors to a reduced number of acoustic signals which is less than a number of said acoustic sensors, wherein said reduced number of acoustic signals is equal to a number of said plurality of said acoustic sources.

15. A method of processing acoustic data representing audio from a plurality of different acoustic sources mixed together to extract the audio from an individual one of the acoustic sources so that it can be listened to separately, the method comprising performing blind source separation by:

capturing the acoustic data representing audio from the plurality of acoustic sources at a plurality of microphones;

processing the captured acoustic data to provide a set of observation matrices, said set of observation matrices representing observations of acoustic signals combined from said plurality of acoustic sources, wherein said set of observation matrices comprises a plurality of observation matrices, wherein each observation matrix is denoted X_ƒ

and comprises data in a time-frequency domain for one of a plurality of frequencies ƒ

;

wherein acoustic data for one of said plurality of acoustic sources and at one of said plurality of frequencies, demixed from said acoustic signals combined from said plurality of acoustic sources, is denoted Y_ƒ

, where Y_ƒ

comprises data in said time-frequency domain, and

processing said set of observation matrices using a demixing matrix W_ƒ

for each of said plurality of frequencies to determine an estimate of said acoustic data, denoted Y_ƒ

, demixed from said acoustic signals combined from said plurality of acoustic sources;

wherein said processing comprises iteratively updating Y_ƒ

from X_ƒ

W_ƒ

; and

wherein said processing is performed based on a probability distribution p(Y_tkf;

σ

_tkf) for Y dependent upon

16. A method as claimed in claim 15 wherein said iterative updating comprises updating W_ƒ

given U_lfk and V_ltk, updating U_lfk given V_ltk and W_ƒ

, and updating V_ltk given W_ƒ

and U_lfk.

17. A method as claimed in claim 16 wherein said updating of W_ƒ

includes determining one or both of a permuted version of W_ƒ

and a scaled version of W_ƒ

.

18. Apparatus to improve audibility of an audio signal by blind source separation, the apparatus comprising:

a set of microphones, each of the set of microphones having a known geometry, to receive signals from a plurality of audio sources disposed around the microphones; and

an audio signal processor coupled to said microphones, and configured to providing a demixed audio signal output;

the audio signal processor comprising;

at least one analog-to-digital converter to digitise said signals received by said microphones to provide digital time-domain signals; and

a digital filter to filter said digital time-domain signals in the time domain in accordance with a set of filter coefficients to provide said demixed audio signal output;

the audio signal processor further comprising;

a time-to-frequency domain converter to divide said digital time-domain signals into time segments and to convert said digital time-domain signals in said time segments into the frequency domain to generate time-frequency domain data;

a blind source separation module, to perform audio signal demixing on said time-frequency domain data to determine a demixing matrix for at least one of said audio sources, wherein said set of filter coefficients is determined by said demixing matrix and is determined asynchronously in said time-frequency domain; and

wherein said audio signal processor is further configured to;

process said demixing matrix, in view of a frequency and phase response of each microphone, determined from the known geometry of the microphone, to select one or more said audio sources responsive to a phase correlation determined from said demixing matrix.

19. Apparatus as claimed in claim 18 wherein said audio signal processor is further configured to reduce a number of audio channels from said microphones prior to said audio signal demixing, and to resolve a scaling ambiguity in said demixing matrix.

20. Apparatus as claimed in claim 19 wherein said blind source separation module is configured to perform joint independent component analysis (ICA) and non-negative matrix factorisation (NMF) to perform said audio signal demixing.