Sound source separation apparatus and sound source separation method

• US 20090012779A1
• Filed: 03/04/2008
• Published: 01/08/2009
• Est. Priority Date: 03/05/2007
• Status: Abandoned Application

• Current Assignee: Klaus Schmidt, Nara Institute of Science and Technology, National University Corporation
• Original Assignee: Klaus Schmidt, Nara Institute of Science and Technology, National University Corporation
• Inventors: Hiekata, Takashi, Morita, Takashi, Ikeda, Yohei, Mori, Yoshimitsu, Saruwatari, Hiroshi
• Application Number:
  US12/073,336
• Publication Number:
  US 20090012779A1
• Time in Patent Office: Days
• US Class Current: 704/205
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: H04R 1/403 loud-speakers

Abstract

A sound source separation apparatus includes: an SIMO-ICA process unit, separating and generating an SIMO signal by the BSS method based on the ICA method; a sound source direction estimation unit, estimating a sound source direction based on a separating matrix, computed by a learning calculation of the BSS method based on the ICA method; a beamformer process unit, performing, on each SIMO signal, a beamformer process of enhancing, according to each frequency bin, a sound component from each sound source direction; an intermediate process unit, performing an intermediate process that includes performing a selection process, etc., according to each frequency bin on signals other than a specific signal among the beamformer processed sound signals; and an untargeted signal component elimination unit, eliminating noise signal components by comparing for one signal in the specific SIMO signal, volumes of the specific beam former processed sound signal and the intermediate processed signal according to each frequency bin.

Claims

1. A sound source separation apparatus, comprising:

a plurality of sound input means, into which a plurality of mixed sound signals in which sound source signals from a plurality of sound sources are superimposed are inputted;

an SIMO-ICA process means, separating and generating SIMO signals each of which corresponds to at least one of the sound source signals from the plurality of mixed sound signals by a sound source separation process of a blind source separation method based on an independent component analysis method;

a sound source direction estimation means, estimating sound source directions which are directions in which the sound sources are present, respectively, based on a separating matrix calculated by a learning calculation executed in the sound source separation process of the blind source separation method based on the independent component analysis method in the SIMO-ICA process means;

a beamformer process means,

applying, to each of the SIMO signals separated and generated in the SIMO-ICA process means, a beamformer process of enhancing, according to each of plurally sectioned frequency components, a sound component from each of the sound source directions estimated by the sound source estimation means, and

outputting beamformer processed sound signals;

an intermediate process execution means,

performing a predetermined intermediate process including a selection process or a synthesis process, according to each of the plurally sectioned frequency components, on the beamformer processed sound signals other than a specific beamformer processed sound signal with which a sound component from a specific sound source direction which is one of the sound source directions is enhanced for a specific SIMO-signal which is one of the SIMO signals, and

outputting an intermediate processed signal obtained thereby; and

an untargeted signal component elimination means,

performing, on one signal in the specific SIMO signal, a process of comparing volumes of the specific beamformer processed sound signal and the intermediate processed signal according to each of the plurally sectioned frequency components and, when a comparison result meets a predetermined condition, of eliminating a signal of the corresponding frequency component, and

generating a signal obtained thereby as a separated signal corresponding to one of the sound source signals.

2. The sound source separation apparatus according to claim 1, wherein the sound source separation process of the blind source separation method based on the independent component analysis method in the SIMO-ICA process means includes a sound source separation process of a blind source separation method based on a frequency domain SIMO independent component analysis method, and

wherein the SIMO-ICA process means comprises:

a short time discrete Fourier transform means, applying a short time discrete Fourier transform process to the plurality of mixed sound signals in a time domain, and converting the mixed sound signals into a plurality of mixed sound signals in a frequency domain;

an FDICA sound source separation process means, applying a separation process based on a predetermined separating matrix on the plurality of mixed sound signals in the frequency domain to generate first separated signals each of which corresponds to one of the sound source signals, according to each mixed sound signal;

a subtraction means, generating second separated signals by subtracting, from each of the plurality of mixed sound signals in the frequency domain, the first separated signals generated by the FDICA sound source separation process means based on the corresponding mixed sound signal; and

a separating matrix calculation means, calculating the separating matrix in the FDICA sound source separation process means by a successive calculation based on the first separated signals and the second separated signals.

3. The sound source separation apparatus according to claim 1, wherein

the sound source separation process of the blind source separation method based on the independent component analysis method in the SIMO-ICA process means includes a sound source separation process of a blind source separation method based on a combination of a frequency domain independent component analysis method and a projection back method.

4. The sound source separation apparatus according to claim 1, wherein

the beamformer process performed by the beamformer process means includes a delay and sum beamformer process or a blind angle beamformer process.

5. The sound source separation apparatus according to claim 1, wherein

the intermediate process execution means corrects the beamformer processed sound signals by a predetermined weighting of signal levels according to the plurally sectioned frequency components, and performs the selection process or the synthesis process on the corrected signals according to each frequency component.

6. The sound source separation apparatus according to claim 5, wherein

the intermediate process execution means performs a process of selecting, from among the corrected signals, a signal having the highest signal level according to each frequency component.

7. The sound source separation apparatus according to claim 1, further comprising:

an intermediate process parameter setting means, setting, in accordance with a predetermined operation input, a parameter used in the intermediate process in the intermediate process setting means.

8. A sound source separation method comprising:

a plurality of sound input steps of inputting a plurality of mixed sound signals in which sound source signals from a plurality of sound sources are superimposed;

an SIMO-ICA process step of separating and generating SIMO signals each of which corresponds to at least one of the sound source signals from the plurality of mixed sound signals by a sound source separation process of a blind source separation method based on an independent component analysis method;

a sound source direction estimating step of estimating sound source directions which are directions in which the sound sources are present, respectively, based on a separating matrix calculated by a learning calculation executed in the sound source separation process of the blind source separation method based on the independent component analysis method in the SIMO-ICA process step;

a beamformer process step of

applying, to each of the SIMO signals separated and generated in the SIMO-ICA process step, a beamformer process of enhancing, according to each of plurally sectioned frequency components, a sound component from each of the sound source directions estimated by the sound source estimation step, and

outputting beamformer processed sound signals;

an intermediate process execution step of

performing a predetermined intermediate process including a selection process or a synthesis process, according to each of the plurally sectioned frequency components, on the beamformer processed sound signals other than a specific beamformer processed sound signal with which a sound component from a specific sound source direction, which is one of the sound source directions is enhanced for a specific SIMO signal which is one of the SIMO signals, and

outputting an intermediate processed signal obtained thereby; and

an untargeted signal component elimination step of

performing, on one signal in the specific SIMO signal, a process of comparing volumes of the specific beamformer processed sound signal and the intermediate processed signal according to each of the plurally sectioned frequency components and, when a comparison result meets a predetermined condition, of eliminating a signal of the corresponding frequency component, and

generating a signal obtained thereby as a separated signal corresponding to one of the sound source signals.

9. The sound source separation method according to claim 8, wherein the sound source separation process of the blind source separation method based on the independent component analysis method in the SIMO-ICA process step includes a sound source separation process of a blind source separation method based on a frequency domain SIMO independent component analysis method and

wherein the SIMO-ICA process step comprises:

a short time discrete Fourier transform step of applying a short time discrete Fourier transform process to the plurality of mixed sound signals in a time domain, and converting the mixed sound signals into a plurality of mixed sound signals in a frequency domain;

an FDICA sound source separation process step of applying a separation process based on a predetermined separating matrix on the plurality of mixed sound signals in the frequency domain to generate first separated signals each of which corresponds to one of the sound source signals, according to each mixed sound signal;

a subtraction step of generating second separated signals by subtracting, from each of the plurality of mixed sound signals in the frequency domain, the first separated signals generated by the FDICA sound source separation process step based on the corresponding mixed sound signal; and

a separating matrix calculation step of calculating the separating matrix in the FDICA sound source separation process step by a successive calculation based on the first separated signals and the second separated signals.

10. The sound source separation method according to claim 8, wherein

the sound source separation process of the blind source separation method based on the independent component analysis method in the SIMO-ICA process step includes a sound source separation process of a blind source separation method based on a combination of a frequency domain independent component analysis method and a projection back method.

11. The sound source separation method according to claim 8, wherein

the beamformer process performed in the beamformer process step includes a delay and sum beamformer process or a blind angle beamformer process.

12. The sound source separation method according to claim 8, wherein

in the intermediate process execution step, the beamformer processed sound signals are corrected by a predetermined weighting of signal levels according to the plurally sectioned frequency components, and the selection process or the synthesis process is performed on the corrected signals according to each frequency component.

13. The sound source separation method according to claim 12, wherein

in the intermediate process execution step, a process of selecting, from among the corrected signals, a signal of the highest signal level according to each frequency component, is performed.

14. The sound source separation method according to claim 8, further comprising:

an intermediate process parameter setting step of setting, in accordance with a predetermined operation input, a parameter used in the intermediate process in the intermediate process setting step.