Sound source localization method and sound source localization apparatus based coherence-to-diffuseness ratio mask

• US 10,593,344 B2
• Filed: 01/23/2019
• Issued: 03/17/2020
• Est. Priority Date: 01/25/2018
• Status: Active Grant

• Current Assignee: Sogang University Research Foundation
• Original Assignee: Sogang University Research Foundation
• Inventors: Park, Hyung Min, Lee, Ran
• Primary Examiner(s): Matar, Ahmad F.
• Assistant Examiner(s): Diaz, Sabrina
• Application Number:
  US16/255,112
• Publication Number:
  US 20190228790A1
• Time in Patent Office: 419 Days
• Field of Search: 381 731
• CPC Class Codes: G01S 3/8006 Multi-channel systems specially adapted for direction-finding, i.e. having a single aerial system capable of giving simultaneous indications of the directions of different signals
• CPC Class Codes: G01S 3/8083 determining direction of source
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/04 for correcting frequency response
• CPC Class Codes: H04R 5/027 Spatial or constructional arrangements of microphones, e.g. in dummy heads
• CPC Class Codes: H04R 5/04 Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

Provided is a sound source localization method including steps of: (a) receiving a mixed signal of a target sound source signal and noise and echo signals through multiple microphones including at least two microphones; (b) generating a binarized mask based on a diffuseness by using a coherence-to-diffuseness ratio CDR, which is information on the target sound source and the noise source, by using the input signal; (c) pre-processing an input signal to multiple microphones by using the generated binarized mask; and (d) performing a predetermined algorithm such as the GCC-PHAT or the SRP-PHAT on the pre-processed input signal to estimate a direction of the target sound source.

Claims

1. A sound source localization method implemented by execution of a processor of a sound source localization apparatus, comprising steps of:

(a) receiving a mixed signal of a target sound source signal and a noise signal through multiple microphones including at least two microphones;

(b) generating a mask based on a diffuseness reflecting information on a target sound source and a noise source by using the mixed signal;

(c) pre-processing the mixed signal received to the multiple microphones by using the generated mask; and

(d) estimating a direction for the target sound source by performing a predetermined algorithm on the pre-processed mixed signal.

2. The sound source localization method according to claim 1, wherein, in the step (b) of generating the mask,

a coherence-to-diffuseness ratio CDR(l,f) for each frequency frame f and each time frame l is calculated,

a diffuseness D(l,f) is calculated by using the coherence-to-diffuseness ratio CDR(l,f), and

a binarized mask M is generated by setting a mask value according to the following Mathematical Formula by using the diffuseness D(l,f),

3. The sound source localization method according to claim 2, wherein, in the step (c) of pre-processing the mixed signal, the mixed signal is binarized by using a binarized mask.

4. The sound source localization method according to claim 1, wherein the predetermined algorithm in the step (d) is a sound source localization method based on generalized cross correlation (GCC) value or a sound source localization method based on a steered response power SRP.

5. The sound source localization method according to claim 4, wherein the predetermined algorithm applies a phase transform (PHAT) scheme for applying a weighting factor ψ

(ω

) according to the following Mathematical Formula to signals of each frequency,

6. The sound source localization method according to claim 2, wherein the coherence-to-diffuseness ratio CDR(l,f) for each frequency frame f and each time frame l is estimated according to the following Mathematical Formula by using a coherence for the noise signal ‘

n’

, the target sound source signal ‘

s’

, and the mixed signal ‘

x’

of the noise signal and the target sound signal,

7. The sound source localization method according to claim 2, wherein the diffuseness D(l,f) is calculated according to the following Mathematical Formula,

8. A sound source localization apparatus having a processor and being operable to estimate a direction of a target sound source by using signals input from multiple microphones by execution of the processor, comprising:

a mixed signal input module which is connected to the multiple microphones and receives mixed signals of a target sound source signal and a noise signal from the multiple microphones;

a mask generation module which generates and outputs a binarized mask based on a diffuseness by using the mixed signal provided from the mixed signal input module;

an input signal pre-processing module which receives the binarized mask from the mask generation module, pre-processes the mixed signal by applying the binarized mask to the mixed signal provided from the mixed signal input module, and outputs the pre-processed mixed signal; and

a target direction estimation module which receives the pre-processed mixed signal from the input signal pre-processing module, estimates a direction of the target sound source by performing a predetermined localization algorithm on the pre-processed mixed signal, and outputs the estimated direction.

9. The sound source localization apparatus according to claim 8, wherein the mask generation module performs:

calculating a coherence-to-diffuseness ratio CDR(l,f) for each frequency frame f and each time frame l of the mixed signal provided from the mixed signal input module;

calculating a diffuseness D(l,f) by using the coherence-to-diffuseness ratio CDR(l,f); and

generating a binarized mask M by setting a mask value according to the following Mathematical Formula by using the diffuseness D(l,f),

10. The sound source localization apparatus according to claim 8, wherein the predetermined localization algorithm of the target direction estimation module is a sound source localization method based on a generalized cross correlation (GCC) value or a sound source localization method based on a steered response power SRP.

11. The sound source localization apparatus according to claim 9, wherein the coherence-to-diffuseness ratio CDR(l,f) for each frequency frame f and each time frame l is estimated according to the following Mathematical Formula by using a coherence for the noise signal ‘

n’

, the target sound source signal ‘

s’

, and the mixed signal ‘

x’

of the noise signal ‘

n’ and

the target sound signal,