Method and system for dereverberation of signals propagating in reverberative environments

• US 8,988,970 B2
• Filed: 03/14/2011
• Issued: 03/24/2015
• Est. Priority Date: 03/12/2010
• Status: Active Grant

• Current Assignee: University of Maryland
• Original Assignee: University of Maryland
• Inventors: Duraiswami, Ramani, Zotkin, Dmitry N., O'Donovan, Adam
• Primary Examiner(s): Ratcliffe, Luke
• Assistant Examiner(s): Baghdasaryan, Hovhannes
• Application Number:
  US13/047,311
• Publication Number:
  US 20110222372A1
• Time in Patent Office: 1,471 Days
• Field of Search: 367 7- 11, 367/47, 367/103
• US Class Current: 367/103
• CPC Class Codes: G01S 3/8083 determining direction of source
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 25/24 the extracted parameters being the cepstrum
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/401 2D or 3D arrays of transducers
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

The dereverberation of signals in reverberating environments is carried out via acquiring the representation (image) of spatial distribution of the signals in space of interest and automatic identification of reflections of the source signal in the reverberative space. The technique relies on identification of prominent features at the image, as well as corresponding directions of propagation of signals manifested by the prominent features at the image, and computation of similarity metric between signals corresponding to the prominent features in the image. The time delays between the correlated signals (i.e., source signal and related reflections) are found and the signals are added coherently. Multiple beamformers operate on the source signal and corresponding reflections, enabling one to improve the signal-to-noise ratio in multi-path environments.

Claims

1. A method for processing of spatially distributed signals, comprising the steps of:

(a) providing a sensor unit and a computational unit operatively coupled to said sensor unit for a dynamical processing of a plurality of signals registered at said sensor unit, wherein each of said plurality of signals received at said computational unit includes embedded therein spatial information corresponding to directions of propagation of said plurality of signals; and

(b) dynamically processing said plurality of signals by said computational unit through the steps of;

(c) dynamically computing, by said computational unit, a real-time dynamical spatial distribution of said plurality of signals in a dynamically changing space of interest based substantially on azimuth and elevation information contained in said spatial information embedded in said each signal arriving at said sensor unit, and

(d) dynamically finding at least a first and at least a second signals originating at said at least one signal source by dynamically calculating, substantially in real-time, a similarity measure for said plurality of signals presented in said real-time dynamical spatial distribution based on said azimuth and elevation information, and attributing said at least first signal and said at least second signal from said plurality thereof presented in said real-time dynamical spatial distribution to said at least one signal source of interest based on said real-time similarity measure for dynamically combining said at least one first and second signals into an output deverberated signal with desired characteristics, said output dereverberated signal being correlated to said at least one signal source of interest.

2. The method of claim 1, further comprising the steps of:

in said step (c), beamforming, by said computational unit, said plurality of signals to create a representation of said spatial distribution of said plurality of signals co-registered with said space of interest, and in said step (d), identifying, at said spatial distribution representation, a plurality of prominent features, and respective propagation directions of said plurality of signals corresponding to said features.

3. The method of claim 2, further comprising the steps of:

in said step (d), identifying at least one first prominent feature from said plurality thereof corresponding to said at least one first signal arrived directly to said sensor unit from said at least one source of interest, and

defining said at least one second signal as a reflection signal of said at least one source of interest in said space of interest, if a predetermined value of said similarity measure is reached.

4. The method of claim 3, after said step (d), further comprising the step of:

(e) beamforming said plurality of signals corresponding to said prominent features for said respective directions thereof.

5. The method of claim 4, after said step (d), further comprising the steps of:

(f) estimating time delays between said at least first signal and said reflection signal, and

(g) summing said beamformed reflection signal coherently to said beamformed at least one first signal, thereby forming said output dereverberated signal.

6. The method of claim 2, therein said signals are of acoustic nature, wherein said representation of the spatial distribution is an audio image, and wherein said sensor unit is an audio camera including an array of microphones arranged on a spherically contoured surface.

7. The method of claim 3, further comprising the steps of:

in said step (c), obtaining a real-time spatial distribution representation containing a plurality N of said first prominent features and a plurality of second prominent features,

in said step (d), specifying a target signal source among a plurality N of signal sources, and

applying the similarity measure to said first signal corresponding to said specified target signal source and said plurality of second signals,

wherein said plurality of second prominent features manifest at least one feature selected from a group consisting of reflections of at least one of said plurality N of signal sources, direct path signals from N−

1 of said signal sources other than said target signal source, and reflections thereof.

8. The method of claim 4, further comprising the steps of:

creating time domain signals for said beamformed at least one directly arrived signal and said beamformed at least one reflection signal.

9. The method of claim 1,

wherein in said step (d), said similarity measure includes feature vectors computed by using Mel-frequency cepstral coefficients (MFCC).

10. The method of claim 1, further comprising the step of:

in said step (d), computing said similarity measure by building vector quantization code book for said at least one first signal, and encoding feature vectors of said at least one second signal with said code book.

11. The method of claim 1, further comprising the step of:

in said step (d), computing said similarity measure by using Gaussian mixture model fitting procedure or support vector machines.

12. The method of claim 3, further comprising the steps of:

after said step (d), grouping said plurality of prominent features found in said spatial distribution representation in accordance with said similarity measure thereof, and

applying time delay extraction procedure, thereby estimating the time delays between signals belonging to the same group.

13. The method of claim 3, further comprising the steps of:

in said step (c), beamforming said signals by operating a plurality of independent beamforms in parallel.

14. The method of claim 5, wherein said steps (b)-(g) are performed in real-time.

15. The method of claim 5, further comprising the steps of:

in said step (c), computing said spatial distribution via a number of parallel beamforming procedures, and storing spatial distribution data in a multi-dimensional database A;

in said step (d), finding, among said data stored in said database A, cells having properties exceeding properties of surrounding cells,

forming a list L1 of spatial locations S associated with said cells,

labeling at least one T of said spatial locations S in said list L1 as a target source of interest, and processing said spatial distribution representation to find N1 signal arrived from directions corresponding to said locations S recorded in said list L1,

forming a list X of said N1 signals,

for each said location S recorded in said list L1, excluding T, computing said similarity measure between X(T) and X(S), and

creating a list L2 containing said locations S from said list L1 for which said similarity measure between X(T) and X(S) exceeds a predetermined threshold.

16. The method of claim 15, further comprising the steps of:

for each location S in said list L2, computing N2 time delays D(S) between corresponding X(T) and X(S),

storing said N2 time delays in list D,

creating a signal Y corresponding to X(T),

for each location S from said list L2, delaying the signals X(S) by −

D(S), and

adding said delayed signal X(S) to said signal Y.

17. A system for processing spatially distributed signals in reverberative environment, comprising:

a sensor unit positioned in a space of interest for detection of a plurality of signals arrived thereat,

a first Beamforming Processor coupled to said sensor unit and configured to receive a plurality of spatially tagged signals from said sensor unit and to form a real-time dynamical image of said plurality of real-time spatially tagged signals spatially mapped to said space of interest, wherein said real time dynamical image is based on azimuth and elevation information embedded in said plurality of signals propagating in said space of interest, and

a computational unit operationally coupled to said real-time image of said plurality of spatially tagged signals and configured to dynamically calculate an output dereverberation signal with desired characteristics,

wherein said computational unit includes;

a Peaks identification Unit configured for dynamical processing of said real-time dynamical image of said plurality of spatially tagged real-time signals spatially mapped to said space of interest and for dynamical identification of at least a first peak and second peaks at said real-time image based on said azimuth and elevation information, wherein said at least first peak corresponds to a first signal arrived directly from at least one source of interest, and wherein said second peaks correspond to said signals including reflection signals,

a second Beamforming Processor for applying beamforming procedure to said first and second signals propagating in directions corresponding to said at least first peak and said second peaks, respectively,

a Similarity Measure Evaluation Unit coupled to said second Beamforming Processor and configured to calculate, substantially in real-time, a similarity measure between said real-time first and real-time second signals identified in said real-time image based on said azimuth and elevation information, and dynamically defining at least one of said second signals as a reflection signal of said at least one source of interest if the real-time similarity measure between said real-time first and said at least one second signals reaches a predetermined threshold, and

a Dereverberation Signal Output Unit coupled to said Similarity Measure Evaluation Unit, said Dereverberation Signal Output Unit including;

a Time-Delay Unit configured to estimate time delays between said at least first signal and said reflection signals, and

a Summing Unit coupled to said Time Delay Unit and configured to dynamically and coherently add said reflection signals to said first signal with an appropriate time delay.

18. The system of claim 17, where said sensor unit includes an audio camera comprising a plurality of microphones arranged on a spherically contoured surface.

19. The system of claim 17, wherein said Similarity Measure Evaluation Unit further includes a Grouping Unit for grouping said peaks in said real-time image in accordance with the similarity measure thereof.