Discrimination of components of audio signals based on multiscale spectro-temporal modulations

• US 20060025989A1
• Filed: 07/28/2005
• Published: 02/02/2006
• Est. Priority Date: 07/28/2004
• Status: Active Grant

• Current Assignee: University of Maryland
• Original Assignee: University of Maryland
• Inventors: Shamma, Shihab A., Mesgarani, Nima
• Granted Patent: US 7,505,902 B2
• Time in Patent Office: Days
• US Class Current: 704/200
• CPC Class Codes: G10L 21/0272 Voice signal separating

Abstract

An audio signal (172) representative of an acoustic signal is provided to an auditory model (105). The auditory model (105) produces a high-dimensional feature set based on physiological responses, as simulated by the auditory model (105), to the acoustic signal. A multidimensional analyzer (106) orthogonalizes and truncates the feature set based on contributions by components of the orthogonal set to a cortical representation of the acoustic signal. The truncated feature set is then provided to classifier (108), where a predetermined sound is discriminated from the acoustic signal.

Claims

1. A method for discriminating sounds in an audio signal comprising the steps of:

forming an auditory spectrogram from the audio signal, said auditory spectrogram characterizing a physiological response to sound represented by the audio signal;

filtering said auditory spectrogram into a plurality of multidimensional cortical response signals, each of said cortical response signals indicative of frequency modulation of said auditory spectrogram over a corresponding predetermined range of scales and of temporal modulation of said auditory spectrogram over a corresponding predetermined range of rates;

decomposing said cortical response signals into orthogonal multidimensional component signals;

truncating said orthogonal multidimensional component signals; and

classifying said truncated component signals to discriminate therefrom a signal corresponding to a predetermined sound.

2. The method for discriminating sounds in an audio signal as recited in claim 1, where said filtering step includes the step of convolving in both time and frequency said auditory spectrogram with each of a plurality of spectro-temporal response fields.

3. The method for discriminating sounds in an audio signal as recited in claim 2, where said filtering step further includes the step of providing a corresponding wavelet as said each spectro-temporal response fields.

4. The method for discriminating sounds in an audio signal as recited in claim 1 further including the step of averaging with respect to time over a predetermined number of time increments said cortical response signals prior to said decomposing step.

5. The method for discriminating sounds in an audio signal as recited in claim 4, where said decomposing step includes the step of decomposing said cortical response signals into orthogonal scale, rate and frequency components.

6. The method for discriminating sounds in an audio signal as recited in claim 1 further including the steps of:

forming a training auditory spectrogram from a known audio signal, said known audio signal associated with a corresponding known sound;

filtering said training auditory spectrogram into a plurality of multidimensional training cortical response signals, each of said training cortical response signals indicative of frequency modulation of said training auditory spectrogram over a corresponding predetermined range of scales and of temporal modulation of said training auditory spectrogram over a corresponding predetermined range of rates;

decomposing said training cortical response signals into orthogonal multidimensional component training signals;

determining a signal size corresponding to each of said orthogonal multidimensional component training signals, said signal size setting a size of said corresponding orthogonal component training signal to retain for classification;

truncating said orthogonal component training signals to said signal size;

classifying said truncated component training signals;

comparing said classification of said truncated component training signals with a classification of said known sound; and

increasing said signal size and repeating the method at said training signal truncating step if said classification of said truncated component training signals does not match said classification of said known sound to within a predetermined tolerance.

7. The method for discriminating sounds in an audio signal as recited in claim 6, where said signal size determining step includes the steps of:

establishing a contribution threshold;

determining a contribution to each said orthogonal component training signals by a corresponding signal component thereof;

selecting as said signal size a number of said corresponding signal components whose contribution to each said orthogonal component training signals is greater than said contribution threshold.

8. The method for discriminating sounds in an audio signal as recited in claim 6, where said orthogonal multidimensional component signal truncating step includes the step of truncating each of said orthogonal component signals to said corresponding signal size.

9. The method for discriminating sounds in an audio signal as recited in claim 1, where said classifying step includes the step of specifying human speech as said predetermined sound.

10. A method for discriminating sounds in an acoustic signal comprising the steps of:

providing a known audio signal associated with a known sound having a known sound classification;

forming a training auditory spectrogram from said known audio signal;

filtering said training auditory spectrogram into a plurality of multidimensional training cortical response signals, each of said training cortical response signals indicative of frequency modulation of said training auditory spectrogram over a corresponding predetermined range of scales and of temporal modulation of said training auditory spectrogram over a corresponding predetermined range of rates;

decomposing said training cortical response signals into orthogonal multidimensional component training signals;

determining a signal size corresponding to each of said orthogonal multidimensional component training signals, said signal size setting a size of said corresponding orthogonal component training signal to retain for classification;

truncating said orthogonal multidimensional component training signals to said signal size;

classifying said truncated component training signals;

comparing said classification of said truncated component training signals with a classification of said known sound;

increasing said signal size and repeating the method at said training signal truncating step if said classification of said truncated component training signals does not match said classification of said known sound to within a predetermined tolerance;

converting the acoustic signal to an audio signal;

forming an auditory spectrogram from said audio signal, said auditory spectrogram characterizing a physiological response to sound represented by the audio signal;

filtering said auditory spectrogram into a plurality of multidimensional cortical response signals, each of said cortical response signals indicative of frequency modulation of said auditory spectrogram over a corresponding predetermined range of scales and temporal modulation of said auditory spectrogram over a corresponding predetermined range of rates;

decomposing said cortical response signals into orthogonal multidimensional component signals;

truncating said orthogonal multidimensional component signals to said signal size; and

classifying said truncated component signals to discriminate therefrom a signal corresponding to a predetermined sound.

11. The method for discriminating sounds in an acoustic signal as recited in claim 10, where said training auditory spectrogram filtering step and said auditory spectrogram filtering step both include the step of filtering via directional selective filters said auditory spectrogram into directional components of said plurality of multidimensional cortical response signals.

12. The method for discriminating sounds in an acoustic signal as recited in claim 11, where said training auditory spectrogram filtering step and said auditory spectrogram filtering step both include the step of selecting maximally directed cortical response signals as said plurality of multidimensional cortical response signals.

13. The method for discriminating sounds in an acoustic signal as recited in claim 11, where said training auditory spectrogram filtering step and said auditory spectrogram filtering step both include the step providing downward selective filters and upward selective filters as said directional selective filters.

14. The method for discriminating sounds in an acoustic signal as recited in claim 10, where said classifying step includes the step of specifying human speech as said predetermined sound.

15. A system to discriminate sounds in an acoustic signal comprising:

an early auditory model execution unit operable to produce at an output thereof an auditory spectrogram of an audio signal provided as an input thereto, said audio signal being a representation of said acoustic signal;

a cortical model execution unit coupled to said output of said auditory model execution unit so as to receive said auditory spectrogram and to produce therefrom at an output thereof a time-varying signal representative of a cortical response to the acoustic signal;

a multi-linear analyzer coupled to said output of said cortical model execution unit and operable to determine a set of multidimensional orthogonal axes from said cortical representations, said multi-linear analyzer further operable to produce a reduced data set relative to said set of multidimensional orthogonal axes; and

a classifier for determining speech from said reduced data set.

16. The system for discriminating sounds in an acoustic signal as recited in claim 15, wherein said cortical model execution unit includes a bank of spectro-temporal modulation selective filters.

17. The system for discriminating sounds in an acoustic signal as recited in claim 16, wherein said each of said spectro-temporal modulation selective filters is characterized by a wavelet.

18. The system for discriminating sounds in an acoustic signal as recited in claim 16, wherein said each of said spectro-temporal modulation selective filters is directionally selective.

19. The system for discriminating sounds in an acoustic signal as recited in claim 15, wherein said classifier includes at least one support vector machine.

20. The system for discriminating sounds in an acoustic signal as recited in claim 15, where said classifier is operable to discriminate human speech.