Method and system for recognizing, indexing, and searching acoustic signals

• US 20010044719A1
• Filed: 05/21/2001
• Published: 11/22/2001
• Est. Priority Date: 07/02/1999
• Status: Abandoned Application

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Casey, Michael A.
• Application Number:
  US09/861,808
• Publication Number:
  US 20010044719A1
• Time in Patent Office: Days
• US Class Current: 704/245
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]
• CPC Class Codes: G10L 21/028 using properties of sound source

Abstract

A computerized method extracts features from an acoustic signal generated from one or more sources. The acoustic signal are first windowed and filtered to produce a spectral envelope for each source. The dimensionality of the spectral envelope is then reduced to produce a set of features for the acoustic signal. The features in the set are clustered to produce a group of features for each of the sources. The features in each group include spectral features and corresponding temporal features characterizing each source. Each group of features is a quantitative descriptor that is also associated with a qualitative descriptor. Hidden Markov models are trained with sets of known features and stored in a database. The database can then be indexed by sets of unknown features to select or recognize like acoustic signals.

Claims

1. A method for extracting features from an acoustic signal generated from a single source, comprising:

windowing and filtering the acoustic signal to produce a spectral envelope; and

reducing the dimensionality of the spectral envelope to produce a set of features, the set including spectral features and corresponding temporal features characterizing the single source.

2. The method of

claim 1

further comprising:

multiplying the spectral features and temporal features using a outer product to reconstruct a spectrogram of the accoustic signal.

3. The method of

claim 1

further comprising:

applying independent component analysis to the set of feature to separate the features in the set.

4. The method of

claim 1

further comprising:

log-scaling and L2-normalizing the spectral envelope to a decibel scale and unit L2-norm before reducing the dimensionality of the spectral envelope.

5. A method for extracting features from an acoustic signal generated from a plurality of sources, comprising:

windowing and filtering the acoustic signal to produce a spectral envelope;

reducing the dimensionality of the spectral envelope to produce a set of features;

clustering the features in the set to produce a group of features for each of the plurality of sources, the features in each group including spectral features and corresponding temporal features characterizing each source.

6. The method of

claim 5

wherein each group of features is a quantitative descriptor of each source, and futher comprising:

associating a qualitative descriptor with each quantitative descriptor to generate a category for each source.

7. The method of

claim 6

further comprising:

organizing the categories in a database as a taxonomy of classified sources;

relating each category with at least one other category in the database by a relational link.

8. The method of

claim 7

wherein the categories are stored in the database using a description definition language.

9. The method of

claim 8

wherein a particular category in a DDL instantiation defines a basis projection matrix that reduces a series of logarithmic frequencies spectra of a particular source to fewer dimensions.

10. The method of

claim 6

wherein the categories include environmental sounds, background noises, sound effects, sound textures, animal sounds, speech, non-speech utterances, and music.

11. The method of

claim 7

further comprising:

combining substantially similar categories in the database as a hierarchy of classes.

12. The method of

claim 6

a particular quantitative descriptor further includes a harmonic envelope descriptor, and fundamental frequency descriptor.

13. The method of

claim 5

wherein the temporal features describe a trajectory of the spectral features over time, and further comprising:

partitions the acoustic signal generated by a particular source into a finite number of states based on the corresponding spectral features;

representing each state by a continuous probability distribution;

representing the temporal features by a transition matrix to model probabilities of transitions to a next state given a current state.

14. The method of

claim 13

wherein the continuous probability distribution is a Gaussian distribution parameterized by a 1×

n vector of means m, and an n×

n covariance matrix K, where n is the number of spectral features in each spectral envelope, and the probabilities of a particular spectral envelope x is given by;

15. The method of

claim 5

wherein each source is known, and further comprising:

training, for each known source, a hidden Markov model with the set of features;

storing each trained hidden Markov model with the associated set of spectral features in a database.

16. The method of

claim 5

wherein a set of acoustic signals belongs to a known category, and further comprising:

extracting a spectral basis for the acoustic signals;

training a hidden Markov model using the temporal features of the acoustic signals;

storing each trained hidden Markov model with the associated spectral basis features.

17. The method of

claim 15

further comprising:

generating an unknown acoustic from an unknown source;

windowing and filtering the unknown acoustic signal to produce an unknown spectral envelope;

reducing the dimensionality of the unknown spectral envelope to produce a set of unknown features, the set including unknown spectral features and corresponding unknown temporal features characterizing the unknown source;

selecting one of the stored hidden Markov models that best-fits the unknown set of features to identify the unknown source.

18. The method of

claim 17

wherein a plurality of the stored hidden Markov models are selected to identify a plurality of known source substantially similar to the unknown source.