Method and apparatus for automatically recognizing audio data

• US 20050027514A1
• Filed: 04/05/2004
• Published: 02/03/2005
• Est. Priority Date: 07/28/2003
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Sun, Xiaobing, Zhang, Jian, Lu, Wei
• Granted Patent: US 8,140,329 B2
• Time in Patent Office: Days
• US Class Current: 704/205
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

A method and apparatus are proposed for automatically recognizing observed audio data. An observation vector is created of audio features extracted from the observed audio data and the observed audio data is recognized from the observation vector. The audio features include features are selected from a group of 3 types of features obtained from the observed audio data: (i) ICA features obtained by processing the observed audio data, (ii) first MFCC features obtained by removing a logarithm step from the conventional MFCC process, or (iii) second MFCC features obtained by applying the ICA process to results of a mel scale filter bank.

Claims

1. A method of identifying, among a plurality of audio files, a first one of the audio files, the method employing a segment of audio data which is derived from the first audio file and comprising the steps of:

(a) creating an observation vector with audio features extracted from the segment of audio data; and

(b) recognizing the first audio file using the observation vector;

wherein the audio features comprise features obtained by analyzing the audio data, or a transformed version of the audio data, to derive a transform based on its significant features, and applying the transform to the audio data, or the transformed version of the audio data respectively, to obtain amplitudes of the significant features.

2. A method according to claim 1 in which the transform is an independent component analysis (ICA) of the audio data or the transformed version of the audio data.

3. A method according to claim 2 in which the audio features include at least one of:

(i) ICA features obtained by an ICA analysis of the observed audio data segment,

(iii) ICA-MFCC features obtained by transforming the audio segment into the frequency domain, applying a mel scale filter bank, and applying an ICA analysis to the output of mel scale filter bank.

4. A method according to claim 3 in which ICA analysis comprises the steps of:

computing ICA basis functions and weight functions from the audio data or Mel spectrum data;

using the basis functions and weight function to transform the audio data or Mel spectrum data into the ICA domain to obtain ICA coefficients.

5. A method according to claim 4 in which the audio features include said ICA features, and the step of computing the ICA features comprises the steps of:

pre-emphasizing the audio data to improve the SNR of the data;

windowing the pre-emphasized data; and

ICA transforming the windowed data with the ICA basis functions and weight functions to obtain the ICA features.

6. A method according to claim 4, in which the audio features include said ICA-MFCC features, and the ICA-MFCC features are obtained by steps of:

preprocessing the audio data to pre-emphasize and window the audio data;

transforming the processed data from time domain into the frequency domain;

passing the transformed data through a set of Mel scale filters to obtain Mel spectrum data; and

ICA processing the Mel spectrum data to obtain ICA coefficients as the first MFCC features.

7. A method according to claim 6, wherein the step of computing ICA basis functions and weight functions comprises the steps of:

partitioning the audio data or Mel spectrum data to provide partitioned signal;

de-correlating the partitioned signal by PCA algorithm to provide de-correlated signals; and

ICA learning the de-correlated signals by a fast ICA algorithm to obtain the ICA basis functions and weight functions.

8. A method according to claim 1 in which the audio features further include MFCC features obtained by transforming the audio segment into the frequency domain, applying a mel scale filter bank, and transforming the output of the mel scale filter bank signal back to time domain without applying a logarithmic step.

9. A method of identifying, among a plurality of audio files, a first one of the audio files, the method employing a segment of audio data which is derived from the first audio file and comprising the steps of:

(a) creating an observation vector with audio features extracted from the segment of audio data; and

(b) recognizing the first audio file using the observation vector;

wherein the audio features comprise MFCC features obtained by transforming the audio segment into the frequency domain, applying a mel scale filter bank, and transforming the output of the mel scale filter bank signal back to time domain without applying a logarithmic step.

10. A method according to claim 9, in which the MFCC features are obtained by:

preprocessing the audio data to pre-emphasize and window the audio data;

transforming the processed data from time domain into the frequency domain;

passing the transformed data through a set of Mel scale filters to obtain Mel spectrum data; and

converting the Mel spectrum data back to time domain data to obtain the second MFCC features.

11. A method according to claim 9, wherein step (b) is performed by determining, within a database containing HMM models for each respective target audio file, the HMM for which probability of the observation vector being obtained given the target audio file is maximal.

12. A method of identifying, among a plurality of audio files, a first one of the audio files, the method employing a segment of audio data which is derived from the first audio file and comprising the steps of:

(a) creating an observation vector with audio features extracted from the segment of audio data; and

(b) recognizing the first audio file using the observation vector;

wherein step (b) is performed by determining, within a database containing HMM models for each respective target audio file, the HMM for which probability of the observation vector being obtained given the target audio file is maximal.

13. A method for generating a database of HMM models for use in a method according to claim 12, the method comprising the steps of:

extracting a plurality of segments from each of the target audio files;

generating training data which is the amplitudes of statistically significant audio features of the segments;

initializing HMM model parameters for the target audio file with the training data by an HMM initialization algorithm;

training the initialized model parameters to optimize the model parameters by an HMM training algorithm; and

establishing an audio modeling database of the trained HMM model parameters.

14. An apparatus for identifying, among a plurality of audio files, a first one of the audio files, based on a segment of audio data which is derived from the first audio file, the apparatus comprising:

(a) means for creating an observation vector with audio features extracted from the segment of audio data; and

(b) means for recognizing the first audio file using the observation vector;

wherein the audio features comprise features obtained by analyzing the audio data, or a transformed version of the audio data, to derive a transform based on its significant features, and applying the transform to the audio data, or the transformed version of the audio data respectively, to obtain amplitudes of the significant features.

15. An apparatus according to claim 14 in which the transform is an independent component analysis (ICA) of the audio data or the transformed version of the audio data.

16. An apparatus according to claim 15 in which the audio features include at least one of:

(i) ICA features obtained by passing the observed audio data segment to ICA analysis means,

(iii) ICA-MFCC features obtained transforming the audio segment into the frequency domain using FFT means, applying the result to a mel scale filter bank, and passing the output of mel scale filter bank to ICA analysis means.

17. An apparatus according to claim 16 in which ICA analysis means comprises:

means for computing ICA basis functions and weight functions from the audio data or Mel spectrum data; and

means for using the basis finctions and weight fuictions to transform the audio data or Mel spectrum data into the ICA domain to obtain ICA coefficients.

18. An apparatus according to claim 14 in which the audio features further include MFCC features obtained by transforming the audio segment into the frequency domain, applying a mel scale filter bank, and transforming the output of the mel scale filter bank signal back to time domain without applying a logarithmic step.

20. An apparatus according to claim 14, wherein said means for recognizing comprises:

a database containing HMM models for each respective target audio file, and

means for determining the HMM in the database for which probability of the observation vector being obtained given the target audio file is maximal.

19. Apparatus for identifying, among a plurality of audio files, a first one of the audio files, based on a segment of audio data which is derived from the first audio file, the apparatus comprising:

(a) means for creating an observation vector with audio features extracted from the segment of audio data; and

(b) means for recognizing the first audio file using the observation vector;

wherein the audio features comprise MFCC features obtained by transforming the audio segment into the frequency domain, applying a mel scale filter bank, and transforming the output of the mel scale filter bank signal back to time domain without applying a logarithmic step.

23. An apparatus according to claim 19, wherein said means for recognizing comprises:

a database containing HMM models for each respective target audio file, and

means for determining the HMM in the database for which probability of the observation vector being obtained given the target audio file is maximal.

21. An apparatus for identifying, among a plurality of audio files, a first one of the audio files, based on a segment of audio data which is derived from the first audio file, the apparatus comprising:

(a) means for creating an observation vector with audio features extracted from the segment of audio data;

(b) a database containing HMM models for each respective target audio file, and

(c) means for determining the HMM in the database for which probability of the observation vector being obtained given the target audio file is maximal.

22. An apparatus according to claim 21, further comprising:

means for extracting as training data a plurality of segments from each of the target audio files;

means for initializing HMM model parameters for the target audio file with the training data by an HMM initialization algorithm;

means for training the initialized model parameters to optimize the model parameters by an HMM training algorithm; and

means for establishing an audio modeling database of the trained HMM model parameters.