FEATURE EXTRACTION FOR IDENTIFICATION AND CLASSIFICATION OF AUDIO SIGNALS

• US 20090012638A1
• Filed: 07/04/2008
• Published: 01/08/2009
• Est. Priority Date: 07/06/2007
• Status: Active Grant

• Current Assignee: Xia Lou
• Original Assignee: Xia Lou
• Inventors: Lou, Xia
• Granted Patent: US 8,140,331 B2
• Time in Patent Office: Days
• US Class Current: 700/94
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

Characteristic features are extracted from an audio sample based on its acoustic content. The features can be coded as fingerprints, which can be used to identify the audio from a fingerprints database. The features can also be used as parameters to separate the audio into different categories.

Claims

1. A method for extracting audio features from an audio frame, the method comprising:

Pre-process the audio;

Transform the audio into a plurality of frequency sub-bands;

Store the said transformation results as sub-band coefficients in S, each sub-band coefficient correlates to the audio signal at a time slice with a certain frequency band.

For every sub-band coefficients in S, subtract the mean value which corresponding to the same time slice, store the results in Z.

For every sub-band in Z, weight and sum a plurality sequential chunk of sub-band coefficients;

For every sub-band, weight the above said result and differential encode with a sub-band coefficient in the same sub-band, store the results in B.

Calculate the sum of all sub-band coefficients in the same time slice as the total band coefficients;

Weight and sum a plurality sequential chunk of total band coefficients;

Quantize the sub-band coefficients with at least one quantization threshold, which is derived from the said above operation, store the results in A.

Calculate audio fingerprints from A and B.

2. The method of claim 1, wherein said A and B are matrixes with one axis corresponding to time slice and the other one axis corresponding to frequency band, and the method further comprising:

Re-sample said A and B;

Operate each element in A with each corresponding element in B;

Store the result of said above in matrix C;

For each time slice in C, encode the values with all different frequency bands into bit sequence to generate the audio fingerprint for that time slice;

Group the sequential audio fingerprint to represent a segment of audio, store in an audio database.

3. The method of claim 1, wherein said A and B are vectors corresponding to a time slice and each elements corresponding to each frequency band, and the method further comprising:

Re-sample said A and B;

Operate each element in A with each corresponding element in B;

Store the result of said above in vector C;

For each time slice, encode the values with all different frequency bands into bit sequence to generate the audio fingerprint for that time slice;

Group the sequential audio fingerprint to represent a segment of audio, store in an audio database.

4. The method of claim 1, wherein said A is a matrix with one axis corresponding to time slice and the other one axis corresponding to frequency band, and said B is a vector which corresponding to a time slice and each elements of B corresponding to each frequency band, and the method further comprising:

Re-sample said A and B;

For the time slice of vector B, find and extract corresponding vector from matrix A;

Operate each element in said vector with each corresponding element in B;

Store the result of said above in vector C;

Encode the values with all different frequency bands into bit sequence to generate the audio fingerprint for that time slice;

Group the sequential audio fingerprint to represent a segment of audio, store in an audio database.

5. The method of claim 1, wherein said B is a matrix with one axis corresponding to time slice and the other one axis corresponding to frequency band, and said A is a vector which corresponding to a time slice and each elements of A corresponding to each frequency band, and the method further comprising:

Re-sample said A and B;

For the time slice of vector A, find and extract corresponding vector from matrix B;

Operate each element in said vector with each corresponding element in A;

Store the result of said above in vector C;

Encode the values with all different frequency bands into bit sequence to generate the audio fingerprint for that time slice;

Group the sequential audio fingerprint to represent a segment of audio, store in an audio database.

6. The method of claim 1, further comprising:

Calculate zero crossing, linear prediction coefficients, mean, variance, derivatives and second order moments of A;

Group the said above results into intra-band features.

Use said inter-band features to facilitate audio classification process.

7. The method of claim 1, further comprising:

Calculate zero crossing, linear prediction coefficients mean, variance, derivatives and second order moments of B;

Group the said above results into inter-band features.

Use said intra-band features to facilitate audio classification process.

8. A processor readable medium bearing processor executable instructions for carrying out the method of claim 1.

20. The method of claim 1, wherein the sub-band coefficients in S are in logarithmic scale.

9. A method for identifying an unknown audio signal by employing audio fingerprints of a plurality of audio signals, comprising the steps of:

Calculating audio fingerprints for the said audio signal or for part of the audio signal, including;

Pre-process the audio;

Transform the audio into a plurality of frequency sub-bands;

Store the said transformation results as sub-band coefficients in S, each sub-band coefficient correlates to the audio signal at a time slice with a certain frequency band.

For every sub-band coefficients in S, subtract the mean value which corresponding to the same time slice, store the results in Z.

For every sub-band in Z, weight and sum a plurality sequential chunk of sub-band coefficients;

For every sub-band, weight the above said result and differential encode with a sub-band coefficient in the same sub-band, store the results in B.

Calculate the sum of all sub-band coefficients in the same time slice as the total band coefficients;

Weight and sum a plurality sequential chunk of total band coefficients;

Quantize the sub-band coefficients with at least one quantization threshold, which is derived from the said above operation, store the results in A.

Obtain audio fingerprints from A and B.

Obtain same length fingerprints from the reference audio fingerprints;

Compare the said two fingerprints to determine the similarity.

10. The method as in claim 9, wherein said comparison step is based on bit error rate (BER) calculation. If the BER is less than a threshold, the positive identification is found, otherwise no identification is found.

11. The method as in claim 9, wherein said comparison step is based on Hamming distance calculation. If the said distance is less than a threshold, the positive identification is found, otherwise no identification is found.

12. A processor readable medium bearing processor executable instructions for carrying out the method of claim 9.

13. A system for extracting audio features from an audio frame, the system comprising:

Pre-process the audio;

Transform the audio into a plurality of frequency sub-bands;

Store the said transformation results as sub-band coefficients in S, each sub-band coefficient correlates to the audio signal at a time slice with a certain frequency band.

For every sub-band coefficients in S, subtract the mean value which corresponding to the same time slice, store the results in Z.

For every sub-band in Z, weight and sum a plurality sequential chunk of sub-band coefficients;

For every sub-band, weight the above said result and differential encode with a sub-band coefficient in the same sub-band, store the results in B.

Calculate the sum of all sub-band coefficients in the same time slice as the total band coefficients;

Weight and sum a plurality sequential chunk of total band coefficients;

Quantize the sub-band coefficients with at least one quantization threshold, which is derived from the said above operation, store the results in A.

Calculate audio fingerprints from A and B.

14. The system of claim 13, wherein said A and B are matrixes with one axis corresponding to time slice and the other one axis corresponding to frequency band, and the system further comprising:

Re-sample said A and B;

Operate each element in A with each corresponding element in B;

Store the result of said above in matrix C;

For each time slice in C, encode the values with all different frequency bands into bit sequence to generate the audio fingerprint for that time slice;

Group the sequential audio fingerprint to represent a segment of audio, store in an audio database.

15. The system of claim 13, wherein said A and B are vectors corresponding to a time slice and each elements corresponding to each frequency band, and the system further comprising:

Re-sample said A and B;

Operate each element in A with each corresponding element in B;

Store the result of said above in vector C;

For each time slice, encode the values with all different frequency bands into bit sequence to generate the audio fingerprint for that time slice;

Group the sequential audio fingerprint to represent a segment of audio, store in an audio database.

16. The system of claim 13, wherein said A is a matrix with one axis corresponding to time slice and the other one axis corresponding to frequency band, and said B is a vector which corresponding to a time slice and each elements of B corresponding to each frequency band, and the system further comprising:

Re-sample said A and B;

For the time slice of vector B, find and extract corresponding vector from matrix A;

Operate each element in said vector with each corresponding element in B;

Store the result of said above in vector C;

Encode the values with all different frequency bands into bit sequence to generate the audio fingerprint for that time slice;

Group the sequential audio fingerprint to represent a segment of audio, store in an audio database.

17. The system of claim 13, wherein said B is a matrix with one axis corresponding to time slice and the other one axis corresponding to frequency band, and said A is a vector which corresponding to a time slice and each elements of A corresponding to each frequency band, and the system further comprising:

Re-sample said A and B;

For the time slice of vector A, find and extract corresponding vector from matrix B;

Operate each element in said vector with each corresponding element in A;

Store the result of said above in vector C;

Encode the values with all different frequency bands into bit sequence to generate the audio fingerprint for that time slice;

Group the sequential audio fingerprint to represent a segment of audio, store in an audio database.

18. The system of claim 13, further comprising:

Calculate zero crossing rate, linear prediction coefficients, mean, variance, derivatives and second order moments of A;

Group the said above results into intra-band features.

Use said inter-band features to facilitate audio classification process.

19. The system of claim 13, further comprising:

Calculate zero crossing rate, linear prediction coefficients, mean, variance, derivatives and second order moments of B;

Group the said above results into inter-band features.

Use said intra-band features to facilitate audio classification process.