Parameterized temporal feature analysis

• US 8,311,821 B2
• Filed: 04/21/2004
• Issued: 11/13/2012
• Est. Priority Date: 04/24/2003
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Breebaart, Dirk Jeroen, McKinney, Martin Franciscus
• Primary Examiner(s): PULLIAS, JESSE SCOTT
• Application Number:
  US10/554,010
• Publication Number:
  US 20060196337A1
• Time in Patent Office: 3,128 Days
• Field of Search: 704/200, 704/503, 704/234
• US Class Current: 704/234
• CPC Class Codes: G06F 16/45 Clustering; Classification
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/70 of video data

Abstract

A method (1) for classifying at least one audio signal (A) into at least one audio class (AC), the method (1) comprising the steps of analyzing (10) said audio signal to extract at least one predetermined audio feature, performing (12) a frequency analysis on a set of values of said audio feature at different time instances, deriving (12) at least one further audio feature representing a temporal behavior of said audio feature based on said frequency analysis, and classifying (14) said audio signal based on said further audio feature. With the further audio feature, information is obtained about the temporal fluctuation of an audio feature, which may be advantageous for a classification of audio.

Claims

1. A method for classifying at least one audio signal into at least one audio class, the method comprising the steps of:

analyzing said audio signal to extract at least one predetermined audio feature;

performing a frequency analysis on a set of values of said extracted predetermined audio feature at different time instances resulting in a power spectrum of said extracted predetermined audio feature;

deriving at least one further audio feature representing a temporal behavior of said extracted predetermined audio feature by parameterizing said power spectrum, wherein parameterizing said power spectrum comprises (a) summarizing a frequency axis of the power spectrum by summing energy within at least one predetermined frequency band and (b) dividing (b)(i) the summed energy within the at least one predetermined frequency band by (b)(ii) an average of subsequent values alone of said extracted predetermined audio feature, not including preceding values, to (c) yield a relative modulation depth representing an amount of envelope modulation in the at least one predetermined frequency band; and

classifying said audio signal based on said further audio feature.

2. The method as claimed in claim 1, wherein said at least one predetermined audio feature comprises at least one of the following audio features:

root-mean-square level;

spectral centroid;

bandwidth;

zero-crossing rate;

spectral roll-off frequency;

band energy ratio;

delta spectrum magnitude;

pitch; and

pitch strength.

3. The method as claimed in claim 1, wherein said predetermined audio feature comprises at least one mel-frequency cepstral coefficient.

4. The method as claimed in claim 1, wherein said predetermined audio feature comprises at least one of the psycho-acoustic audio features loudness and sharpness.

5. The method as claimed in claim 1, wherein said deriving step comprises the steps of:

calculating an average value of said set of values of said extracted predetermined audio feature at different time instances;

defining at least one frequency band;

calculating the amount of energy within said frequency band from said frequency analysis; and

defining said further audio feature as said amount of energy divided by said average value.

6. The method as claimed in claim 5, wherein at least one of the following modulation frequency bands are used in said parameterizing said power spectrum:

1-2 Hz;

3-15 Hz; and

20-150 Hz.

7. The method as claimed in claim 1, wherein said at least one further audio feature is defined as at least one coefficient obtained by performing a discrete cosine transformation on the result of said frequency analysis.

8. The method as claimed in claim 1, wherein performing a frequency analysis on a set of values of said extracted predetermined audio feature at different time instances results in a log power spectrum of said extracted predetermined audio feature.

9. A system for classifying at least one audio signal into at least one audio class, the system comprising:

means for analyzing said audio signal to extract at least one predetermined audio feature;

means for performing a frequency analysis on a set of values of said extracted predetermined audio feature at different time instances resulting in a power spectrum of said extracted predetermined audio feature;

means for deriving at least one further audio feature representing a temporal behavior of said extracted predetermined audio feature by parameterizing said power spectrum, wherein parameterizing said power spectrum comprises (a) summarizing a frequency axis of the power spectrum by summing energy within at least one predetermined frequency band and (b) dividing (b)(i) the summed energy within the at least one predetermined frequency band by (b)(ii) an average of subsequent values alone of said extracted predetermined audio feature, not including preceding values, to (c) yield a relative modulation depth representing an amount of envelope modulation in the at least one predetermined frequency band; and

means for classifying said audio signal based on said further audio feature.

10. A music system comprising:

means for playing audio data from a medium; and

a system as claimed in claim 9 for classifying said audio data.

11. A multi-media system comprising:

means for playing audio data from a medium;

a system as claimed in claim 9 for classifying said audio data;

means for displaying video data from a further medium;

means for analyzing said video data; and

means for combining the results obtained from analyzing said video data with the results obtained from classifying said audio data.