Apparatus and method for robust classification of audio signals, and method for establishing and operating an audio-signal database, as well as computer program

• US 7,580,832 B2
• Filed: 08/31/2004
• Issued: 08/25/2009
• Est. Priority Date: 07/26/2004
• Status: Expired due to Fees

• Current Assignee: M2Any GmbH
• Original Assignee: M2Any GmbH
• Inventors: Cremer, Markus, Herre, Juergen, Allamanche, Eric, Kastner, Thorsten, Hellmuth, Oliver
• Primary Examiner(s): Lerner; Martin
• Application Number:
  US10/931,635
• Publication Number:
  US 20060020958A1
• Time in Patent Office: 1,820 Days
• Field of Search: 704/205, 704/206, 704/231, 704/500, 704/503, 704/230, 341/50, 341/51, 341/67, 341/200, 375/240.03, 375/243, 375/245
• US Class Current: 704/205
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

An apparatus for producing a fingerprint signal from an audio signal includes a means for calculating energy values for frequency bands of segments of the audio signal which are successive in time, so as to obtain, from the audio signal, a sequence of vectors of energy values, a means for scaling the energy values to obtain a sequence of scaled vectors, and a means for temporal filtering of the sequence of scaled vectors to obtain a filtered sequence which represents the fingerprint, or from which the fingerprint may be derived. Thus, a fingerprint is produced which is robust against disturbances due to problems associated with coding or with transmission channels, and which is especially suited for mobile radio applications.

Claims

1. An apparatus for producing a fingerprint signal from an audio signal, comprising:

a calculator for calculating energy values for frequency bands of segments of the audio signal which are successive in time, an energy value for a frequency band depending on an energy of the audio signal in the frequency band, so as to obtain a sequence of vectors of energy values from the audio signal, a vector component being an energy value in a frequency band;

a scaler for scaling the energy values to obtain a sequence of scaled vectors;

wherein the scaler includes a means for taking the logarithm and a suppressor for suppressing a steady component which is connected downstream of the means for taking the logarithm,

wherein the suppressor for suppressing a steady component includes a high-pass filter;

a low pass filter for temporally filtering the sequence of scaled vectors to obtain a filtered sequence which represents the fingerprint signal, or from which the fingerprint signal maybe derived; and

a quantizer connected downstream of the filters and configured to quantize the filtered sequence or a signal based thereon so as to derive the fingerprint signal from the filtered sequence,

wherein the quantizer is configured such that a width of a quantization level for a high energy value is larger than a width of a quantization level for a small energy value.

2. The apparatus as claimed in claim 1, wherein one segment of the audio signal has a length in time of at least 10 ins.

3. The apparatus as claimed in claims 1 or 2, wherein the calculator for calculating energy values for frequency bands is configured to perform a discrete Fourier transform (DFT) by means of a fast Fourier transform (FFT) on the audio signal of a segment, to obtain Fourier coefficients, to square amounts of the Fourier coefficients, to obtain squared amounts of the Fourier coefficients, and to sum up the squared amounts of the Fourier coefficients band by band to obtain energy values for a frequency band.

4. The apparatus as claimed in claim 1, wherein the frequency bands have a variable bandwidth, wherein a bandwidth with frequency bands having higher frequencies is larger than a bandwidth with frequency bands having lower frequencies.

5. The apparatus as claimed in claim 1, wherein the scaler is configured to compress a range of values of the energy values such that a range of values of compressed energy values is smaller than a range of non-compressed energy values.

6. The apparatus as claimed in claim 1, wherein the scaler is configured to normalize the energy values.

7. The apparatus as claimed in claim 1, wherein the scaler is configured to scale the energy values to a range of values between a lower limit and an upper limit, or to take a logarithm of the energy values.

8. The apparatus as claimed in claim 1, wherein the scaler is configured to scale the energy values so as to correspond to the human loudness perception.

9. The apparatus as claimed in claim 1, wherein the scaler includes a means for taking the logarithm and a suppressor for suppressing a steady component which is connected downstream of the means for taking the logarithm.

10. The apparatus as claimed in claim 9, wherein the suppressor for suppressing a steady component includes a high-pass filter.

11. The apparatus as claimed in claim 1, wherein the scaler is configured to perform a normalization of the energy values using a total energy created by forming a sum of several energy values, the normalization being performed by dividing the energy values, in a band-by-band manner, by a normalization factor which is identical with the total energy.

12. The apparatus as claimed in claim 1, wherein the filter for temporally filtering the sequence of scaled vectors is configured to achieve temporal smoothing of the sequence of scaled vectors.

13. Apparatus as claimed in claim 1, wherein the filter for temporal filtering includes a low-pass filter having a cutoff frequency of less than 50 Hz.

14. The apparatus as claimed in claim 1, wherein the filter for temporally filtering the sequence of scaled vectors includes a high-pass filter with a cutoff frequency of less than 10 Hz.

15. The apparatus as claimed in claim 1, wherein the filter for temporally filtering the sequence of scaled vectors includes a means for forming the difference between two energy values in the same frequency band which are successive in time.

16. The apparatus as claimed in claim 1, wherein the filter for temporal filtering includes a low-pass filter as well as a decimation means connected to an output of the low-pass filter and configured to reduce the number of vectors derived from the audio signal.

17. The apparatus as claimed in claim 1, wherein the filter for temporal filtering comprises a high-pass filter configured to reduce the range of values of the values to be quantized.

18. The apparatus as claimed in claim 1, wherein the quantizer comprises such a classification of the quantization levels that a maximum relative quantization error is identical for large and small energy values within a tolerance range.

19. The apparatus as claimed in claim 18, wherein the tolerance range is ±

3 db.

20. The apparatus as claimed in claim 1, wherein the quantizer is configured to use quantization levels on the grounds of an amplitude statistic, the quantization levels being adapted in accordance with the amplitude statistic of the signal to be quantized, which statistic includes a statement about a relative frequency of values of the signal to be quantized, a fine classification of the quantizing steps being effected for a range of values with values of the signal to be quantized having a high relative abundance, and a coarse classification of the quantization levels being effected for a range of values with values of the signal to be quantized having a low relative abundance.

21. The apparatus as claimed in claim 1, wherein the quantizer is configured such that it associates a symbol with a vector of the filtered sequence.

22. A method for producing a fingerprint signal from an audio signal, comprising:

calculating energy values for frequency bands of segments of the audio signal which are successive in time, an energy value for a frequency band depending on an energy of the audio signal in the frequency band, so as to obtain a sequence of vectors of energy values from the audio signal, a vector component being an energy value in a frequency band;

scaling the energy values to obtain a sequence of scaled vectors wherein scaling comprises taking the logarithm of the energy values; and

suppressing, downstream with respect to taking the logarithm, a steady component, using a high-pass filtering operation;

temporally low-pass filtering the sequence of scaled vectors to obtain a filtered sequence which represents the fingerprint signal, or from which the fingerprint signal may be derived; and

quantizing the filtered sequence or a signal based thereon so as to derive the fingerprint signal from the filtered sequence or from the signal based thereon, wherein a width of a quantization level for a high energy value is larger than a width of a quantization level for a small energy value.

23. An apparatus for characterizing an audio signal, comprising:

an apparatus or producing a fingerprint signal from an audio signal, comprising;

a calculator for calculating energy values for frequency bands of segments of the audio signal which are successive in time, an energy value for a frequency band depending on an energy of the audio signal in the frequency band, so as to obtain a sequence of vectors of energy values from the audio signal, a vector component being an energy value in a frequency band;

a scaler for scaling the energy values to obtain a sequence of scaled vectors wherein the scaler includes a means for taking the logarithm and a suppressor for suppressing a steady component which is connected downstream of the means for taking the logarithm,

wherein the suppressor for suppressing a stead component includes a high-pass filter;

a low pass filter for temporally filtering the sequence of scaled vectors to obtain a filtered sequence which represents he fingerprint signal, or from which the fingerprint signal may be derived; and

a quantizer connected downstream of the filters and configured to quantize the filtered sequence or a signal based thereon so as to derive the fingerprint signal from the filtered sequence, wherein the quantizer is configured such that a width of a quantization level for a high energy value is larger than a width of a quantization level for a small energy value; and

a statement-maker about the audio content of the audio signal on the grounds of the fingerprint signal.

24. A method for characterizing an audio signal, comprising:

producing a fingerprint signal using a method for producing a fingerprint signal from an audio signal, the method comprising;

calculating energy values for frequency bands of segments of the audio signal which are successive in time, an energy value for a frequency band depending on an energy of the audio signal in the frequency band, so as to obtain a sequence of vectors of energy values from the audio signal, a vector component being an energy value in a frequency band;

scaling the energy values to obtain a sequence of scaled vectors wherein scaling comprises taking the logarithm of the energy values;

suppressing, downstream with respect to taking the logarithm, a steady component, using a high-pass filtering operation;

temporally low-pass filtering the sequence of scaled vectors to obtain a filtered sequence which represents the fingerprint signal, or from which the fingerprint signal may be derived; and

quantizing the filtered sequence or a signal based thereon so as to derive the fingerprint signal from the filtered sequence or from the signal based thereon, wherein a width of a quantization level for a high energy value is larger than a width of a quantization level for a small energy value, and

making a statement about the audio content of the audio signal on the grounds of the fingerprint signal.

25. A method for establishing an audio database, comprising:

producing a fingerprint for each audio signal to be captured in the audio database, using the method for producing a fingerprint signal from an audio signal, the method comprising;

calculating energy values for frequency bands of segments of the audio signal which are successive in time, an energy value for a frequency band depending on an energy of the audio signal in the frequency band, so as to obtain a sequence of vectors of energy values from the audio signal, a vector component being an energy value in a frequency band;

scaling the energy values to obtain a sequence of scaled vectors wherein scaling comprises taking the logarithm of the energy values;

suppressing, downstream with respect to taking the logarithm, a steady component, using a high-pass filtering operation;

temporally low-pass filtering the sequence of scaled vectors to obtain a filtered sequence which represents the fingerprint signal, or from which the fingerprint signal may be derived; and

quantizing the filtered sequence or a signal based thereon so as to derive the fingerprint signal from the filtered sequence or from the signal based thereon, wherein a width of a quantization level for a high energy value is larger than a width of a quantization level for a small energy value,

for each audio signal to be captured, storing in the fingerprint as well as further information in the audio database which belongs to the audio signal, so that an association of a fingerprint and the corresponding information is given.

26. A method for obtaining information on the grounds of an audio-signal database, wherein associated fingerprint signals having been formed by a method for producing a fingerprint signal from an audio signal, the method comprising:

calculating energy values for frequency bands of segments of the audio signal which are successive in time, an energy value for a frequency band depending on an energy of the audio signal in the frequency band, so as to obtain a sequence of vectors of energy values from the audio signal, a vector component being an energy value in a frequency band;

scaling the energy values to obtain a sequence of scaled vectors wherein scaling comprises taking the logarithm of the energy values;

suppressing, downstream with respect to taking the logarithm, a steady component, using a high-pass filtering operation;

temporally low-pass filtering the sequence of scaled vectors to obtain a filtered sequence which represents the fingerprint signal, or from which the fingerprint signal may be derived; and

quantizing the filtered sequence or signal based thereon so as to derive the fingerprint signal from the filtered sequence or from the signal based thereon, wherein a width of a quantization level for a high energy value is larger than a width of a quantization level for a small energy value,

are stored for several audio signals, and for obtaining a predefined search audio signals, the method comprising;

forming a search fingerprint signal belonging to the search audio signal using a method for producing a fingerprint signal from an audio signal, comprising;

calculating energy values for frequency bands of segments of the audio signal which are successive in time, an energy value for a frequency band depending on an energy of the audio signal in the frequency band, so as to obtain a sequence of vectors of energy values from the audio signal, a vector component being an energy value in a frequency band;

scaling the energy values to obtain a sequence of scaled vectors wherein scaling comprises taking the logarithm of the energy values;

suppressing, downstream with respect to taking the logarithm, a steady component, using a high-pass filtering operation;

temporally low-pass filtering the sequence of scaled vectors to obtain a filtered sequence which represents the fingerprint signal, or from which the fingerprint signal may be derived; and

quantizing the filtered sequence or a signal based thereon so as to derive the fingerprint signal from the filtered sequence or from the signal based thereon, wherein a width of a quantization level for a high energy value is larger than a width of a quantization level for a small energy value,

comparing the search fingerprint signal with at least one fingerprint signal stored in the database, and making a statement about the similarity thereof.

27. The method as claimed in claimed 29, further comprising:

outputting metadata to the audio signals on which the fingerprint signals stored in the database are based, depending on the statement about the similarity of the search fingerprint signal with the fingerprint signals stored in the database.

28. A computer readable medium having stored thereon a computer program having a program code for performing the method for producing a fingerprint signal from an audio signal, the method comprising:

calculating energy values for frequency bands of segments of the audio signal which are successive in time, an energy value for a frequency band depending on an energy of the audio signal in the frequency band, so as to obtain a sequence of vectors of energy values from the audio signal, a vector component being an energy value in a frequency band;

scaling the energy values to obtain a sequence of scaled vectors wherein scaling comprises taking the logarithm of the energy values;

suppressing, downstream with respect to taking the logarithm, a steady component, using a high-pass filtering operation;

temporally low-pass filtering the sequence of scaled vectors to obtain a filtered sequence which represents the fingerprint signal, or from which the fingerprint signal may be derived, and

quantizing the filtered sequence or a signal based thereon so as to derive the fingerprint signal from the filtered sequence or from the signal based thereon, wherein a width of a quantization level for a high energy value is larger than a width of a quantization level for a small energy value when the computer program runs on a computer.

29. An apparatus for producing a fingerprint signal from an audio signal, comprising:

a calculator for calculating energy values for frequency bands of segments of the audio signal which are successive in time, an energy value for a frequency band depending on an energy of the audio signal in the frequency band, so as to obtain a sequence of vectors of energy values from the audio signal, a vector component being an energy value in a frequency band;

a scaler for scaling the energy values to obtain a sequence of scaled vectors wherein the scaler includes a means for taking the logarithm and a suppressor for suppressing a steady component which is connected downstream of the means for taking the logarithm,

wherein the suppressor for suppressing a steady component includes a high pass filter;

a low-pass filter for temporally filtering the sequence of scaled vectors to obtain a filtered sequence which represents the fingerprint signal, or from which the fingerprint signal may be derived; and

a quantizer connected downstream of the filters and configured to quantize the filtered sequence or a signal based thereon so as to derive the fingerprint signal from the filtered sequence,

wherein the quantizer is configured to use quantization levels on the grounds of an amplitude statistic, the quantization levels being adapted in accordance with the amplitude statistic of the signal to be quantized, which statistic includes a statement about a relative frequency of values of the signal to be quantized, a fine classification of the quantizing levels being effected for a range of values with values of the signal to be quantized having a high relative abundance, and a coarse classification of the quantization levels being effected for a range of values with values of the signal to be quantized having a low relative abundance.

30. An apparatus for producing a fingerprint signal from an audio signal, comprising:

a calculator for calculating energy values for frequency bands of segments of the audio signal which are successive in time, an energy value for a frequency band depending on an energy of the audio signal in the frequency band, so as to obtain a sequence of vectors of energy values from the audio signal, a vector component being an energy value in a frequency band;

a scaler for scaling the energy values to obtain a sequence of scaled vectors wherein the scaler includes a means for taking the logarithm and a suppressor for suppressing a steady component which is connected downstream of the means for taking the logarithm,

wherein the suppressor for suppressing a steady component includes a high-pass filter;

a low-pass filter for temporally filtering the sequence of scaled vectors to obtain a filtered sequence which represents the fingerprint signal, or from which the fingerprint signal may be derived; and

a quantizer connected downstream of the filters and configured to quantize the filtered sequence or a signal based thereon so as to derive the fingerprint signal from the filtered sequence,

wherein the quantizer comprises such a classification of the quantization levels that a maximum relative quantization error is identical for large and small energy values within a tolerance range.

31. A method for producing a fingerprint signal from an audio signal, comprising:

calculating energy values for frequency bands of segments of the audio signal which are successive in time, an energy value for a frequency band depending on an energy of the audio signal in the frequency band, so as to obtain a sequence of vectors of energy values from the audio signal, a vector component being an energy value in a frequency band;

scaling the energy values to obtain a sequence of scaled vectors wherein scaling comprises taking the logarithm of the energy values, and

suppressing, downstream with respect to taking the logarithm, a steady component, using a high-pass filtering operation; and

temporally low-pass filtering the sequence of scaled vectors to obtain a filtered sequence which represents the fingerprint signal, or from which the fingerprint signal may be derived; and

quantizing the filtered sequence or a signal based thereon so as to derive the fingerprint signal from the filtered sequence or the signal based thereon using such a classification of the quantization levels that a maximum relative quantization error is identical for large and small energy values within a tolerance range.

32. A method for producing a fingerprint signal from an audio signal, comprising:

calculating energy values for frequency bands of segments of the audio signal which are successive in time, an energy value for a frequency band depending on an energy of the audio signal in the frequency band, so as to obtain a sequence of vectors of energy values from the audio signal, a vector component being an energy value in a frequency band;

scaling the energy values to obtain a sequence of scaled vectors wherein scaling comprises taking the logarithm of the energy values; and

suppressing, downstream with respect to taking the logarithm, a steady component, using a high-pass filtering operation;

temporally low-pass filtering the sequence of scaled vectors to obtain a filtered sequence which represents the fingerprint signal, or from which the fingerprint signal may be derived; and

quantizing the filtered sequence or a signal based thereon so as to derive the fingerprint signal from the filtered sequence or the signal based thereon, wherein quantization levels on the grounds of an amplitude statistic are used, the quantization levels being adapted in accordance with the amplitude statistic of the signal to be quantized, which statistic includes a statement about a relative frequency of values of the signal to be quantized, a fine classification of the quantizing levels being effected for a range of values with values of the signal to be quantized having a high relative abundance, and a coarse classification of the quantization levels being effected for a range of values with values of the signal to be quantized having a low relative abundance.