Extraction and Matching of Characteristic Fingerprints from Audio Signals

• US 20090157391A1
• Filed: 02/24/2009
• Published: 06/18/2009
• Est. Priority Date: 09/01/2005
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Bilobrov, Sergiy
• Granted Patent: US 8,396,705 B2
• Time in Patent Office: Days
• US Class Current: 704/200.100
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

An audio fingerprint is extracted from an audio sample, where the fingerprint contains information that is characteristic of the content in the sample. The fingerprint may be generated by computing an energy spectrum for the audio sample, resampling the energy spectrum logarithmically in the time dimension, transforming the resampled energy spectrum to produce a series of feature vectors, and computing the fingerprint using differential coding of the feature vectors. The generated fingerprint can be compared to a set of reference fingerprints in a database to identify the original audio content.

Claims

1. An article of manufacture comprising an audio fingerprint stored on a computer readable storage medium, wherein the audio fingerprint contains characteristic information about an audio frame and is produced by a process comprising:

filtering the audio frame into a plurality of frequency bands to produce a corresponding plurality of filtered audio signals;

resampling the filtered audio signals at a nonlinear timescale;

transforming the resampled audio signals for each frequency band to produce a feature vector for the frequency band; and

computing the audio fingerprint based on the set of feature vectors.

2. The article of manufacture of claim 1, wherein filtering the audio frame into a plurality of frequency bands comprises band pass filtering the audio frame in each of the plurality of frequency bands.

3. The article of manufacture of claim 1, wherein filtering the audio frame into a plurality of frequency bands comprises performing a Fast Fourier Transform (FFT) on the audio sample.

4. The article of manufacture of claim 1, wherein the audio frame is part of an audio file stored in an MP3 format, and the filtered audio signals are obtained from an MP3 hybrid filterbank associated with the audio file.

5. The article of manufacture of claim 1, wherein the filtered audio signals are resampled at a logarithmic timescale.

6. The article of manufacture of claim 1, wherein the frequency bands are spaced linearly in a frequency axis.

7. The article of manufacture of claim 1, wherein the frequency bands overlap.

8. The article of manufacture of claim 1, wherein transforming the resampled filtered audio signal of a particular frequency band comprises performing a Fast Fourier Transform (FFT) on the resampled audio signal.

9. The article of manufacture of claim 1, wherein computing the audio fingerprint comprises differentially encoding the feature vectors for the frequency bands.

10. The article of manufacture of claim 1, wherein the process further comprises:

computing an index value for the audio fingerprint, the index value comprising a portion of the audio fingerprint.

11. The article of manufacture of claim 10, wherein the index value comprises a portion of the audio fingerprint that corresponds to a set of low frequency components of the transformed audio signals.

12. The article of manufacture of claim 1, wherein the process further comprises:

disregarding a portion of the audio fingerprint, where the disregarded portion of the audio fingerprint corresponds to a frequency range determined to be insignificant according to an acoustic model.

13. The article of manufacture of claim 12, wherein the acoustic model is a psychoacoustic model.

14. The article of manufacture of claim 12, wherein the acoustic model mimics the properties of an audio encoding process.

15. The article of manufacture of claim 12, wherein the acoustic model mimics the properties of an environment.

16. The article of manufacture of claim 12, wherein the acoustic model mimics the properties of an audio signal.

17. The article of manufacture of claim 1, wherein transforming the resampled filtered audio signal of a particular frequency band comprises performing a Discrete Cosine Transform (DCT) on the resampled audio signal.

18. The article of manufacture of claim 1, wherein the frequency bands have a logarithmic mid-frequency distribution in a frequency axis.

19. The article of manufacture of claim 1, wherein the filtered audio signals are resampled at an exponential timescale.

20. The article of manufacture of claim 1, wherein computing the audio fingerprint comprises encoding the feature vectors for the frequency bands by assigning bit values according to a codebook table.

21. The article of manufacture of claim 20, wherein the process further comprises:

training the codebook table to determine a set of codebook values; and

tuning the codebook table based on the determined set of codebook values.

22. A database of audio fingerprints, the database comprising a computer readable storage medium that contains:

a plurality of audio fingerprints, wherein each audio fingerprint contains characteristic information about a corresponding audio frame and is produced by a process comprising;

filtering the corresponding audio frame into a plurality of frequency bands to produce a corresponding plurality of filtered audio signals,

resampling the filtered audio signals at a nonlinear timescale,

transforming the resampled audio signals for each frequency band to produce a feature vector for the frequency band, and

computing the audio fingerprint based on the set of feature vectors; and

one or more index values for one or more of the audio fingerprints, wherein the audio fingerprints are organized in the database according to their index values.

23. The database of claim 22, wherein each index value comprises a portion of an audio fingerprint associated with the index value.

24. An article of manufacture comprising an audio fingerprint stored on a computer readable storage medium, wherein the audio fingerprint contains characteristic information about an audio frame and is produced by a process comprising:

a step for computing a spectrogram for the audio frame;

sampling the spectrogram at a nonlinear time scale for a plurality of frequency bands in the spectrogram;

a step for extracting a long-term feature vector using the samples from each of the sampled frequency bands; and

a step for generating the audio fingerprint based on the feature vectors.

25. The article of manufacture of claim 24, wherein the spectrogram is sampled at a logarithmic timescale.