Device, method, and medium for generating audio fingerprint and retrieving audio data

• US 20070112565A1
• Filed: 11/13/2006
• Published: 05/17/2007
• Est. Priority Date: 11/11/2005
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Jiyeun, Zhu, Xuan, She, Yuan, Kim, Hyounggook, Eom, Kiwan
• Granted Patent: US 8,380,518 B2
• Time in Patent Office: Days
• US Class Current: 704/229
• CPC Class Codes: G06Q 30/0276 Advertisement creation
• CPC Class Codes: G10L 19/0212 using orthogonal transformation
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

Provided are device, method, and medium for generating an audio fingerprint and retrieving audio data. The device for generating an audio fingerprint includes: a coefficient extracting section partially decoding audio data in a compression area and extracting MDCT (Modified Discrete Cosine Transform) coefficients; a coefficient selecting section selecting an MDCT coefficient robust to noises from the extracted MDCT coefficients; a modulation spectrum generating section transforming the selected MDCT coefficient by the use of a Fourier transform method and generating a modulation spectrum; and a bit conversion section quantizing the generated modulation spectrum and generating an audio fingerprint. As a result, it is possible to accurately and rapidly retrieve the audio data recorded in a variety of environments. Since elements based on MP3 are used, it is possible to apply to MP3 applications in various manners. In addition, it is possible to apply to classification of audio data such as classification of music moods and classification of music genres and various other fields such as extraction of a specific event from moving images of sports.

Claims

1. A device for generating an audio fingerprint, the device comprising:

a coefficient extracting section to partially decode audio data in a compression area and to extract MDCT (Modified Discrete Cosine Transform) coefficients;

a coefficient selecting section to select an MDCT coefficient robust to noises from the extracted MDCT coefficients;

a modulation spectrum generating section to process the selected MDCT coefficient and to generate a modulation spectrum; and

a bit conversion section to quantize the generated modulation spectrum and to generate an audio fingerprint.

2. The device of claim 1, wherein the coefficient selecting section selects the MDCT coefficient having a stable and low bit error rate from the extracted MDCT coefficients.

3. The device of claim 1, wherein the coefficient selecting section selects the MDCT coefficient corresponding to a low frequency domain from the extracted MDCT coefficients.

4. The device of claim 1, wherein the coefficient selecting section sets the number of MDCT coefficients to be selected in accordance with the data size of the audio fingerprint.

5. The device of claim 1, wherein the modulation spectrum generating section generates the modulation spectrum by transforming the selected MDCT coefficient by the use of a Fourier transform method.

6. The device of claim 5, wherein the modulation spectrum generating section sets the number of modulation frequencies to be selected in accordance with the data size of the audio fingerprint.

7. The device of claim 5, wherein the modulation spectrum generating section determines the magnitude of the Fourier transform in accordance with the length of the audio data.

8. A device for generating an audio fingerprint, the device comprising:

a coefficient extracting section to partially decode audio data in a compression area and to extract MDCT (Modified Discrete Cosine Transform) coefficients;

a synthesization section to synthesize PPF (PolyPhase Filter) sub bands from the extracted MDCT coefficients;

a band selecting section to select a PPF sub band robust to noises from the synthesized PPF sub bands;

a modulation spectrum generating section to process the selected PPF sub band and to generate a modulation spectrum; and

a bit conversion section to quantize the generated modulation spectrum and to generate an audio fingerprint.

9. The device of claim 8, wherein the band selecting section selects the PPF sub band having a stable and low bit error rate from the synthesized PPF bands.

10. The device of claim 8, wherein the band selecting section selects the PPF sub band corresponding to a low frequency domain from the synthesized PPF bands.

11. The device of claim 8, wherein the band selecting section sets the number of PPF sub bands to be selected in accordance with the data size of the audio fingerprint.

12. The device of claim 8, wherein the modulation spectrum generating section generates the modulation spectrum by transforming the selected PPF sub band by the use of a Fourier transform method.

13. The device of claim 12, wherein the modulation spectrum generating section sets the number of modulation frequencies to be selected in accordance with the data size of the audio fingerprint.

14. The device of claim 12, wherein the modulation spectrum generating section determines the magnitude of the Fourier transform in accordance with the length of the audio data.

15. A device for retrieving audio data, which corresponds to an audio fingerprint generated by partially decoding the audio data in a compression area, selecting an MDCT (Modified Discrete Cosine Coefficient) coefficient or a PPF (PolyPhase Filter) sub band, generating a modulation spectrum by transforming the selected MDCT coefficient or PPF sub band by the use of a Fourier transform method, and quantizing the generated modulation spectrum, the device comprising:

an audio fingerprint storage section to convert the audio data into an audio fingerprint and to store the audio fingerprint;

a hashing table to store a hashing value corresponding to the stored audio fingerprint;

a hashing value generating section to extract an indexing bit from the generated audio fingerprint and to generate a hashing value;

a table searching section to search the hashing table for the hashing value matched with the generated hashing value;

a BER calculating section to calculate a BER (Bit Error Ratio) between the generated audio fingerprint and the audio fingerprint corresponding to the searched hashing value;

a comparison section to compare the calculated BER with a threshold value; and

an audio fingerprint detecting section to detect the audio fingerprint in accordance with the comparison result.

16. The device of claim 15, further comprising a threshold value adjusting section to adjust the threshold value in accordance with the detection result.

17. The device of claim 15, wherein the hashing value generating section extracts the indexing bit not having a continuous bit error from the generated audio fingerprint.

18. The device of claim 15, further comprising:

an information storage section to store information on the audio data corresponding to the stored audio fingerprint; and

an information providing section to read information on the audio data corresponding to the detected audio fingerprint from the information storage section and to provide the read information.

19. A method of generating an audio fingerprint, the method comprising:

partially decoding audio data in a compression area and extracting MDCT (Modified Discrete Cosine Transform) coefficients;

selecting an MDCT coefficient robust to noises from the extracted MDCT coefficients;

processing the selected MDCT coefficient and generating a modulation spectrum; and

quantizing the generated modulation spectrum and generating an audio fingerprint.

20. The method of claim 19, wherein in the selecting an MDCT coefficient, the MDCT coefficient having a stable and low bit error rate is selected from the extracted MDCT coefficients.

21. The method of claim 19, wherein in the selecting an MDCT coefficient, the MDCT coefficient corresponding to a low frequency domain is selected from the extracted MDCT coefficients.

22. The method of claim 19, wherein in the selecting an MDCT coefficient, the number of MDCT coefficients to be selected is set in accordance with the data size of the audio fingerprint.

23. The method of claim 19, wherein in the generating a modulation spectrum, the modulation spectrum is generated by transforming the selected MDCT coefficient by the use of a Fourier transform method.

24. The method of claim 23, wherein in the generating a modulation spectrum, the number of modulation frequencies to be selected is set in accordance with the data size of the audio fingerprint.

25. The method of claim 23, wherein in the generating a modulation spectrum, the magnitude of the Fourier transform is determined in accordance with the length of the audio data.

37. At least one medium comprising computer readable instructions for implementing the method of claim 19.

26. A method of generating an audio fingerprint, the method comprising:

partially decoding audio data in a compression area and extracting MDCT (Modified Discrete Cosine Transform) coefficients;

synthesizing PPF (PolyPhase Filter) sub bands from the extracted MDCT coefficients;

selecting a PPF sub band robust to noises from the synthesized PPF sub bands;

processing the selected PPF sub band and generating a modulation spectrum; and

quantizing the generated modulation spectrum and generating an audio fingerprint.

27. The method of claim 26, wherein in the selecting a PPF sub band, the PPF sub band having a stable and low bit error rate is selected from the synthesized PPF bands.

28. The method of claim 26, wherein in the selecting a PPF sub band, the PPF sub band corresponding to a low frequency domain is selected from the synthesized PPF bands.

29. The method of claim 26, wherein in the selecting a PPF sub band, the number of PPF sub bands to be selected is set in accordance with the data size of the audio fingerprint.

30. The method of claim 26, wherein in the generating a modulation spectrum, the modulation spectrum is generated by transforming the selected PPF sub band by the use of a Fourier transform method.

31. The method of claim 30, wherein in the generating a modulation spectrum, the number of modulation frequencies to be selected is set in accordance with the data size of the audio fingerprint.

32. The method of claim 30, wherein in the generating a modulation spectrum, the magnitude of the Fourier transform is determined in accordance with the length of the audio data.

38. At least one medium comprising computer readable instructions for implementing the method of claim 26.

33. A method of retrieving audio data, which corresponds to an audio fingerprint generated by partially decoding the audio data in a compression area, selecting an MDCT (Modified Discrete Cosine Coefficient) coefficient or a PPF (PolyPhase Filter) sub band, generating a modulation spectrum by transforming the selected MDCT coefficient or PPF sub band by the use of a Fourier transform method, and quantizing the generated modulation spectrum, the method comprising:

extracting an indexing bit from the generated audio fingerprint and generating a hashing value;

retrieving a hashing value matched with the generated hashing value from a hashing table storing a hashing value corresponding to an audio fingerprint stored in a storage medium;

calculating a BER (Bit Error Ratio) between the generated audio fingerprint and the audio fingerprint corresponding to the retrieved hashing value;

comparing the calculated BER with a threshold value; and

detecting the audio fingerprint in accordance with the comparison result.

34. The method of claim 33, further comprising adjusting the threshold value in accordance with the detection result.

35. The method of claim 33, wherein in the generating a hashing value, the indexing bit not having a continuous bit error is extracted from the generated audio fingerprint.

36. The method of claim 33, further comprising:

providing information on the audio data corresponding to the detected audio fingerprint to a user.

39. At least one medium comprising computer readable instructions for implementing the method of claim 33.

40. A method of generating an audio fingerprint, the method comprising:

selecting an MDCT (Modified Discrete Cosine Transform) coefficient robust to noises from MDCT coefficients extracted from audio data;

processing the selected MDCT coefficient and generating a modulation spectrum; and

quantizing the generated modulation spectrum and generating an audio fingerprint.

41. At least one medium comprising computer readable instructions for implementing the method of claim 40.

42. A method of generating an audio fingerprint, the method comprising:

synthesizing PPF (PolyPhase Filter) sub bands from MDCT (Modified Discrete Cosine Transform) coefficients extracted from audio data;

selecting a PPF sub band robust to noises from the synthesized PPF sub bands;

processing the selected PPF sub band and generating a modulation spectrum; and

quantizing the generated modulation spectrum and generating an audio fingerprint.

43. At least one medium comprising computer readable instructions for implementing the method of claim 42.