MUSIC INFORMATION SEARCHING METHOD AND APPARATUS THEREOF

• US 20140135964A1
• Filed: 11/13/2013
• Published: 05/15/2014
• Est. Priority Date: 11/13/2012
• Status: Active Grant

• Current Assignee: Kwangwoon University Industry, Samsung Electronics Co. Ltd.
• Original Assignee: Kwangwoon University Industry, Samsung Electronics Co. Ltd.
• Inventors: EOM, Ki-wan, KIM, Hyoung-Gook, KIM, Kwang-ki
• Granted Patent: US 9,659,092 B2
• Time in Patent Office: Days
• US Class Current: 700/94
• CPC Class Codes: G06F 16/61 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: G10L 25/54 for retrieval

Abstract

A music information searching method includes extracting modulating spectrums from audio data, generating modulating spectrum peak point audio fingerprints by using position information which relates to preset peak points from the extracted modulating spectrums, converting the generated modulating spectrum peak point audio fingerprints into hash keys which indicate addresses of hash tables and hash values that are stored on the hash tables via hash functions, and searching music information by extracting hash keys which relate to audio query clips and comparing the extracted hash keys with the indicated addresses of the hash tables.

Claims

1. A music information searching method, comprising:

extracting modulating spectrums from audio data;

generating modulating spectrum peak point audio fingerprints by using position information which relates to preset peak points from the extracted modulating spectrums;

converting the generated modulating spectrum peak point audio fingerprints into hash keys which indicate addresses of hash tables and hash values that are stored on the hash tables via at least one hash function; and

extracting hash keys which relate to audio query clips, and searching music information by comparing the extracted hash keys with the indicated addresses of the hash tables.

2. A music information searching method, comprising:

extracting modulating spectrums from audio data;

normalizing, via a modulating spectrum normalization process, the extracted modulating spectrums in order to delete overlapping with unnecessary areas that do not indicate unique features of audio signals from the extracted modulating spectrums;

distinguishing position information which relates to first modulating spectrum peak points which are weaker than a first threshold with respect to noisy environments from second modulating spectrum peak points which are stronger than the first threshold with respect to the noisy environments, and extracting the second modulating spectrum peak points;

converting position information which relates to the extracted second modulating spectrum peak points into modulating spectrum peak point audio fingerprints;

converting, by using at least one hash function, identification numbers which are respectively allocated to the modulating spectrum peak point audio fingerprints and music songs into hash keys that indicate addresses of hash tables and hash values that are stored on the hash tables, and storing a result of the converting on the hash tables;

extracting modulating spectrums from audio query clips;

normalizing, via the modulating spectrum normalization process, the extracted audio query modulating spectrums in order to delete overlapping with unnecessary areas that do not indicate unique features of audio query signals from the extracted audio query modulating spectrums;

distinguishing position information which relates to first audio query modulating spectrum peak points which are weaker than a second threshold with respect to the noisy environments from second audio query modulating spectrum peak points which are stronger than the second threshold with respect to the noisy environments from among the normalized audio query modulating spectrums, and extracting the second audio query modulating spectrum peak points;

converting position information which relates to the extracted second audio query modulating spectrum peak points into audio fingerprints which relate to the second audio query modulating spectrum peak points;

extracting position information which relates to the second audio query modulating spectrum peak points from among the extracted modulating spectrums of audio query clips;

converting position information which relates to the second extracted modulating spectrum peak points into modulating spectrum peak point audio fingerprints;

obtaining hash values of hash tables which respectively correspond to the addresses by calculating, by using at least one hash function, hash keys from the extracted modulating spectrum peak point audio fingerprints of audio query clips; and

determining final searching results by obtaining audio indexing information from the obtained hash values.

3. The music information searching method of claim 2, wherein the extracting the modulating spectrums from the audio data comprises extracting spectrum coefficients of the audio data by performing a fast Fourier transform (FFT), and converting the extracted spectrum coefficients into modulating spectrums via at least one modulating function.

4. The music information searching method of claim 2, wherein the normalizing the modulating spectrums comprises using at least one normalizing method from among a Zero-Mean normalization, a Cepstral Mean normalization, a Decibel-scale normalization, a normalization by using average values, a normalization by using medium values, and a Quantile normalization.

5. The music information searching method of claim 2, wherein the extracting the modulating spectrum peak points comprises using at least one from among a finite impulse response filter, an infinite impulse response filter, a Kalman filter, a spectrum deduction, a method for calculating minimum components of modulating spectrums and calculating modulating spectrum peak points by using the calculated minimum components, a method for extracting distinguished peak points from noise components by using adaptive threshold values which are calculable by using a highest peak point of the modulating spectrums, and a method for deleting noises by using a zero crossing rate and energy.

6. The music information searching method of claim 2, wherein the converting into the audio query modulating spectrum peak point audio fingerprints comprises obtaining position information which exists within preset intervals between the distinguished second modulating spectrum peak points by using values which indicate positions of the extracted second modulating spectrum peak points as modulating spectrum peak point audio fingerprints.

7. The music information searching method of claim 2, wherein the storing on the hash tables comprises:

calculating hash keys which indicate respective addresses of the hash tables and hash values which relate to the hash tables by using information which relates to the generated modulating spectrum peak point audio fingerprints via at least one hash function; and

generating the hash tables by storing the modulating spectrum peak point audio fingerprints on the hash tables based on the calculated hash keys and hash values.

8. The music information searching method of claim 2, wherein the extracting the modulating spectrums from the audio query clips comprises:

extracting spectrum coefficients by performing a respective FFT of each of the audio data and the audio query clip data; and

converting the extracted spectrum coefficients into the modulating spectrums via at least one modulating function.

9. The music information searching method of claim 2, wherein the extracting the position information which relates to the second audio query modulating spectrum peak points from among the extracted modulating spectrums of audio query clips comprises using at least one from among a finite impulse response filter, an infinite impulse response filter, a Kalman filter, a spectrum deduction, a method for calculating minimum components of modulating spectrums and calculating modulating spectrum peak points by using the calculated minimum components, a method for extracting distinguished peak points from noise components by using adaptive threshold values which are calculable by using a highest peak point of the modulating spectrums, and a method for deleting noises by using a zero crossing rate and energy.

10. The music information searching method of claim 2, wherein the converting into the modulating spectrum peak point audio fingerprints comprises obtaining position information which relates to two points by using values which indicate positions of the extracted modulating spectrum peak points as modulating spectrum peak point audio fingerprints.

11. The music information searching method of claim 2, wherein the obtaining the hash values of the hash tables comprises:

obtaining hash keys which indicate respective addresses of the hash tables by using information which relates to the generated modulating spectrum peak point audio fingerprints of audio query clips via at least one hash function; and

obtaining the hash values by using the obtained hash keys.

12. The music information searching method of claim 2, wherein the determining the final searching results comprises:

converting the hash values into audio indexing information by using the at least one hash function; and

determining a maximum amount of audio indexing information as final searching audio information from among the obtained audio indexing information.

13. An audio information searching method, comprising:

extracting modulating spectrums from audio data;

generating audio fingerprints by using position information which relates to preset peak points from the extracted modulating spectrums;

comparing information relating to the generated audio fingerprints with information relating to at least one user-selected audio query; and

determining a search result based on a result of the comparing.

14. The audio information searching method of claim 13, further comprising:

applying at least one hash function to the generated audio fingerprints in order to obtain the information relating to the generated audio fingerprints, and

applying the at least one hash function to the at least one user-selected audio query in order to obtain the information relating to the at least one user-selected audio query.

15. The audio information searching method of claim 13, wherein the information relating to at least one user-selected audio query is stored in a database, and the comparing comprises comparing the information relating to the generated audio fingerprints with information which is stored in the database.

16. A non-transitory computer-readable storage medium storing a program which is executable by a computer for performing the method of claim 13.

17. An apparatus for performing an audio information search, comprising:

an extractor which is configured to extract modulating spectrums from audio data;

an audio fingerprint generator which is configured to generate audio fingerprints by using position information which relates to preset peak points from the extracted modulating spectrums; and

an audio data searcher which is configured to compare information relating to the generated audio fingerprints with information relating to at least one user-selected audio query, and to determine a search result based on a result of the comparing.

18. The apparatus of claim 17, wherein the audio data searcher includes a hash value generator which is configured to:

apply at least one hash function to the generated audio fingerprints in order to obtain the information relating to the generated audio fingerprints, and

apply the at least one hash function to the at least one user-selected audio query in order to obtain the information relating to the at least one user-selected audio query.

19. The apparatus of claim 17, wherein the information relating to at least one user-selected audio query is stored in a database, and wherein the audio data searcher is further configured to compare the information relating to the generated audio fingerprints with information which is stored in the database.