Similar music search method and apparatus using music content summary

• US 7,626,111 B2
• Filed: 07/17/2006
• Issued: 12/01/2009
• Est. Priority Date: 01/26/2006
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Ji Yuen, Kim, Hyoung Gook, Eom, Ki Wan
• Primary Examiner(s): Warren; David S.
• Application Number:
  US11/487,399
• Publication Number:
  US 20070169613A1
• Time in Patent Office: 1,233 Days
• Field of Search: 84600-602, 700/94
• US Class Current: 84/600
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 2240/081 Genre classification, i.e. descriptive metadata for classification or selection of musical pieces according to style analysis genre G10H2210/036
• CPC Class Codes: G10H 2240/085 Mood, i.e. generation, detection or selection of a particular emotional content or atmosphere in a musical piece
• CPC Class Codes: G10H 2240/131 Library retrieval, i.e. searching a database or selecting a specific musical piece, segment, pattern, rule or parameter set
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A similar music search method, including: extracting an audio feature value from a compression zone of a music file and generating a music content summary using the extracted audio feature; classifying a mood and a genre of the music file by using the generated music content summary; extracting a similarity feature for a similarity retrieval from the music content summary; storing an information on the classified mood and a genre and the extracted similarity feature information in a database; receiving a query music; detecting the mood and the genre of the query music; measuring a similarity between the query music and music files having a mood and a genre identical to the query music by referring to the database; and retrieving the similar music with respect to the query music based on the measured similarity.

Claims

1. A similar music search method, comprising:

extracting an audio feature value from a music file by decoding only a select compressed portion of the music file of all compressed portions of the music file, wherein the select compressed portion includes the extracted audio feature value, extracting the audio feature from the decoded portion, and generating a music content summary using the extracted audio feature;

classifying a mood and a genre of the music file using the generated music content summary;

extracting a similarity feature for a similarity retrieval from the music content summary;

storing information on the classified mood and genre and the extracted similarity feature information in a database;

receiving a query music;

detecting a mood and a genre of the query music;

measuring a similarity between the query music and music files that are identical in the mood and the genre to the query music by referring to the database; and

retrieving the similar music with respect to the query music based on the measured similarity.

2. The method of claim 1, wherein the extracting an audio feature value from the music file and generating a music content summary includes:

reconfiguring a segment by tracking a point at which music contents change using the extracted feature value;

measuring the similarity and a repeatability between each segment by clustering a fragment with a certain length acquired from each reconfigured segment; and

generating the music content summary using the segment selected by the measured similarity and repeatability between each segment.

3. The method of claim 1, wherein the classifying a mood and a genre of the music file using the generated music content summary includes:

extracting a modified discrete cosine transformation (MDCT)-based timbre feature from the music content summary;

extracting an MDCT-based tempo feature from the music content summary; and

classifying the mood and the genre of the music file based on the extracted timbre feature tempo feature.

4. The method of claim 3, wherein the extracting an MDCT-based timbre feature from the music content summary includes:

extracting MDCT coefficients by decoding a part of the music content summary;

selecting a predetermined number of MDCT coefficients of a sub-band from the extracted MDCT coefficients; and

extracting a spectral centroid, a bandwidth, a rolloff, a flux, and a flatness from the selected MDCT coefficients.

5. The method of claim 4, wherein the extracting a similarity feature for similarity retrieval from the music content summary includes computing a maximum, a mean, and a standard deviation of the extracted timbre feature.

6. The method of claim 3, wherein the extracting a MDCT-based tempo feature from the music content summary includes:

extracting the MDCT coefficients by decoding a part of the music content summary;

selecting an MDCT coefficient of a predetermined sub-band from the MDCT coefficients;

extracting an MDCT modulation spectrum (MS) from the selected MDCT coefficient by performing a Discrete Fourier Transformation (DFT); and

dividing the extracted MDCT-MS into N sub-bands and extracting energy from the sub-bands usable as an MDCT-MS-based tempo feature.

7. The method of claim 6, wherein, in the extracting a similarity feature for a similarity retrieval from the music content summary, a centroid, a bandwidth, a flux, and a flatness according to the MDCT-MS-based tempo feature are extracted as the similarity features.

8. The method of claim 1, wherein, in the measuring a similarity between the query music and music files identical in the mood and the genre to the query music by referring to the database, the similarity is measured by computing a Euclidean distance of the features of the music file that are identical in the mood and the genre with the query music.

9. The method of claim 8, wherein, in the retrieving the similar music with respect to the query music based on the measured similarity, a number of the music files with a computed Euclidean distance is less than a predetermined value.

10. A computer-readable recording medium in which a program for executing a similar music search method is recorded, the method comprising:

extracting an audio feature value from a music file by decoding only a select compressed portion of the music file of all compressed portions of the music file, wherein the select compressed portion includes the extracted audio feature value, extracting the audio feature from the decoded portion, and generating a music content summary using the extracted audio feature;

classifying a mood and a genre of the music file using the generated music content summary;

extracting a similarity feature for a similarity retrieval from the music content summary;

storing information on the classified mood and genre and the extracted similarity feature information in a database;

receiving a query music from a user;

detecting a mood and a genre of the query music;

measuring a similarity between the query music and music files that are identical in the mood and the genre to the query music by referring to the database; and

retrieving the similar music with respect to the query music based on the measured similarity.

11. An apparatus for retrieving a similar music by using a music content summary, the apparatus comprising:

a music content summary generation unit extracting an audio feature value from a music file by decoding only a select compressed portion of the music file of all compressed portions of the music file, wherein the select compressed portion includes the extracted audio feature value, extracting the audio feature from the decoded portion, and generating the music content summary using the extracted audio feature value;

a first feature extraction unit extracting an MDCT-based timbre feature and an MDCT-based tempo feature from the generated music content summary;

a mood/genre classification unit classifying a mood and a genre of the music file according to the feature extracted by the first feature extraction unit;

a second feature extraction unit extracting a similarity feature for a similarity retrieval from the music content summary;

a database storing information on the classified mood and genre and the extracted similarity feature information;

a music input unit receiving a query music;

query music detection unit detecting a mood and a genre of the query music; and

a retrieval unit measuring a similarity between the query music and music files that are identical in the mood and the genre to the query music and retrieving the similar music with respect to the query music according to the measured similarity.

12. The apparatus of claim 11, wherein the mood/genre classification unit classifies the mood and the genre of the music file based on the extracted timbre feature and the tempo feature.

13. The apparatus of claim 11, wherein:

the music file includes tag information including genre information; and

the mood/genre classification unit classifies the genre of the music file using the genre information of the tag information extracted from the music file.

14. The apparatus of claim 11, wherein:

the second feature detection unit computes a maximum, a mean, and a standard deviation with respect to the extracted feature; and

the database stores the computed maximum, the computed mean, and the computed standard deviation as metadata.

15. The apparatus of claim 14, wherein the retrieval unit searches the similar music with respect to the query music by using the maximum, the mean, and the standard deviation, from the music files that are identical in the mood and the genre with the retrieved query music.

16. The apparatus of claim 15, wherein the retrieval unit computes a Euclidean distance with respect to the maximum, the mean, and the standard deviation and retrieves candidates with a computed distance less than a predetermined value as the similar music.

17. The apparatus of claim 11, wherein the music file includes any one of a moving picture experts group audio layer-3 (MP3) file and an advanced audio coding (ACC) file.

18. A similar music search method, comprising:

classifying a mood and a genre of a music file using a music content summary generated based on an audio feature value extracted from the music file by decoding only a select compressed portion of the music file of all compressed portions of the music file, wherein the select compressed portion includes the extracted audio feature value and the extracted audio feature value is a Modified Discrete Cosine Transformation-based feature;

extracting a similarity feature for a similarity retrieval from the music content summary;

storing information on the classified mood and genre and an extracted similarity feature from the music content summary;

detecting a mood and a genre of an input query music;

measuring a similarity between the query music and music files that are identical in mood and genre to the query music by referring to the database; and

retrieving the similar music with respect to the query music based on the measured similarity.

19. A computer-readable storage medium in which a program for executing the similar music search method of claim 18 is recorded.

20. A music contents summary generation method, the method comprising:

extracting an audio feature value from a music file;

reconfiguring segments of the music file by tracking a music content change point using the extracted feature values;

first clustering the reconfigured segments using a Bayesian Information Criterion (BIC) method, and measuring a similarity and a repeatability between the reconfigured segments;

extracting an MDCT-based timbre feature and an MDCT-MS-based tempo feature from each of the reconfigured segments;

second clustering the reconfigured segments by calculating a Euclidean distance for the extracted timbre and tempo features and measuring the similarity and the repeatability between the reconfigured segments, the reconfigured segments being second clustered according to the similarity values based on the Euclidean distances; and

determining the similarity and the repeatability between the segments based on a result of a comparison between the first clustering a the second clustering.

21. The method of claim 20, wherein the determining includes determining the similarity and the repeatability between the reconfigured segments using a consistent part of the segments, when there is a consistent part.

22. The method of claim 20, wherein the determining includes determining the similarity and the repeatability between the reconfigured segments according to the second clustering result, when there is no consistent part.