System and methods for providing automatic classification of media entities according to tempo properties

• US 6,657,117 B2
• Filed: 07/13/2001
• Issued: 12/02/2003
• Est. Priority Date: 07/14/2000
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Weare, Christopher B., Daskalovic, Marc
• Primary Examiner(s): Fletcher, Marlon T.
• Application Number:
  US09/905,345
• Publication Number:
  US 20020037083A1
• Time in Patent Office: 872 Days
• Field of Search: 846/00--06, 846/09--12, 846/15--18, 846/34--36, 846/49--56, 846/66--68
• US Class Current: 84/668
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/635 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/637 Administration of user profiles, e.g. generation, initialization, adaptation or distribution
• CPC Class Codes: G06F 16/639 using playlists
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 21/466 Learning process for intelligent management, e.g. learning user preferences for recommending movies services using the results of monitoring in broadcast systems H04H60/61
• CPC Class Codes: H04N 21/4665 involving classification methods, e.g. Decision trees

Abstract

In connection with a classification system for classifying media entities that merges perceptual classification techniques and digital signal processing classification techniques for improved classification of media entities, a system and methods are provided for automatically classifying and characterizing tempo properties of media entities. Such a system and methods may be useful for the indexing of a database or other storage collection of media entities, such as media entities that are audio files, or have portions that are audio files. The methods also help to determine media entities that have similar, or dissimilar as a request may indicate, tempo(s) by utilizing classification chain techniques that test distances between media entities in terms of their properties. For example, a neighborhood of songs may be determined within which each song has similar tempo characteristics.

Claims

1. A method for automatically classifying tempo properties of audio data, comprising:

reducing input audio data to a lower resolution;

applying an accentuation process to the input audio data to accentuate the tempo properties of the data and applying a smoothing process to smooth at least one of an aberration, bias and trend of the tempo properties of the input audio data;

generating a plurality of bins of data that correspond to at least one tempo property of the input audio data; and

generating a tempo property feature vector from said at least one tempo property.

2. A method according to claim 1, wherein said reducing includes decimating the input audio data to a fixed sampling rate.

3. A method according to claim 2, wherein said reducing includes decimating the input audio data to approximately a 43 Hz sampling rate.

4. A method according to claim 1, wherein said applying of the smoothing process includes applying a median filter to smooth said at least one of an aberration, bias and trend of the tempo properties of the input audio data.

5. A method according to claim 1, wherein said applying of the accentuation process and smoothing process includes first performing of a first order difference calculation followed by performing a first half wave rectification.

6. A method according to claim 5, wherein said applying of the accentuation process and smoothing process includes subtracting a mean value of the audio data followed by performing a second half wave rectification.

7. A method according to claim 6, wherein said applying of the accentuation process and smoothing process includes second of performing a second first order difference calculation followed by performing a third half wave rectification to generate said at least one tempo property.

8. A method according to claim 1, wherein said generating a plurality of bins of data that correspond to at least one tempo property of the input audio data includes generating a plurality of bins reflective of the different correlation strengths at different time periods or frequencies.

9. A method according to claim 8, wherein said generating a plurality of bins of data that correspond to at least one tempo property of the input audio data includes generating 130 bins.

10. A method according to claim 1, wherein said generating a plurality of bins of data that correspond to at least one tempo property of the input audio data includes looping through the audio data and accumulating the base two logarithm of the product of pairs of points located at a fixed offset from each other.

11. A method according to claim 10, wherein said looping is performed 130 times for offsets spanning from 1 to 130 samples resulting in said bins of data.

12. A method according to claim 10, wherein said tempo property feature vector includes said plurality of bins of data, and further comprising inputting said tempo property feature vector to a classification chain.

13. A method according to claim 1, further comprising applying the tempo property feature vector to a classification stage which determines at least one of (1) at least one tempo properties value and (2) at least one tempo properties class that describes the audio data.

14. A method according to claim 1, wherein the audio data is formatted according to pulse code modulated format.

15. A method according to claim 14, wherein the audio data is previously in a format other than pulse code modulated format, and the method further comprises converting the audio data to pulse code modulated format from the other format.

16. A computer readable medium bearing computer executable instructions for carrying out the method of claim 1.

17. A modulated data signal carrying computer executable instructions for performing the method of claim 1.

18. At least one computing device comprising means for performing the method of claim 1.

19. A method of classifying data according to tempo properties of the data, comprising:

assigning to each media entity of a plurality of media entities in a data set to at least one tempo properties class;

processing each media entity of said data set to extract at least one tempo properties characteristic based on digital signal processing of each media entity;

generating pluralities of tempo properties bins of data for said plurality of media entities, wherein each plurality of tempo properties bins includes said at least one tempo properties class and at least one tempo properties characteristic based on digital signal processing; and

forming a classification chain based upon said pluralities of tempo properties bins of data.

20. A method according to claim 19, further comprising:

processing an unclassified media entity to extract at least one tempo properties characteristic based on digital signal processing of the unclassified media entity;

presenting said at least one digital signal processing tempo properties characteristic for the unclassified media entity to the classification chain;

calculating an estimate of the tempo properties class via a comparison process between said at least one digital signal processing tempo properties characteristic and the classification chain; and

classifying the unclassified entry with the estimate of the tempo properties class.

21. A method according to claim 20, wherein said processing includes cycling through each of the points in the classification chain and generating a table of distance values and tempo values.

22. A method according to claim 21, wherein cycling through a point in the classification chain includes computing a plurality of distances, with each distance being computed between a dimension represented by the at least one digital signal processing tempo properties characteristic and a dimension represented by a point in the classification chain, and taking the square root of the result of summing the squares of the plurality of distances, to produce a distance for the point.

23. A method according to claim 22, wherein if said distance for the point is less than a threshold value, the distance for the point and the tempo value associated with the point is added to the table.

24. A method according to claim 21, wherein confidence values for the distance values in the table are calculated in proportion to the distance values, respectively.

25. A method according to claim 24, wherein a confidence value for a distance value is computed in exponential proportion to the distance value.

26. A method according to claim 24, wherein a tempo component and a harmonic component is computed for each of said tempo values in the table.

27. A method according to claim 26, wherein for each of said tempo values, the log base two of the tempo value divided by 10 is performed, and the integer portion of the result is the harmonic component and the fractional portion of the result is the tempo component.

28. A method according to claim 26, further including an averaging process wherein said tempo components are averaged to generate an intermediate estimated tempo class.

29. A method according to claim 28, wherein calculating the intermediate estimated tempo class includes mapping the tempo components to angular representations between 0 and 2π

, calculating the sine and cosine of the angular representations and multiplying the sine values and cosine values by the corresponding confidence values, calculating the inverse tangent of the division of the sum of the multiplied sine values by the sum of the multiplied cosine values and mapping the resulting angle to a value between zero and one.

30. A method according to claim 28, further including generating a harmonic component estimate of the unclassified media entity based on the intermediate estimated tempo class.

31. A method according to claim 30, wherein said estimated tempo class of the unclassified media entity is calculated based on said harmonic component estimate and said intermediate estimated tempo class.

32. A method according to claim 20, wherein said classifying of the unclassified entry with the estimate of the tempo properties class includes returning at least one number indicating an overall confidence of the estimate of the tempo properties class.

33. A computer readable medium bearing computer executable instructions for carrying out the method of claim 19.

34. A modulated data signal carrying computer executable instructions for performing the method of claim 19.

35. At least one computing device comprising means for performing the method of claim 19.