System and methods for providing automatic classification of media entities according to melodic movement properties

• US 20060096447A1
• Filed: 12/21/2005
• Published: 05/11/2006
• Est. Priority Date: 08/29/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Weare, Christopher B., Hoekman, Jeffrey S.
• Granted Patent: US 7,574,276 B2
• Time in Patent Office: Days
• US Class Current: 84/616
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/635 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/639 using playlists
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content

Abstract

In connection with a classification system for classifying media entities that merges perceptual classification techniques and digital signal processing classification techniques for improved classification of media entities, a system and methods are provided for automatically classifying and characterizing melodic movement properties of media entities. Such a system and methods may be useful for the indexing of a database or other storage collection of media entities, such as media entities that are audio files, or have portions that are audio files. The methods also help to determine media entities that have similar, or dissimilar as a request may indicate, melodic movement by utilizing classification chain techniques that test distances between media entities in terms of their properties. For example, a neighborhood of songs may be determined within which each song has similar melodic movement properties.

Claims

1. A method to be performed by a computer for automatically classifying melodic movement properties of a media entity, comprising:

applying a peak detection process to the media entity;

detecting a location of at least one prominent peak;

determining an energy of the at least one prominent peak;

storing the location of the at least one prominent peak and the energy of the at least one prominent peak into at least one output matrix;

applying the output matrix data stored in said at least one output matrix to a melodic movement vector calculation process that determines pitch class movement data for the media entity.

2. A method according to claim 1, wherein the media entity is divided into frames, and the method is performed frame by frame and wherein the number of peaks detected in said application of the peak detection process is limited by a pre-defined parameter.

3. A method according to claim 1, further comprising performing Nth order interpolation on at least one of the location of the at least one prominent peak and the energy of the at least one prominent peak to increase precision.

4. A method according to claim 1, further comprising applying the melodic movement vector to a classification stage which determines at least one of (1) at least one melodic movement value and (2) at least one melodic movement class that describes the media entity.

5. A method according to claim 1, wherein the pitch class movement data is stored into a melodic movement vector that is 1×

24.

6. A method according to claim 1, wherein the media entity is divided into frames, and the method is performed frame by frame and wherein the peak continuation process keeps track of peaks that last more than a predetermined number of frames.

7. A method according to claim 1, wherein said critical band masking filtering removes a peak that is masked by surrounding peaks with more energy.

8. A method according to claim 1, wherein said storing includes providing an output of the peak detection and interpolation stage in two matrices, one holding the location of the at least one prominent peak, and the second holding the respective energy of the at least one prominent peak.

9. The method of claim 1, further comprising converting input audio data of the media entity from the time domain to the frequency domain.

10. At least one computing device comprising means for performing the method of claim 1.

11. A method for classifying songs according to melodic movement properties of the songs, comprising:

assigning to each song of a plurality of songs in a data set to at least one melodic movement class;

processing each song of said data set to extract at least one melodic movement characteristic based on digital signal processing of each song;

generating a plurality of melodic movement properties vectors for said plurality of songs, wherein each melodic movement properties vector includes said at least one melodic movement class and said at least one melodic movement characteristic based on digital signal processing; and

forming a classification chain based upon said plurality of feature vectors.

12. A method according to claim 11, further comprising:

processing an unclassified song to extract at least one melodic movement characteristic based on digital signal processing of the unclassified song;

generating a vector for the unclassified song including said at least one melodic movement characteristic;

presenting the vector for the unclassified song to the classification chain; and

classifying the unclassified song with an estimate of the melodic movement class by calculating the representative melodic movement characteristic of the subset of the plurality of vectors of the classification chain located in the neighborhood of the vector for the unclassified song.

13. A method according to claim 12, further including calculating a neighborhood distance that defines a distance within which two vectors in the classification chain space are in the same neighborhood for purposes of being in the same melodic movement class.

14. A method according to claim 12, wherein said classifying of the unclassified song includes classifying the unclassified song with a median melodic movement class represented by the neighborhood.

15. A method according to claim 12, wherein said melodic movement class is described by a numerical value and said classifying of the unclassified song includes classifying the unclassified song with a mean of numerical melodic movement properties values found in the neighborhood.

16. A method according to claim 12, wherein said classifying includes returning at least one number indicating the level of confidence of the melodic movement class estimate.

17. At least one computing device comprising means for performing the method of claim 11.

18. A computing system, comprising:

a computing device including;

a classification chain data structure stored thereon having a plurality of classification vectors, wherein each vector includes data representative of a melodic movement class as classified by humans and melodic movement characteristics as determined by digital signal processing; and

processing means for comparing an unclassified song to the classification chain data structure to determine an estimate of the melodic movement class of the unclassified song.

19. A computing system according to claim 18, wherein said determining of an estimate of the melodic movement class includes returning at least one number indicating the level of confidence of the melodic movement class assignment.

20. A method according to claim 19, wherein the performance level of the classification chain improves over time due to the examination of unclassified songs that have a low confidence level associated with the melodic movement class assignment.