System and methods for providing adaptive media property classification

• US 8,082,279 B2
• Filed: 04/18/2008
• Issued: 12/20/2011
• Est. Priority Date: 08/20/2001
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Weare, Christopher B.
• Primary Examiner(s): LE, DEBBIE M
• Application Number:
  US12/105,961
• Publication Number:
  US 20080195654A1
• Time in Patent Office: 1,341 Days
• Field of Search: 707/804, 707/807, 707/738
• US Class Current: 707/804
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/635 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/639 using playlists
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: Y10S 707/914 Video
• CPC Class Codes: Y10S 707/915 Image
• CPC Class Codes: Y10S 707/916 Audio
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

A system and methods are provided for automatically classifying data according to perceptual properties of the data to form a classification chain that is suited to the searching and sorting of large databases of media entities. During classification, experts assign each media entity in the training data set to one or more classes, with each class corresponding to a given subset of perceptual properties of the data. In conjunction with digital signal processing properties of the data corresponding to the perceptual properties, the classified data is then used to construct an initial classification chain. During operation, when presented with an unclassified entry, the classification chain returns an estimate of the class of the entry, as well as a confidence measure that is proportional to the level of confidence of the class assignment. Over time, as the classification chain evolves, the classification chain becomes more and more effective for quickly characterizing media entities.

Claims

1. A method of forming a classification chain for a plurality of media entities, the classification chain being adapted to search databases of media entities, the method comprising:

assigning to each media entity of the plurality of media entities in a data set at least one first class of a plurality of classes, each class of the plurality of classes corresponding to a subset of predetermined perceptual properties, the assigning of the at least one first class being by a first human expert;

processing each media entity of the plurality of media entities in the data set to determine at least one digital signal processing characteristic for each media entity of the plurality of media entities in the data set; and

forming the classification chain for the plurality of media entities in the data set, the classification chain comprising components including the at least one first class and the at least one digital signal processing characteristic, the classification chain components forming rules for classifying media entities including the plurality of media entities in the data set and media entities other than the plurality of media entities in the data set.

2. The method of claim 1, further comprising:

analyzing at least one media entity other than one of the plurality of media entities in the data set; and

generating a classification according to the classification chain of the at least one media entity other than one of the plurality of media entities in the data set.

3. The method of claim 2, further comprising:

assigning to each media entity of the plurality of media entities in the data set at least one second class of the plurality of classes, the second class being different from the first class, the assigning of the at least one second class being by a second human expert,

the classification chain components further including the at least one second class.

4. The method of claim 3, further comprising:

generating a level of confidence for the classification of the at least one media entity other than one of the plurality of media entities in the data set, the level of confidence being defined by a value of a predetermined range; and

adjusting the classification of the at least one media entity other than one of the plurality of media entities in the data set upon the value of the level of confidence being below a predetermined threshold, the adjusting being by the second human expert.

5. The method of claim 4, wherein the second human expert has less expertise with analysis of the plurality of media entities of the data set than the first human expert.

6. The method of claim 2, further comprising:

generating a level of confidence for the classification of the at least one media entity other than one of the plurality of media entities in the data set, the level of confidence being defined by a value of a predetermined range.

7. The method of claim 6, further comprising:

adjusting the classification of the at least one media entity other than one of the plurality of media entities in the data set upon the value of the level of confidence being below a predetermined threshold, the adjusting being by the first human expert.

8. The method of claim 1, further comprising:

assigning to each media entity of the plurality of media entities in the data set at least one second class of the plurality of classes, the second class being different from the first class, the assigning of the at least one second class being by a second human expert,

the classification chain components further including the at least one second class.

9. The method of claim 8, wherein the second human expert has less expertise with analysis of the plurality of media entities in the data set than the first human expert.

10. The method of claim 1, wherein the subset of predetermined perceptual properties includes at least one of a spectral property, consonance, melodic movement and tempo.

11. A computer-readable storage medium having stored thereon instructions for forming a classification chain for a plurality of media entities, the classification chain being adapted for searching databases of media entities, the instructions when executed perform the steps of:

assigning to each media entity of a plurality of media entities in a data set at least one first class of a plurality of classes, each class of the plurality of classes corresponding to a subset of predetermined perceptual properties, the assigning of the at least one first class being by a first human expert;

processing each media entity of the plurality of media entities in the data set to determine at least one digital signal processing characteristic for each media entity of the plurality of media entities in the data set; and

forming a classification chain for the plurality of media entities in the data set, the classification chain comprising components including the at least one first class and the at least one digital signal processing characteristic, the classification chain components forming rules for classifying media entities including the plurality of media entities in the data set and media entities other than the plurality of media entities in the data set.

12. The computer-readable storage medium of claim 11, the instructions stored thereon when executed further performing the steps of:

analyzing at least one media entity other than one of the plurality of media entities in the data set; and

generating a classification according to the classification chain of the at least one media entity other than one of the plurality of media entities in the data set.

13. The computer-readable storage medium of claim 12, the instructions stored thereon when executed further performing the steps of:

generating a level of confidence for the classification of the at least one media entity other than one of the plurality of media entities in the data set, the level of confidence being defined by a value of a predetermined range.

14. The computer-readable storage medium of claim 13, the instructions stored thereon when executed further performing the steps of:

adjusting the classification of the at least one media entity other than one of the plurality of media entities in the data set upon the value of the level of confidence being below a predetermined threshold, the adjusting being by the first human expert.

15. The computer-readable storage medium of claim 12, the instructions stored thereon when executed further performing the steps of:

assigning to each media entity of the plurality of media entities in the data set at least one second class of the plurality of classes, the second class being different from the first class, the assigning of the at least one second class being by a second human expert,

the classification chain components further including the at least one second class.

16. The computer-readable storage medium of claim 15, further comprising:

generating a level of confidence for the classification of the at least one media entity other than one of the plurality of media entities in the data set, the level of confidence being defined by a value of a predetermined range; and

adjusting the classification of the at least one media entity other than one of the plurality of media entities in the data set upon the value of the level of confidence being below a predetermined threshold, the adjusting being by the second human expert.

17. The computer-readable storage medium of claim 11, the instructions stored thereon when executed further performing the steps of:

assigning to each media entity of the plurality of media entities in the data set at least one second class of the plurality of classes, the second class being different from the first class, the assigning of the at least one second class being by a second human expert,

the classification chain components further including the at least one second class.

18. The computer-readable storage medium of claim 11, wherein the subset of predetermined perceptual properties includes at least one of a spectral property, consonance, melodic movement and tempo.