Method and apparatus for identifying an unknown work

• US 6,968,337 B2
• Filed: 07/09/2002
• Issued: 11/22/2005
• Est. Priority Date: 07/10/2001
• Status: Active Grant

• Current Assignee: Audible Magic Corporation
• Original Assignee: Audible Magic Corporation
• Inventors: Wold, Erling H.
• Primary Examiner(s): Coby, Frantz
• Application Number:
  US10/192,783
• Publication Number:
  US 20030023852A1
• Time in Patent Office: 1,232 Days
• Field of Search: 707/100, 707/101, 707/102, 707/103.R, 707/104.1, 705/26, 386/83, 725/22, 380/54, 380/28
• US Class Current: 1/1
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G10L 25/00 Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 muting semiconductor-based amplifiers when some special characteristics of a signal are sensed by a speech detector, e.g. sensing when no signal is present, H03G3/34
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A system for determining an identity of a received work. The system receives audio data for an unknown work. The audio data is divided into segments. The system generates a signature of the unknown work from each of the segments. Reduced dimension signatures are then generated at least a portion of the signatures. The reduced dimension signatures are then compared to reduced dimensions signatures of known works that are stored in a database. A list of candidates of known works is generated from the comparison. The signatures of the unknown works are then compared to the signatures of the known works in the list of candidates. The unknown work is then identified as the known work having signatures matching within a threshold.

Claims

1. A method for determining an identity of a received work comprising:

receiving audio data for an unknown work;

dividing said audio data into a plurality of segments;

generating a plurality of signatures of said unknown work wherein each signature is generated from one of said plurality of segments;

generating a plurality of reduced dimension signatures of said unknown works wherein each of said plurality of reduced dimension signatures is generated from one of at least a portion of said plurality of signatures;

comparing said plurality of reduced dimension signatures to at least one reduced dimension signature for each one of a portion of a plurality of known works having a record stored in a works database wherein each record includes an identification of said work and at least one signature of said work;

determining a list of candidates from said plurality of known works responsive to said comparisons;

comparing said plurality of signatures to at least a portion of said plurality of signatures of each of said plurality of known works in said list of candidates;

determining one of said plurality of known works in said list of candidates that matches said unknown work from said comparison; and

identifying said unknown work as said one of said plurality of known works that matches said unknown work.

2. The method of claim 1, wherein said step of generating said plurality of signatures comprises:

calculating a plurality of mel frequency cepstral coefficients (MFCCs) for each of a plurality segments to generate each of said plurality of signatures.

3. The method of claim 1, wherein said step of generating said plurality of signatures comprises:

calculating each of said plurality of signatures using a plurality of acoustical features from one of said plurality of segments selected from a group consisting of loudness, pitch, brightness, bandwidth, spectrum and mel frequency cepstral coefficients (MFCCs).

4. The method of claim 1 further comprising:

indexing said records of said plurality of known references in said database.

5. The method of claim 4 wherein said indexing is performed by using an indexing strategy chosen from the group consisting of:

the k-d tree, the SS-tree, the R-tree, and the SR-tree.

6. The method of claim 1, wherein each of said plurality of segments has a segment size of approximately 0.5 to 3 seconds.

7. The method of claim 1, wherein each of said plurality of segments has a segment size of approximately 1 second.

8. The method of claim 1, wherein a hop size between consecutive ones of said plurality of segments is less than 50% of a segment size of each of said plurality of segments.

9. The method of claim 1, wherein a hop size between consecutive ones of said plurality of segments is approximately 0.1 seconds.

10. The method of claim 1, wherein said step of generating said plurality of signatures comprises:

calculating an average for each of a plurality of acoustical features selected from a group consisting of;

loudness, pitch, brightness, bandwidth, spectral features, and mod frequency cepstral coefficients.

11. The method of claim 1, wherein said step of generating said plurality of reduced dimension signatures comprises:

projecting features of each said portion of said plurality of signatures in a Karhunen-Loeve basis.

12. The method of claim 1 wherein said step of generating said plurality of reduced dimension signatures comprises:

selecting predetermined one of a plurality of mid frequency cepstral coefficients from each one of said portion of said plurality of signatures.

13. The method of claim 1, wherein said list of candidates includes a predetermined number of said plurality of known works.

14. An apparatus for determining an identity of a received work comprising:

means for receiving audio data for an unknown work;

means for dividing said audio data into a plurality of segments;

means for generating a plurality of signatures of said unknown work wherein each signature is generated from one of said plurality of segments;

means for generating a plurality of reduced dimension signatures of said unknown works wherein each of said plurality of reduced dimension signatures is generated from one of at least a portion of said plurality of signatures;

means for comparing said plurality of reduced dimension signatures to at least one reduced dimension signature for each one of a portion of a plurality of known works having a record stored in a works database wherein each record includes a identification of said work and at least one signature of said work;

means for determining a list of candidates from said plurality of known works responsive to said comparisons;

means for comparing said plurality of signatures to at least a portion of said plurality of signatures of each of said plurality of known works in said list of candidates; and

means for determining one of said plurality of known works in said list of candidates that matches said unknown work from said comparison.

15. The apparatus of claim 14, wherein said means for generating said plurality of signatures comprises:

means for calculating a plurality of mel frequency cepstral coefficients (MFCCs) for each of a plurality segments to generate each of said plurality of signatures.

16. The apparatus of claim 14, wherein said means for generating said plurality of signatures comprises:

means for calculating each of said plurality of signatures using a plurality of acoustical features from one of said plurality of segments selected from a group consisting of loudness, pitch, brightness, bandwidth, spectrum and mel frequency cepstral coefficients (MFCCs).

17. The apparatus of claim 14 further comprising:

means for indexing said records of said plurality of known references in said database.

18. The apparatus of claim 17 wherein said indexing is performed by using an indexing strategy chosen from the group consisting of:

the k-d tree, the SS-tree, the R-tree, and the SR-tree.

19. The apparatus of claim 14, wherein each of said plurality of segments has a segment size of approximately 0.5 to 3 seconds.

20. The apparatus of claim 14, wherein each of said plurality of segments has a segment size of approximately 1 second.

21. The apparatus of claim 14, wherein a hop size between consecutive ones of said plurality of segments is less than 50% of a segment size of each of said plurality of segments.

22. The apparatus of claim 14, wherein a hop size between consecutive ones of said plurality of segments is approximately 0.1 seconds.

23. The apparatus of claim 14, wherein said means for generating said plurality of signatures comprises:

means for calculating an average for each of a plurality of acoustical features selected from a group consisting of;

loudness, pitch, brightness, bandwidth, spectral features, and mod frequency cepstral coefficients.

24. The apparatus of claim 14, wherein said means for generating said plurality of reduced dimension signatures comprises:

means for projecting features of each said portion of said plurality of signatures in a Karhunen-Loeve basis.

25. The apparatus of claim 14 wherein said means for generating said plurality of reduced dimension signatures comprises:

means for selecting predetermined ones of a plurality of mid frequency cepstral coefficients from each one of said portion of said plurality of signatures.

26. The apparatus of claim 14, wherein said list of candidates includes a predetermined number of said plurality of known works.