Audio fingerprinting system and method

• US 7,013,301 B2
• Filed: 09/23/2003
• Issued: 03/14/2006
• Est. Priority Date: 09/23/2003
• Status: Active Grant

• Current Assignee: Gracenote, Inc. (RR Donnelley & Sons Company)
• Original Assignee: Predixis Corporation (AmpliFIND Music Services, Inc.)
• Inventors: Hicken, Wendell T., Holm, Frode
• Primary Examiner(s): Corrielus, Jean M.
• Assistant Examiner(s): To, Baoquoc N.
• Application Number:
  US10/668,926
• Publication Number:
  US 20050065976A1
• Time in Patent Office: 903 Days
• Field of Search: 707/100, 707/103.R, 707/3, 707/9, 707/102, 707/103, 713/170, 713/178
• US Class Current: 707/741
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: Y10S 707/916 Audio
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99939 Privileged access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

An audio fingerprinting system and method. A server receives an audio fingerprint of a first audio piece, searches a database for the audio fingerprint, retrieves an audio profile vector associated with the audio fingerprint, updates user preference information based on the audio profile vector, and selects a second audio piece based on the user preference information. The audio fingerprint is generated by creating a matrix based on the frequency measurements of the audio piece, and performing a singular value decomposition of the matrix. To expedite the search of the database and to increase matching accuracy, a subset of candidates in the database is identified based on the most prominent musical notes of the audio piece, and the search is limited to the identified subset. One of the attributes of the audio profile vector is a particular audio class. An identifier for the audio class is generated based on an average of audio fingerprints of the audio pieces belonging to the audio class.

Claims

1. A computer-implemented method for audio fingerprinting comprising:

receiving an audio signal associated with an audio piece;

obtaining a plurality of frequency measurements of the audio signal;

building an N×

M matrix A based on the frequency measurements;

performing a singular value decomposition on the matrix A for generating an audio fingerprint, wherein A=USV^T, and wherein U is an N×

M orthogonal matrix, S in an M×

M diagonal matrix, and V^T is a transpose of an M×

M orthogonal matrix;

retrieving one or more rows of matrix V^T; and

storing the retrieved rows of matrix V^T in a data store in association with the audio piece as the audio fingerprint for the audio piece, wherein a request including the retrieved rows of matrix V^T is received by an application program and information stored in a database for the audio piece identified and retrieved based on the rows of matrix V^T in the request.

2. The method of claim 1, wherein rows of the matrix A represent time, and columns of the matrix A represent the frequency measurements.

3. The method of claim 1, wherein the information is an audio profile vector storing acoustic analysis data for the audio piece.

9. The method of claim 3 further comprising recommending a second audio piece based on the acoustic analysis data.

10. The method of claim 3, wherein the audio profile vector quantifies a degree of similarity of the audio piece to audio pieces classified into a particular audio class.

11. The method of claim 10 further comprising generating an identifier for the particular audio class, the generating including:

selecting audio pieces associated with the particular audio class;

computing a second audio fingerprint for each selected audio piece;

calculating an average of the computed second audio fingerprints;

associating the calculated average to the particular audio class; and

storing the calculated average in a data store as the identifier of the particular audio class.

12. The method of claim 11, wherein computing of the second audio fingerprint comprises:

obtaining from a particular audio signal associated with the selected audio piece a plurality of frequency measurements;

building an N×

M matrix A′

based on the frequency measurements;

performing a singular value decomposition on the matrix A′

, wherein A′

=U′

S′

V^T′

, and wherein U′

is an N×

M orthogonal matrix, S′

in an M×

M diagonal matrix, and V^T′

is a transpose of an M×

M orthogonal matrix;

retrieving one or more rows of matrix V^T′

; and

associating the retrieved rows of matrix V^T′

with the selected audio piece.

13. The method of claim 12, wherein rows of the matrix A′

represent time, and columns of the matrix A′

represent the frequency measurements.

14. The method of claim 1, further comprising generating an index of the audio piece, the generating including:

automatically obtaining from the audio signal associated with the audio piece a list of musical notes included in the audio piece;

determining from the audio signal a prominence of the musical notes in the audio piece; and

selecting a pre-determined number of most prominent musical notes in the audio piece as the index.

15. The method of claim 14, wherein the selected musical notes are translated to musical note numbers, and the index includes the translated musical note numbers.

16. The method of claim 14, wherein data stored in the database is organized into one or more groups, wherein each group is identified by a particular index.

17. The method of claim 16, wherein a search of the database for the information for the audio piece limited to a group identified by the generated index.

4. A computer-implemented system for audio fingerprinting comprising:

an audio file reader reading an audio file storing an audio piece;

a processor coupled to the audio file reader, the processor being configured to;

obtain a plurality of frequency measurements of audio signals associated with the audio piece;

build an N×

M matrix A based on the frequency measurements;

perform a singular value decomposition on the matrix A for generating an audio fingerprint, wherein A=USV^T, and wherein U is an N×

M orthogonal matrix, S in an M×

M diagonal matrix, and V^T is a transpose of an M×

M orthogonal matrix; and

retrieve one or more rows of matrix V^T;

a data store coupled to the processor storing the retrieved rows of matrix V^T in association with the audio piece as the audio fingerprint for the audio piece; and

a database storing information for a plurality of audio pieces, wherein a request including the retrieved rows of matrix V^T is received by an application program and information for the audio piece stored in the database identified and retrieved based on the rows of matrix V^T in the request.

5. The system of claim 4, wherein rows of the matrix A represent time, and columns of the matrix A represent the frequency measurements.

6. The system of claim 4, wherein the information is an audio profile vector storing acoustic analysis data for the audio piece.

18. The system of claim 4, wherein the processor is further configured to recommend a second audio piece based on the acoustic analysis data.

19. The system of claim 6, wherein the audio profile vector quantifies a degree of similarity of the audio piece to audio pieces classified into a particular audio class.

20. The system of claim 19, wherein the processor is further configured to generate an identifier for the particular audio class, the generating including:

selecting audio pieces associated with the particular audio class;

computing a second audio fingerprint for each selected audio piece;

calculating an average of the computed second audio fingerprints;

associating the calculated average to the particular audio class; and

storing the calculated average in a data store as the identifier of the particular audio class.

21. The system of claim 20, wherein computing of the second audio fingerprint comprises:

obtaining from a particular audio signal associated with the selected audio piece a plurality of frequency measurements;

building an N×

M matrix A′

based on the frequency measurements;

performing a singular value decomposition on the matrix A′

, wherein A′

=U′

S′

V^T′

, and wherein U′

is an N×

M orthogonal matrix, S′

in an M×

M diagonal matrix, and V^T′

is a transpose of an M×

M orthogonal matrix;

retrieving one or more rows of matrix V^T′

; and

associating the retrieved rows of matrix V^T′

with the selected audio piece.

22. The system of claim 21, wherein rows of the matrix A′

represent time, and columns of the matrix A′

represent the frequency measurements.

23. The system of claim 4, wherein the processor is further configured to generate an index of the audio piece, the generating including:

automatically obtaining from the audio signal associated with the audio piece a list of musical notes included in the audio piece;

determining from the audio signal a prominence of the musical notes in the audio piece; and

selecting a pre-determined number of most prominent musical notes in the audio piece as the index.

24. The system of claim 23, wherein the selected musical notes are translated to musical note numbers, and the index includes the translated musical note numbers.

25. The system of claim 23, wherein data stored in the database is organized into one or more groups, wherein each group is identified by a particular index.

26. The system of claim 25, wherein a search of the database for the information for the audio piece limited to a group identified by the generated index.

7. An article of manufacture comprising a computer readable medium having computer usable program code containing executable instructions that, when executed, cause a computer to perform the steps of:

obtaining a plurality of frequency measurements of an audio signal associated with an audio piece;

building an N×

M matrix A based on the frequency measurements;

performing a singular value decomposition on the matrix A for generating an audio fingerprint, wherein A=USV^T, and wherein U is an N×

M orthogonal matrix, S in an M×

M diagonal matrix, and V^T is a transpose of an M×

M orthogonal matrix;

retrieving one or more rows of matrix V^T; and

storing the retrieved rows of matrix V^T in a data store in association with the audio piece as the audio fingerprint for the audio piece, wherein a request including the retrieved rows of matrix V^T is received by an application program and in formation stored in a database for the audio piece identified and retrieved based on the rows of matrix V^T in the request.

8. The article of manufacture of claim 7, wherein rows of the matrix A represent time, and columns of the matrix A represent the frequency measurements.