Audio fingerprinting system and method

• US 20060190450A1
• Filed: 01/31/2006
• Published: 08/24/2006
• Est. Priority Date: 09/23/2003
• Status: Active Grant

• Current Assignee: Gracenote, Inc. (RR Donnelley & Sons Company)
• Original Assignee: Predixis Corporation (AmpliFIND Music Services, Inc.)
• Inventors: Hicken, Wendell T., Holm, Frode
• Granted Patent: US 7,487,180 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: Y10S 707/916 Audio
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99939 Privileged access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

An audio fingerprinting system and method. A server receives an audio fingerprint of a first audio piece, searches a database for the audio fingerprint, retrieves an audio profile vector associated with the audio fingerprint, updates user preference information based on the audio profile vector, and selects a second audio piece based on the user preference information. The audio fingerprint is generated by creating a matrix based on the frequency measurements of the audio piece, and performing a singular value decomposition of the matrix. To expedite the search of the database and to increase matching accuracy, a subset of candidates in the database is identified based on the most prominent musical notes of the audio piece, and the search is limited to the identified subset. One of the attributes of the audio profile vector is a particular audio class. An identifier for the audio class is generated based on an average of audio fingerprints of the audio pieces belonging to the audio class.

Claims

1. An audio recognition method comprising:

receiving an audio fingerprint of an audio piece;

comparing the received audio fingerprint against a plurality of stored audio fingerprints for a match;

identifying the audio piece responsive to a match of the audio fingerprint; and

retrieving information stored in association with the identified audio piece.

2. The method of claim 1, wherein the audio fingerprint is a representation of matrix V^T generated from a singular value decomposition (SVD) of an N×

M matrix A, the matrix A being built based on frequency measurements of audio signals associated with the audio piece, wherein SVD(A)=USV^T, where U is an N×

M orthogonal matrix, S in an M×

M diagonal matrix, and V^T is a transpose of an M×

M orthogonal matrix.

3. The method of claim 2, wherein rows of the matrix A represent time, and columns of the matrix A represent the frequency measurements.

4. The method of claim 1, wherein the retrieved information is acoustic analysis data, the acoustic analysis data being generated based on an automatic processing of audio signals of the audio piece, the acoustic analysis data providing numerical measurements for a plurality of predetermined acoustic attributes.

5. The method of claim 4, wherein one of the plurality of predetermined acoustic attributes is a particular audio class, the acoustic analysis data providing a measurement for determining a degree of similarity of the audio piece to audio pieces associated with the particular audio class.

6. The method of claim 5, wherein the audio class is identified based on an average of audio fingerprints of audio pieces associated with the audio class.

7. The method of claim 1 further comprising:

identifying an index of a subset of the plurality of stored audio fingerprints, the index identifying a plurality of musical notes determined to be most prominent for the audio fingerprints in the subset; and

searching the identified subset for the match.

8. An audio fingerprinting method comprising:

identifying an audio piece containing audio signals;

obtaining a plurality of frequency measurements associated with the audio signals;

building an N×

M matrix A based on the frequency measurements;

performing a singular value decomposition (SVD) on the matrix A for generating an audio fingerprint, wherein SDV(A)=USV^T, and wherein U is an N×

M orthogonal matrix, S is an M×

M diagonal matrix, and V^T is a transpose of an M×

M orthogonal matrix;

retrieving one or more rows of matrix V^T;

storing a representation of the retrieved rows of matrix V^T in a data store in association with the audio piece as the audio fingerprint for the audio piece; and

transmitting the audio fingerprint to a remote computer.

9. An audio recognition system comprising:

a first data store storing a plurality of audio fingerprints for a plurality of audio pieces;

means for receiving an audio fingerprint of a particular audio piece;

means for comparing the received audio fingerprint against the plurality of stored audio fingerprints for a match;

means for identifying the audio piece responsive to a match of the audio fingerprint; and

means for retrieving information stored in association with the identified audio piece.

10. The system of claim 9, wherein the audio fingerprint is a representation of matrix V^T generated from a singular value decomposition (SVD) of an N×

M matrix A, the matrix A being built based on frequency measurements of audio signals associated with the audio piece, wherein SVD(A)=USV^T, where U is an N×

M orthogonal matrix, S in an M×

M diagonal matrix, and V^T is a transpose of an M×

M orthogonal matrix.

11. The system of claim 10, wherein rows of the matrix A represent time, and columns of the matrix A represent the frequency measurements.

12. The system of claim 9 further comprising:

a second data store storing acoustic analysis data in association with the audio piece, the acoustic analysis data being generated based on an automatic processing of audio signals of the audio piece, the acoustic analysis data providing numerical measurements for a plurality of predetermined acoustic attributes, wherein the retrieved information is the acoustic analysis data.

13. The system of claim 12, wherein one of the plurality of predetermined acoustic attributes is a particular audio class, the acoustic analysis data providing a measurement for determining a degree of similarity of the audio piece to audio pieces associated with the particular audio class.

14. The system of claim 13, wherein the audio class is identified based on an average of audio fingerprints of audio pieces associated with the audio class.

15. The system of claim 9 further comprising:

means for identifying an index of a subset of the plurality of stored audio fingerprints based on the audio fingerprint, the index identifying a plurality of musical notes determined to be most prominent for the audio fingerprints in the subset; and

means for searching the identified subset for the match.

16. An audio fingerprinting system comprising:

means for identifying an audio piece containing audio signals;

means for obtaining a plurality of frequency measurements associated with the audio signals;

means for building an N×

M matrix A based on the frequency measurements;

means for performing a singular value decomposition (SVD) on the matrix A for generating an audio fingerprint, wherein SDV(A)=USV^T, and wherein U is an N×

M orthogonal matrix, S is an M×

M diagonal matrix, and V^T is a transpose of an M×

M orthogonal matrix;

means for retrieving one or more rows of matrix V^T;

means for storing the retrieved rows of matrix V^T in a data store in association with the audio piece as the audio fingerprint for the audio piece; and

means for transmitting the audio fingerprint to a remote computer.