System and method for music identification

• US 20030106413A1
• Filed: 12/06/2001
• Published: 06/12/2003
• Est. Priority Date: 12/06/2001
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Said, Amir, Samadani, Ramin
• Granted Patent: US 6,995,309 B2
• Time in Patent Office: Days
• US Class Current: 84/603
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 2240/135 Library retrieval index, i.e. using an indexing scheme to efficiently retrieve a music piece
• CPC Class Codes: G10H 2240/141 Library retrieval matching, i.e. any of the steps of matching an inputted segment or phrase with musical database contents, e.g. query by humming, singing or playing; the steps may include, e.g. musical analysis of the input, musical feature extraction, query formulation, or details of the retrieval process
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

A system and method that allows users to find a song name, artist and performance without having to proceed through many false results. In one aspect, the system and method use signal matching to produce reliable matches. In another aspect, the system and method use a combination of signal and feature representation and an automatic decision rule together with a human-in-the-loop approach. A feature vector and a processed time signal are computed for each song in a database and extracted from a microphone-recorded sample of music. The database songs are first sorted by feature space distance with respect to the feature vector of the recorded sample. The processed time signals of the database songs and the processed time signal from the recorded sample are processed using signal matching. A decision rule presents likely matches to the user for confirmation. Using signal matching, feature-ordered search and a decision rule results in an effective framework for finding a song from a brief microphone recorded sample of music.

Claims

1. A method for identifying music, comprising the steps of:

(a) recording a sample of audio data;

(b) deriving a sample time signal from the audio data;

(c) sorting a plurality of songs, wherein each song is represented by a time signal; and

(d) matching the sample time signal with the time signal of a song in the plurality of songs.

2. The method of claim 1, where the sorting step further comprises:

generating a sample feature vector for the sample time signal;

generating a feature vector for each time signal of the songs; and

sorting the songs in an ascending order based on feature space distance between the sample feature vector and respective feature vectors for each time signal of the songs.

3. The method of claim 2, where the generating steps further comprise extracting features from the sample time signal and the time signals of the songs.

4. The method of claim 3, where the features comprise beat, noise, tone, pitch, loudness and tempo.

5. The method of claim 1, where the sorting step further comprises:

generating a sample feature vector for the sample time signal;

generating a plurality of feature vectors for one or more time signals of the songs, wherein each feature vector of the plurality of feature vectors for a time signal is generated from a different segment of the song corresponding to the time signal;

separating the plurality of feature vectors for each time signal as distinct entries; and

sorting the entries in an ascending order based on feature space distance between the sample feature vector and respective feature vectors for the entries.

6. The method of claim 2, where the matching step further comprises:

comparing the sample time signal to a first time signal in the ascending order;

computing a signal match waveform for the first time signal in relation to the sample time signal;

playing the song corresponding to the first time signal if the signal match waveform satisfies a decision rule;

indicating by a user whether the played song matches the sample of audio data; and

presenting to the user song information corresponding to the first time signal if the user indicates a match.

7. The method of claim 6, where the signal match waveform is computed by calculating a cross-correlation of the first time signal in relation to the sample time signal.

8. The method of claim 7, where the decision rule is satisfied if an overall absolute maximum of the cross-correlation computed for the first time signal is greater than an average cross-correlation by a predetermined factor.

9. The method of claim 7, where the decision rule is satisfied if an overall absolute maximum of the cross-correlation computed for the first time signal is greater than an average cross-correlation by a predetermined factor, and no incorrect songs have been presented to the user.

10. The method of claim 6, where the song information comprises song title, artist and performance.

11. The method of claim 6, further comprising, after the computing step, the step of:

comparing the sample time signal to a next time signal in the ascending order, and subsequently repeating the computing step with respect to the next time signal, if the signal match waveform does not satisfy the decision rule for the first time signal.

12. The method of claim 6, further comprising the step of:

comparing the sample time signal to a next time signal in the ascending order, and subsequently repeating the computing, playing, indicating and presenting steps with respect to the next time signal, if the user indicates that the played song does not match the sample of audio data.

13. A system for identifying music, comprising:

a means for recording a sample of audio data;

a means for deriving a sample time signal from the audio data;

a means for sorting a plurality of songs, wherein each song is represented by a time signal; and

a means for matching the sample time signal with the time signal of a song in the plurality of songs.

14. The system of claim 13, where the means for sorting further comprises:

a means for generating a sample feature vector for the sample time signal;

a means for generating a feature vector for each time signal of the songs; and

a means for sorting the songs in an ascending order based on feature space distance between the sample feature vector and respective feature vectors for each time signal of the songs.

15. The system of claim 14, where the means for generating a sample feature vector and the means for generating a feature vector for each time signal of the songs further comprise a means for extracting features from the sample time signal and the time signals of the songs.

16. The system of claim 15, where the features comprise beat, noise, tone, pitch, loudness and tempo.

17. The system of claim 13, where the means for sorting further comprises:

a means for generating a sample feature vector for the sample time signal;

a means for generating a plurality of feature vectors for one or more time signals of the songs, wherein each feature vector of the plurality of feature vectors for a time signal is generated from a different segment of the song corresponding to the time signal;

a means for separating the plurality of feature vectors for each time signal as distinct entries; and

a means for sorting the entries in an ascending order based on feature space distance between the sample feature vector and respective feature vectors for the entries.

18. The system of claim 14, where the means for matching further comprises:

a means for comparing the sample time signal to a first time signal in the ascending order;

a means for computing a signal match waveform for the first time signal in relation to the sample time signal;

a means for playing the song corresponding to the first time signal if the signal match waveform satisfies a decision rule;

a means for indicating by a user whether the played song matches the sample of audio data; and

a means for presenting to the user song information corresponding to the first time signal if the user indicates a match.

19. The system of claim 18, where the signal match waveform is computed by calculating a cross-correlation of the first time signal in relation to the sample time signal.

20. The system of claim 19, where the decision rule is satisfied if an overall absolute maximum of the cross-correlation computed for the first time signal is greater than an average cross-correlation by a predetermined factor.

21. The system of claim 19, where the decision rule is satisfied if an overall absolute maximum of the cross-correlation computed for the first time signal is greater than an average cross-correlation by a predetermined factor, and no incorrect songs have been presented to the user.

22. The system of claim 18, where the song information comprises song title, artist and performance.

23. A method for identifying music, comprising the steps of:

(a) recording a sample of audio data;

(b) deriving a sample time signal from the audio data; and

(c) matching the sample time signal with a time signal of a plurality of time signals in a database, wherein each of the plurality of times signals represents a song in the database.

24. The method of claim 23, where the matching step further comprises:

computing a signal match intensity for the plurality of time signals in the database in relation to the sample time signal;

selecting a time signal of the plurality of time signals having a maximum signal match intensity; and

presenting to a user song information corresponding to the selected time signal.

25. The method of claim 24, where the song information comprises song title, artist and performance.

26. A system for identifying music, comprising:

a means for recording a sample of audio data;

a means for deriving a sample time signal from the audio data; and

a means for matching the sample time signal with a time signal of a plurality of time signals in a database, wherein each of the plurality of time signals represents a song in the database.

27. The system of claim 26, where the means for matching further comprises:

a means for computing a signal match intensity for the plurality of time signals in the database in relation to the sample time signal;

a means for selecting a time signal of the plurality of time signals having a maximum signal match intensity; and

a means for presenting to a user song information corresponding to the selected time signal.

28. The method of claim 27, where the song information comprises song title, artist and performance.

29. A method for identifying music, comprising the steps of:

(a) recording a sample of audio data;

(b) generating a first plurality of time signals from the sample of audio data, wherein the first plurality of time signals are generated in distinct frequency bands;

(c) generating a second plurality of time signals from songs in a database, wherein the second plurality of time signals are generated in the same distinct frequency bands as the first plurality of time signals; and

(d) matching the first plurality of time signals with the second plurality of time signals.