World Wide Web-based melody retrieval system with thresholds determined by using distribution of pitch and span of notes

• US 6,121,530 A
• Filed: 03/19/1999
• Issued: 09/19/2000
• Est. Priority Date: 03/19/1998
• Status: Expired due to Fees

• Current Assignee: Sonoda, Tomonari
• Original Assignee: Sonoda, Tomonari
• Inventors: Sonoda, Tomonari
• Primary Examiner(s): Ip, Paul
• Assistant Examiner(s): Fletcher, Marlon T.
• Application Number:
  US09/272,211
• Time in Patent Office: 550 Days
• Field of Search: 84/601-607, 84/609-613, 84/622-625, 84/645, 84/649-652, 348/7, 348/12-14, 395/200.31, 395/200.33, 395/200.47, 395/200.49
• US Class Current: 84/609
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G10H 1/0041 in coded form
• CPC Class Codes: G10H 2240/141 Library retrieval matching, i.e. any of the steps of matching an inputted segment or phrase with musical database contents, e.g. query by humming, singing or playing; the steps may include, e.g. musical analysis of the input, musical feature extraction, query formulation, or details of the retrieval process
• CPC Class Codes: G10H 2240/251 Mobile telephone transmission, i.e. transmitting, accessing or controlling music data wirelessly via a wireless or mobile telephone receiver, analog or digital, e.g. DECT GSM, UMTS smartphone, PDA or palmtop used for musical purposes G10H2230/015; mobile ringtone G10H2230/021

Abstract

A World Wide Web-based melody retrieval system takes a sung melody as a query and retrieves the song'"'"'s title or other information from a music database over a WWW network which comprises a method of obtaining search clues with the maximum quantity of information from pitch and span (dynamic threshold determination) and a method of effectively reducing the number of answer candidates (coarse-to-fine matching), thus increasing the matching accuracy, and it is characterized in that a user can retrieve music or media with music by singing.

Claims

1. A World Wide Web-based melody retrieval method, comprising:

obtaining sequences of relative-pitch difference and relative-span ratio from sequences of pitch and span values of each musical composition in a database;

making a single histogram of the relative-pitch values of the database based on the distribution of all notes in the database;

making a single histogram of the relative-span values of the database based on the distribution of all notes in the database;

determining thresholds for relative-pitch so that each category, which will be defined by the thresholds, equally includes M1(=Sum1/Category_-- Num1) values, wherein M1 is the number of frequency of relative-pitch values which each category will contain, Sum1 is the total frequency of values of a histogram of relative-pitch, and Category_-- Num1 is the number of categories for relative-pitch values, and simultaneously determining thresholds for relative-span values so that each category equally includes M2(Sum2/Category_-- Num2) values, wherein M2 is the number of frequency of relative-span values which each category will contain, Sum2 is the total frequency of values of a histogram of relative-span, and Category_-- Num2 is the number of categories;

converting the relative-pitch values and the relative-span values into coarse pitch values, which are approximate relative-values of pitch, and coarse span values, which are approximate relative-values of span, respectively, according to the previously obtained thresholds;

inputting a song by singing, humming, or whistling a melody into a microphone;

subjecting the inputted melody to an A/D conversion so as to produce a digital signal;

detecting a voiced sound from a sound signal in the digital signal produced by the A/D conversion;

defining a fundamental frequency of each frame from the detected voiced sound;

dividing off an onset time of the voiced sound as an onset time of each note;

determining a time difference in units of frame number as the span value of the note;

determining the maximum value among the fundamental frequencies of each note contained during the span of said each note as the highest pitch value;

calculating the relative pitch and span values of each note by using the determined highest pitch value and the determined span value of the note preceding the note for which the relative pitch and span values are being calculated;

transmitting the calculated relative pitch and relative span values to a melody retrieval system over the World Wide Web network;

converting the relative pitch values and the relative span values of the inputted melody into coarse pitch values and coarse span values, respectively, by using the previously obtained thresholds;

comparing the coarse pitch values and coarse span values of the inputted melody with the coarse pitch values and coarse span values, respectively, of each melody in the music database;

calculating a distance between the pitch and span of each melody in the music database and the inputted melody; and

displaying information indicative of musical compositions matching the inputted melody on a display of an input device.

2. A World Wide Web-based media retrieval method as claimed in claim 1, further comprising retrieving a media with music by melody.

3. A World Wide Web-based melody retrieval system, comprising:

means for obtaining sequences of relative-pitch difference and relative-span ratio from sequences of pitch and span values of each musical composition in a database;

means for making a single histogram of the relative-pitch values of the database based on the distribution of all notes in the database;

means for making a single histogram of the relative-span values of the database based on the distribution of all notes in the database;

means for determining thresholds for relative-pitch so that each category, which will be defined by the thresholds, equally includes M1(=Sum1/Category_-- Num1) values, wherein M1 is the number of frequency of relative-pitch values which each category will contain, Sum1 is the total frequency of values of a histogram of relative-pitch, and Category_-- Num1 is the number of categories for relative-pitch values, and simultaneously determining thresholds for relative-span values so that each category equally includes M2(=Sum2/Category_-- Num2) values, wherein M2 is the number of frequency of relative-span values which each category will contain, Sum2 is the total frequency of values of a histogram of relative-span, and Category_-- Num2 is the number of categories;

means for converting the relative-pitch values and the relative-span values into coarse pitch values, which are approximate relative-values of pitch, and coarse span values, which are approximate relative-values of span, respectively, by using the previously obtained thresholds;

means for inputting a song by singing, humming, or whistling a melody into a microphone;

means for subjecting the inputted melody to an A/D conversion so as to produce a digital signal;

means for detecting a voiced sound from a sound signal in the digital signal produced by the A/D conversion;

means for defining a fundamental frequency of each frame from the detected voiced sound;

means for dividing off an onset time of the voiced sound as an onset time of each note;

means for determining a time difference in units of frame number as the span value of the note;

means for determining the maximum value among the fundamental frequencies of each note contained during the span of said each note as the highest pitch value;

means for calculating the relative pitch and span values of each note by using the determined highest pitch value and the determined span value of the note preceding the note for which the relative pitch and span values are being calculated;

means for transmitting the calculated relative pitch and relative span values to a song retrieval system over the World Wide Web network;

means for converting the relative pitch values and the relative span values of the inputted melody into coarse pitch values and coarse span values, respectively, according to the previously obtained thresholds;

means for comparing the coarse pitch values and coarse span values of the inputted melody with the coarse pitch values and coarse span values, respectively, of each melody in the music database;

means for calculating a distance between the pitch and span of each melody in the music database and the inputted melody; and

means for display information indicative of musical compositions matching the inputted melody on a display of an input device.

4. A World Wide Web-based media retrieval system as claimed in claim 3, further comprising means for retrieving a media with music by melody.