Method, Apparatus, and Program for Assessing Similarity of Performance Sound

• US 20090025538A1
• Filed: 07/22/2008
• Published: 01/29/2009
• Est. Priority Date: 07/26/2007
• Status: Active Grant

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: ARIMOTO, Keita
• Granted Patent: US 7,659,472 B2
• Time in Patent Office: Days
• US Class Current: 84/605
• CPC Class Codes: G09B 15/00 Teaching music metronomes G04F5/02
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 2210/056 for extraction or identification of individual instrumental parts, e.g. melody, chords, bass; Identification or separation of instrumental parts by their characteristic voices or timbres
• CPC Class Codes: G10H 2210/066 for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
• CPC Class Codes: G10H 2240/131 Library retrieval, i.e. searching a database or selecting a specific musical piece, segment, pattern, rule or parameter set
• CPC Class Codes: G10H 2240/325 Synchronizing two or more audio tracks or files according to musical features or musical timings synchronised lyrics, e.g. for karaoke G10H2220/011

Abstract

A similarity assessment apparatus is provided for assessing a performance sound based on a model performance sound. In the apparatus, a probability density function generating unit divides data of a performance sound into a sequence of frames each having a predetermined temporal length, and generates a probability density function of a fundamental frequency for each frame of the performance sound. A probability density function providing portion provides a probability density function of a fundamental frequency for each frame of the model performance sound. A similarity assessment unit compares the generated probability density function of a frame of the performance sound with the provided probability density function of a frame of the model performance sound so as to assess a similarity between the performance sound and the model performance sound.

Claims

1. A similarity assessment method of a performance sound, comprising:

a probability density function generating process of dividing one performance sound into a sequence of frames each having a predetermined temporal length and also dividing another performance sound to be compared with said one performance sound into another sequence of frames each having the predetermined temporal length, and generating a probability density function of a fundamental frequency for each frame of the respective performance sounds; and

a similarity assessment process of comparing the probability density function of a frame of said one performance sound with the probability density function of a frame of said another performance sound so as to assess a similarity between said one performance sound and said another performance sound, wherein

the probability density function generating process uses a plurality of tone models which simulate various harmonic structures of sounds generated from a musical instrument and defines a weighted mixture of the tone models corresponding to various fundamental frequencies, and recurrently updates and optimizes respective weight values of the tone models so that a frequency distribution of the weighted mixture of the tone models represents frequency components of the performance sound, thereby outputting the optimized weight values as the probability density function of the fundamental frequency of the performance sound.

2. A similarity assessment apparatus for assessing a performance sound based on a model performance sound, comprising:

a probability density function generating unit that divides data of a performance sound into a sequence of frames each having a predetermined temporal length, and generates a probability density function of a fundamental frequency for each frame of the performance sound,

wherein the probability density function generating unit uses a plurality of tone models which simulate various harmonic structures of sounds generated from a musical instrument and defines a weighted mixture of the tone models corresponding to various fundamental frequencies, and recurrently updates and optimizes respective weight values of the tone models so that a frequency distribution of the weighted mixture of the tone models represents frequency components of the performance sound, thereby outputting the optimized weight values as the probability density function of the fundamental frequency of the performance sound;

a probability density function providing portion that provides a probability density function of a fundamental frequency for each frame of the model performance sound; and

a similarity assessment unit that compares the generated probability density function of a frame of the performance sound with the provided probability density function of a frame of the model performance sound so as to assess a similarity between the performance sound and the model performance sound.

3. The similarity assessment apparatus according to claim 2, further comprising a collecting unit that collects the performance sound and outputs data of the collected performance sound in realtime basis to the probability density generating unit so that the probability density generating unit generates the probability density function of the collected performance sound in realtime basis.

4. The similarity assessment apparatus according to claim 2, wherein the probability density function providing portion comprises:

a memory unit that stores music data representing the model performance sound;

a reading unit that reads out the music data from the memory unit; and

another probability density function generating unit that divides the read music data representing the model performance sound into a sequence of frames each having the predetermined temporal length and generates the probability density function of the fundamental frequency for each frame of the model performance sound, wherein

said another probability density function generating unit uses a plurality of tone models which simulate various harmonic structures of sounds generated from a musical instrument and defines a weighted mixture of the tone models corresponding to various fundamental frequencies, and recurrently updates and optimizes respective weight values of the tone models so that a frequency distribution of the weighted mixture of the tone models represents frequency components of the model performance sound, thereby outputting the optimized weight values as the probability density function of the fundamental frequency of the model performance sound.

5. The similarity assessment apparatus according to claim 2, wherein the probability density function providing portion comprises:

a memory unit that stores the probability density function of the fundamental frequency for each frame of the model performance sound, wherein the stored probability density function is provisionally prepared by using a plurality of tone models which simulate various harmonic structures of sounds generated from a musical instrument and defining a weighted mixture of the tone models corresponding to various fundamental frequencies, and by recurrently updating and optimizing respective weight values of the tone models so that a frequency distribution of the weighted mixture of the tone models represents frequency components of the model performance sound, and thereby outputting the optimized weight values as the probability density function of the fundamental frequency of the model performance sound; and

a reading unit that reads out the probability density function of the model performance sound from the memory unit and provides the read probability density function of the model performance sound to the similarity assessment unit.

6. The similarity assessment apparatus according to claim 2, wherein the similarity assessment unit calculates a degree of the similarity of the probability density functions between a frame of the model performance sound and each of a predetermined number of frames of the performance sound containing a frame corresponding to the frame of the model performance sound, and outputs a maximum one of the calculated values as the degree of the similarity.

7. The similarity assessment apparatus according to claim 6, wherein the similarity assessment unit calculates a time difference between a frame of the performance sound which brings about the maximum one of the calculated values and the frame of the model performance sound, and outputs the time difference as a degree of temporal matching between the performance sound and the model performance sound.

8. A machine readable medium for use in a computer, the medium containing program instructions executable by the computer to perform processes of assessing a performance sound based on a model performance sound, wherein the processes comprise:

a probability density function generating process of dividing data of a performance sound into a sequence of frames each having a predetermined temporal length, and generating a probability density function of a fundamental frequency for each frame of the performance sound,

wherein the probability density function generating process uses a plurality of tone models which simulate various harmonic structures of sounds generated from a musical instrument and defines a weighted mixture of the tone models corresponding to various fundamental frequencies, and recurrently updates and optimizes respective weight values of the tone models so that a frequency distribution of the weighted mixture of the tone models represents frequency components of the performance sound, thereby outputting the optimized weight values as the probability density function of the fundamental frequency of the performance sound;

a probability density function acquiring process of acquiring a probability density function of a fundamental frequency for each frame of the model performance sound; and

a similarity assessment process of comparing the generated probability density function of a frame of the performance sound with the acquired probability density function of a frame of the model performance sound so as to assess a similarity between the performance sound and the model performance sound.