Sound signal analysis apparatus, sound signal analysis method and sound signal analysis program

• US 9,171,532 B2
• Filed: 03/14/2014
• Issued: 10/27/2015
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: Maezawa, Akira
• Primary Examiner(s): WARREN, DAVID S
• Application Number:
  US14/212,022
• Publication Number:
  US 20140260912A1
• Time in Patent Office: 592 Days
• Field of Search: 846/12, 846/36, 846/52, 846/68
• US Class Current: 1/1
• CPC Class Codes: G10H 1/40 Rhythm
• CPC Class Codes: G10H 2210/031 Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal neural networks for electrophonic musical instruments or musical processing G10H2250/311
• CPC Class Codes: G10H 2210/051 for extraction or detection of onsets of musical sounds or notes, i.e. note attack timings
• CPC Class Codes: G10H 2210/061 for extraction of musical phrases, isolation of musically relevant segments, e.g. musical thumbnail generation, or for temporal structure analysis of a musical piece, e.g. determination of the movement sequence of a musical work
• CPC Class Codes: G10H 2210/076 for extraction of timing, tempo; Beat detection tempo display G10H2220/081; tempo control G10H2210/375
• CPC Class Codes: G10H 2210/385 Speed change, i.e. variations from preestablished tempo, tempo change, e.g. faster or slower, accelerando or ritardando, without change in pitch with repetitive changes in pitch, e.g. scratch DJ effects G10H2210/241
• CPC Class Codes: G10H 2220/081 Beat indicator, e.g. marks or flashing LEDs to indicate tempo or beat positions analysis tempo G10H2210/076; tempo or beat alterations G10H2210/375; rhythm pattern G10H2210/341
• CPC Class Codes: G10H 2250/015 Markov chains, e.g. hidden Markov models [HMM], for musical processing, e.g. musical analysis or musical composition
• CPC Class Codes: G10H 7/00 Instruments in which the tones are synthesised from a data store, e.g. computer organs

Abstract

A sound signal analysis apparatus 10 includes sound signal input portion for inputting a sound signal indicative of a musical piece; feature value calculation portion for calculating a first feature value indicative of a feature relating to existence of a beat in one of sections of the musical piece and a second feature value indicative of a feature relating to tempo in one of the sections of the musical piece; and estimation portion for concurrently estimating a beat position and a change in tempo in the musical piece by selecting, from among a plurality of probability models described as sequences of states q classified according to a combination of a physical quantity relating to existence of a beat in one of the sections of the musical piece and a physical quantity relating to tempo in one of the sections of the musical piece, a probability model whose sequence of observation likelihoods each indicative of a probability of concurrent observation of the first feature value and the second feature value in corresponding one of the sections of the musical piece satisfies a certain criterion.

Claims

1. A sound signal analysis apparatus comprising:

a non-transitory memory device;

a processor;

a sound signal input portion for inputting a sound signal indicative of a musical piece;

a feature value calculation portion for calculating a first feature value indicative of a feature relating to existence of a beat in one of sections of the musical piece and a second feature value indicative of a feature relating to tempo in one of the sections of the musical piece; and

an estimation portion for concurrently estimating a beat position and a change in tempo in the musical piece by selecting, from among a plurality of probability models described as sequences of states classified according to a combination of a physical quantity relating to existence of a beat in one of the sections of the musical piece and a physical quantity relating to tempo in one of the sections of the musical piece, a probability model whose sequence of observation likelihoods each indicative of a probability of concurrent observation of the first feature value and the second feature value in corresponding one of the sections of the musical piece satisfies a certain criterion,

wherein the sound signal input portion, the feature value calculation portion, and the estimation portion are implemented at least in part by the processor executing at least one program recorded on the non-transitory memory device.

2. The sound signal analysis apparatus according to claim 1, wherein

the estimation portion concurrently estimates a beat position and a change in tempo in the musical piece by selecting a probability model of the most likely sequence of observation likelihoods from among the plurality of probability models.

3. The sound signal analysis apparatus according to claim 1, wherein

the estimation portion has first probability output portion for outputting, as a probability of observation of the first feature value, a probability calculated by assigning the first feature value as a probability variable of a probability distribution function defined according to the physical quantity relating to existence of beat.

4. The sound signal analysis apparatus according to claim 3, wherein

as a probability of observation of the first feature value, the first probability output portion outputs a probability calculated by assigning the first feature value as a probability variable of any one of normal distribution, gamma distribution and Poisson distribution defined according to the physical quantity relating to existence of beat.

5. The sound signal analysis apparatus according to claim 1, wherein

the estimation portion has second probability output portion for outputting, as a probability of observation of the second feature value, goodness of fit of the second feature value to a plurality of templates provided according to the physical quantity relating to tempo.

6. The sound signal analysis apparatus according to claim 1, wherein

the estimation portion has second probability output portion for outputting, as a probability of observation of the second feature value, a probability calculated by assigning the second feature value as a probability variable of probability distribution function defined according to the physical quantity relating to tempo.

7. The sound signal analysis apparatus according to claim 6, wherein

as a probability of observation of the second feature value, the second probability output portion outputs a probability calculated by assigning the first feature value as a probability variable of any one of multinomial distribution, Dirichlet distribution, multidimensional normal distribution, and multidimensional Poisson distribution defined according to the physical quantity relating to existence of beat.

8. The sound signal analysis apparatus according to claim 1, wherein

the sections of the musical piece correspond to frames, respectively, formed by dividing the input sound signal at certain time intervals;

and

the feature value calculation portion has;

a first feature value calculation portion for calculating amplitude spectrum for each of the frames, applying a plurality of window functions each having a different frequency band to the amplitude spectrum to generate amplitude spectrum for each frequency band, and outputting, as the first feature value, a value calculated on the basis of a change in amplitude spectrum provided for the each frequency band between the frames; and

a second feature value calculation portion having a filter that outputs a value in response to each input of a value corresponding to a frame, that has keeping portion for keeping the output value for a certain period of time, and that combines the input value and the value kept for the certain period of time at a certain ratio, and output the combined value, the second feature value calculation portion outputting, as a sequence of the second feature values, a data sequence obtained by inputting, to the filter, a data sequence obtained by reversing a time sequence of a data sequence obtained by inputting a sequence of the first feature values to the filter.

9. The sound signal analysis apparatus according to claim 1, further comprising:

a correction information input portion for inputting correction information indicative of corrected content of one of or both of a beat position and a change in tempo in the musical piece;

a observation likelihood correction portion for correcting the observation likelihoods in accordance with the input correction information; and

a re-estimation portion for re-estimating a beat position and a change in tempo in the musical piece concurrently by selecting, by use of the estimation portion, a probability model whose sequence of the corrected observation likelihoods satisfies the certain criterion from among the plurality of probability models,

wherein the correction information input portion, the observation likelihood correction portion, and the re-estimation portion are implemented at least in part by the processor executing at least one program recorded on the non-transitory memory device.

10. A sound signal analysis method comprising:

inputting a sound signal indicative of a musical piece;

calculating a first feature value indicative of a feature relating to existence of a beat in one of sections of the musical piece and a second feature value indicative of a feature relating to tempo in one of the sections of the musical piece; and

concurrently estimating a beat position and a change in tempo in the musical piece by selecting, from among a plurality of probability models described as sequences of states classified according to a combination of a physical quantity relating to existence of a beat in one of the sections of the musical piece and a physical quantity relating to tempo in one of the sections of the musical piece, a probability model whose sequence of observation likelihoods each indicative of a probability of concurrent observation of the first feature value and the second feature value in corresponding one of the sections of the musical piece satisfies a certain criterion.

11. A non-transitory computer-readable storage medium having stored thereon a sound signal analysis program configured to cause a computer to execute a sound signal analysis method comprising:

inputting a sound signal indicative of a musical piece;

calculating a first feature value indicative of a feature relating to existence of a beat in one of sections of the musical piece and a second feature value indicative of a feature relating to tempo in one of the sections of the musical piece; and

concurrently estimating a beat position and a change in tempo in the musical piece by selecting, from among a plurality of probability models described as sequences of states classified according to a combination of a physical quantity relating to existence of a beat in one of the sections of the musical piece and a physical quantity relating to tempo in one of the sections of the musical piece, a probability model whose sequence of observation likelihoods each indicative of a probability of concurrent observation of the first feature value and the second feature value in corresponding one of the sections of the musical piece satisfies a certain criterion.