SOUND ANALYSIS APPARATUS AND PROGRAM

• US 20080053295A1
• Filed: 08/31/2007
• Published: 03/06/2008
• Est. Priority Date: 09/01/2006
• Status: Active Grant

• Current Assignee: National Institute of Advanced Industrial Science and Technology (Government of Japan), Yamaha Corporation
• Original Assignee: National Institute of Advanced Industrial Science and Technology (Government of Japan), Yamaha Corporation
• Inventors: Goto, Masataka, Arimoto, Keita, Fujishima, Takuya
• Granted Patent: US 7,754,958 B2
• Time in Patent Office: Days
• US Class Current: 84/616
• CPC Class Codes: G10H 2210/066 for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
• CPC Class Codes: G10H 3/125 Extracting or recognising the pitch or fundamental frequency of the picked up signal

Abstract

A sound analysis apparatus stores sound source structure data defining a constraint on one or more of sounds that can be simultaneously generated by a sound source of an input audio signal. A form estimation part selects fundamental frequencies of one or more of sounds likely to be contained in the input audio signal with peaked weights from various fundamental frequencies during sequential updating and optimizing of weights of tone models corresponding to the various fundamental frequencies, so that the sounds of the selected fundamental frequencies satisfy the sound source structure data, and creates form data specifying the selected fundamental frequencies. A previous distribution imparting part imparts a previous distribution to the weights of the tone models corresponding to the various fundamental frequencies so as to emphasize weights corresponding to the fundamental frequencies specified by the form data created by the form estimation part.

Claims

1. A sound analysis apparatus for analyzing an input audio signal based on a weighted mixture of a plurality of tone models which represent harmonic structures of sound sources and which correspond to probability density functions of various fundamental frequencies, the apparatus comprising:

a probability density estimation part that sequentially updates and optimizes respective weights of the plurality of the tone models, so that a mixed distribution of frequencies obtained by the weighted mixture of the plurality of the tone models corresponding respectively to the various fundamental frequencies approximates an actual distribution of frequency components of the input audio signal, and that estimates the optimized weights of the tone models to be a fundamental frequency probability density function of the various fundamental frequencies corresponding to the sound sources; and

a fundamental frequency determination part that determines an actual fundamental frequency of the input audio signal based on the fundamental frequency probability density function estimated by the probability density estimation part, wherein

the probability density estimation part comprises;

a storage part that stores sound source structure data defining a constraint on one or more of sounds that can be simultaneously generated by a sound source of the input audio signal,

a form estimation part that selects fundamental frequencies of one or more of sounds likely to be contained in the input audio signal with peaked weights from the various fundamental frequencies during the sequential updating and optimizing of the weights of the tone models corresponding to the various fundamental frequencies, so that the sounds of the selected fundamental frequencies satisfy the sound source structure data, and that creates form data specifying the selected fundamental frequencies, and

a previous distribution imparting part that imparts a previous distribution to the weights of the tone models corresponding to the various fundamental frequencies so as to emphasize weights corresponding to the fundamental frequencies specified by the form data created by the form estimation part.

2. The sound analysis apparatus according to claim 1, wherein the fundamental frequency determination part includes a part for calculating a threshold value according to a maximum one of respective peak values of probability densities which are provided by the fundamental frequency probability density function and which correspond to the fundamental frequencies specified by the form data, selecting a fundamental frequency with a probability density whose peak value is greater than the threshold value from the fundamental frequencies specified by the form data, and determining the selected fundamental frequency to be the actual fundamental frequency of the input audio signal.

3. The sound analysis apparatus according to claim 1, wherein the probability density estimation part further includes a part for selecting each fundamental frequency specified by the form data, setting a weight corresponding to the selected fundamental frequency to zero, performing a process of updating the weights of the tone models corresponding to the various fundamental frequencies once, and excluding the selected fundamental frequency from the fundamental frequencies of the sounds that are estimated to be likely to be contained in the input audio signal if the updating process makes no great change in the weights of the tone models corresponding to the various fundamental frequencies.

4. A sound analysis apparatus for analyzing an input audio signal based on a weighted mixture of a plurality of tone models which represent harmonic structures of sound sources and which correspond to probability density functions of various fundamental frequencies, the apparatus comprising:

a probability density estimation part that sequentially updates and optimizes respective weights of the plurality of the tone models, so that a mixed distribution of frequencies obtained by the weighted mixture of the plurality of the tone models corresponding respectively to the various fundamental frequencies approximates a distribution of frequency components of the input audio signal, and that estimates the optimized weights of the tone models to be a fundamental frequency probability density function of the various fundamental frequencies corresponding to the sound sources; and

a fundamental frequency determination part that determines an actual fundamental frequency of the input audio signal based on the fundamental frequency probability density function estimated by the probability density estimation part, wherein

the fundamental frequency determination part comprises;

a storage part that stores sound source structure data defining a constraint on one or more of sounds that can be simultaneously generated by a sound source of the input audio signal,

a form estimation part that selects, from the various fundamental frequencies, fundamental frequencies of one or more of sounds which have weights peaked in the fundamental frequency probability density function estimated by the probability density estimation part and which are estimated to be likely contained in the input audio signal so that the selected fundamental frequencies satisfy the constraint defined by the sound source structure data, and that creates form data representing the selected fundamental frequencies, and

a determination part that determines the actual fundamental frequency of the input audio signal based on the form data.

5. The sound analysis apparatus according to claim 4, wherein the fundamental frequency determination part includes a part for calculating a threshold value according to a maximum one of respective peak values of probability densities which are provided by the fundamental frequency probability density function and which correspond to the fundamental frequencies specified by the form data, selecting a fundamental frequency with a probability density whose peak value is greater than the threshold value from the fundamental frequencies specified by the form data, and determining the selected fundamental frequency to be the actual fundamental frequency of the input audio signal.

6. The sound analysis apparatus according to claim 4, wherein the probability density estimation part includes a part for selecting each fundamental frequency specified by the form data, setting a weight corresponding to the selected fundamental frequency to zero, performing a process of updating the weights of the tone models corresponding to the various fundamental frequencies once, and excluding the selected fundamental frequency from the fundamental frequencies of the sounds that are estimated to be likely to be contained in the input audio signal if the updating process makes no great change in the weights of the tone models corresponding to the various fundamental frequencies.

7. A sound analysis apparatus for analyzing an input audio signal based on a weighted mixture of a plurality of tone models which represent harmonic structures of sound sources and which correspond to probability density functions of various fundamental frequencies, the apparatus comprising:

a probability density estimation part that sequentially updates and optimizes respective weights of the plurality of the tone models, so that a mixed distribution of frequencies obtained by the weighted mixture of the plurality of the tone models corresponding to the various fundamental frequencies approximates a distribution of frequency components of the input audio signal, and that estimates the optimized weights of the tone models to be a fundamental frequency probability density function of the various fundamental frequencies corresponding to the sound sources; and

a fundamental frequency determination part that determines an actual fundamental frequency of the input audio signal based on the fundamental frequency probability density function estimated by the probability density estimation part,

wherein the probability density estimation part comprises;

a storage part that stores sound source structure data defining a constraint on one or more of sounds that can be simultaneously generated by a sound source of the input audio signal,

a first update part that updates the weights of the tone models corresponding to the various fundamental frequencies a specific number of times for approximating the frequency components of the input audio signal,

a fundamental frequency selection part that obtains fundamental frequencies with peaked weights based on the weights updated by the first update part from the various fundamental frequencies and that selects fundamental frequencies of one more sounds likely to be contained in the input audio signal from the obtained fundamental frequencies with the peaked weights so that the selected fundamental frequencies satisfy the constraint defined by the sound source structure data, and

a second update part that imparts a previous distribution to the weights of the tone models corresponding to the various fundamental frequencies so as to emphasize the weights corresponding to the fundamental frequencies selected by the fundamental frequency selection part, and that updates the weights of the tone models corresponding to the various fundamental frequencies a specific number of times for further approximating the frequency components of the input audio signal.

8. The sound analysis apparatus according to claim 7, wherein the probability density estimation part further includes a third update part that updates the weights, updated by the second update part, of the tone models corresponding to the various fundamental frequencies a specific number of times for further approximating the frequency components of the input audio signal, without imparting the previous distribution.

9. The sound analysis apparatus according to claim 7, wherein the fundamental frequency determination part includes a part for calculating a threshold value according to a maximum one of respective peak values of probability densities which are provided by the fundamental frequency probability density function and which correspond to the fundamental frequencies specified by the form data, selecting a fundamental frequency with a probability density whose peak value is greater than the threshold value from the fundamental frequencies specified by the form data, and determining the selected fundamental frequency to be the actual fundamental frequency of the input audio signal.

10. The sound analysis apparatus according to claim 7, wherein the probability density estimation part further includes a part for selecting each fundamental frequency specified by the form data, setting a weight corresponding to the selected fundamental frequency to zero, performing a process of updating the weights of the tone models corresponding to the various fundamental frequencies once, and excluding the selected fundamental frequency from the fundamental frequencies of the sounds that are estimated to be likely to be contained in the input audio signal if the updating process makes no great change in the weights of the tone models corresponding to the various fundamental frequencies.

11. A machine readable medium for use in a sound analysis apparatus having a processor for analyzing an input audio signal based on a weighted mixture of a plurality of tone models which represent harmonic structures of sound sources and which correspond to probability density functions of various fundamental frequencies, the machine readable medium containing program instructions executable by the processor for causing the sound synthesis apparatus to perform:

a probability density estimation process of sequentially updating and optimizing respective weights of the plurality of the tone models, so that a mixed distribution of frequencies obtained by the weighted mixture of the plurality of the tone models corresponding respectively to the various fundamental frequencies approximates an actual distribution of frequency components of the input audio signal, and estimating the optimized weights of the tone models to be a fundamental frequency probability density function of the various fundamental frequencies corresponding to the sound sources; and

a fundamental frequency determination process of determining an actual fundamental frequency of the input audio signal based on the fundamental frequency probability density function estimated by the probability density estimation process, wherein

the probability density estimation process comprises;

a storage process of storing sound source structure data defining a constraint on one or more of sounds that can be simultaneously generated by a sound source of the input audio signal,

a form estimation process of selecting fundamental frequencies of one or more of sounds likely to be contained in the input audio signal with peaked weights from the various fundamental frequencies during the sequential updating and optimizing of the weights of the tone models corresponding to the various fundamental frequencies, so that the sounds of the selected fundamental frequencies satisfy the sound source structure data, and creating form data specifying the selected fundamental frequencies, and

a previous distribution impart process of imparting a previous distribution to the weights of the tone models corresponding to the various fundamental frequencies so as to emphasize weights corresponding to the fundamental frequencies specified by the form data created by the form estimation process.

12. A machine readable medium for use in a sound analysis apparatus having a processor for analyzing an input audio signal based on a weighted mixture of a plurality of tone models which represent harmonic structures of sound sources and which correspond to probability density functions of various fundamental frequencies, the machine readable medium containing program instructions executable by the processor for causing the sound synthesis apparatus to perform:

a probability density estimation process of sequentially updating and optimizing respective weights of the plurality of the tone models, so that a mixed distribution of frequencies obtained by the weighted mixture of the plurality of the tone models corresponding respectively to the various fundamental frequencies approximates a distribution of frequency components of the input audio signal, and estimating the optimized weights of the tone models to be a fundamental frequency probability density function of the various fundamental frequencies corresponding to the sound sources; and

a fundamental frequency determination process of determining an actual fundamental frequency of the input audio signal based on the fundamental frequency probability density function estimated by the probability density estimation process, wherein

the fundamental frequency determination process comprises;

a storage process of storing sound source structure data defining a constraint on one or more of sounds that can be simultaneously generated by a sound source of the input audio signal,

a form estimation process of selecting, from the various fundamental frequencies, fundamental frequencies of one or more of sounds which have weights peaked in the fundamental frequency probability density function estimated by the probability density estimation process and which are estimated to be likely contained in the input audio signal so that the selected fundamental frequencies satisfy the constraint defined by the sound source structure data, and creating form data representing the selected fundamental frequencies, and

a determination process of determining the actual fundamental frequency of the input audio signal based on the form data.

13. A machine readable medium for use in a sound analysis apparatus having a processor for analyzing an input audio signal based on a weighted mixture of a plurality of tone models which represent harmonic structures of sound sources and which correspond to probability density functions of various fundamental frequencies, the machine readable medium containing program instructions executable by the processor for causing the sound synthesis apparatus to perform:

a probability density estimation process of sequentially updating and optimizing respective weights of the plurality of the tone models, so that a mixed distribution of frequencies obtained by the weighted mixture of the plurality of the tone models corresponding to the various fundamental frequencies approximates a distribution of frequency components of the input audio signal, and estimating the optimized weights of the tone models to be a fundamental frequency probability density function of the various fundamental frequencies corresponding to the sound sources; and

a fundamental frequency determination process of determining an actual fundamental frequency of the input audio signal based on the fundamental frequency probability density function estimated by the probability density estimation process,

wherein the probability density estimation process comprises;

a storage process of storing sound source structure data defining a constraint on one or more of sounds that can be simultaneously generated by a sound source of the input audio signal,

a first update process of updating the weights of the tone models corresponding to the various fundamental frequencies a specific number of times for approximating the frequency components of the input audio signal,

a fundamental frequency selection process of obtaining fundamental frequencies with peaked weights based on the weights updated by the first update process from the various fundamental frequencies and selecting fundamental frequencies of one or ore more sounds likely to be contained in the input audio signal from the obtained fundamental frequencies with the peaked weights so that the selected fundamental frequencies satisfy the constraint defined by the sound source structure data, and

a second update process of imparting a previous distribution to the weights of the tone models corresponding to the various fundamental frequencies so as to emphasize the weights corresponding to the fundamental frequencies selected by the fundamental frequency selection process, and updating the weights of the tone models corresponding to the various fundamental frequencies a specific number of times for further approximating the frequency components of the input audio signal.