MUSIC AUDIO SIGNAL GENERATING SYSTEM

• US 20120046771A1
• Filed: 02/16/2010
• Published: 02/23/2012
• Est. Priority Date: 02/17/2009
• Status: Active Grant

• Current Assignee: Kyoto University
• Original Assignee: Kyoto University
• Inventors: Itoyama, Katsutoshi, Okuno, Hiroshi, Abe, Takehiro, Yasuraoka, Naoki
• Granted Patent: US 8,831,762 B2
• Time in Patent Office: Days
• US Class Current: 700/94
• CPC Class Codes: G10H 1/16 by non-linear elements G10H1/14 takes precedence
• CPC Class Codes: G10H 2210/066 for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
• CPC Class Codes: G10H 2250/615 Waveform editing, i.e. setting or modifying parameters for waveform synthesis. graphical sound editing G10H2220/116
• CPC Class Codes: G10L 2021/0135 Voice conversion or morphing
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals

Abstract

A system for timbral change, capable of changing timbres included in an existing music audio signal to arbitrary timbres. Replaced harmonic peak parameters are created by replacing a plurality of harmonic peaks included in harmonic peak parameters, which are stored in a separated audio signal analyzing and storing section 3 and indicate relative amplitudes of n-th order harmonic components of each tone generated by a musical instrument of a first kind, with harmonic peaks included in harmonic peak parameters, which are stored in a replacement parameter storing section 6 and indicate relative amplitudes of n-th order harmonic components of each tone generated by a musical instrument of a second kind and corresponding to each tone generated by the musical instrument of the first kind. A synthesized separated audio signal generating section 7 generates a synthesized separated audio signal for each tone using parameters other than the harmonic peak parameters and the replaced harmonic peak parameters.

Claims

1. A music audio signal generating system comprising:

a signal extracting and storing section configured to extract a separated audio signal including only an audio signal of musical instrument sounds generated by a musical instrument of a first kind from a music audio signal including the audio signal of the musical instrument sounds generated by the musical instrument of the first kind and store the separated audio signal for each tone of the musical instrument sounds, and also store a residual audio signal;

a separated audio signal analyzing and storing section configured to analyze a plurality of parameters for each tone including at least harmonic peak parameters indicating relative amplitudes of n-th order harmonic components and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components and then store the plurality of parameters in order to represent the separated audio signal for each tone using a harmonic model that is formulated by the plurality of parameters;

a replacement parameter storing section configured to store harmonic peak parameters indicating relative amplitudes of n-th order harmonic components of a plurality of tones generated by a musical instrument of a second kind, the harmonic peak parameters being created from an audio signal of musical instrument sounds generated by the musical instrument of the second kind that is different from the musical instrument of the first kind, and required to represent, using the harmonic model, audio signals of the plurality of tones generated by the musical instrument of the second kind and corresponding to all of the tones included in the separated audio signal;

a replaced parameter creating and storing section configured to create replaced harmonic peak parameters by replacing a plurality of harmonic peaks included in the harmonic peak parameters, which are stored in the separated audio signal analyzing and storing section and indicate the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind, with harmonic peaks included in the harmonic peak parameters, which are stored in the replacement parameter storing section and indicate the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind, and then store the replaced harmonic peak parameters thus created;

a synthesized separated audio signal generating section configured to generate a synthesized separated audio signal for each tone using parameters other than the harmonic peak parameters, which are stored in the separated audio signal analyzing and storing section, and the replaced harmonic peak parameters stored in the replaced parameter creating and storing section; and

a signal adding section configured to add the synthesized separated audio signal and the residual audio signal to output a music audio signal including the audio signal of music instrument sounds generated by the musical instrument of the second kind.

8. The music audio signal generating system according to claim 1, further comprising an audio signal separating section configured to separate the music audio signal from a polyphonic audio signal including the music audio signal.

9. The music audio signal generating system according to claim 1, further comprising an audio signal separating section configured to separate the music audio signal from a polyphonic audio signal including the music audio signal, wherein audio signals other than the music audio signal are included in the residual audio signal.

10. The music audio signal generating system according to claim 9, wherein musical instrument sounds generated by the musical instrument of the second kind are acquired from another music audio signal obtained from the polyphonic audio signal including the music audio signal.

11. The music audio signal generating system according to claim 1, wherein the harmonic model is a harmonic model having inharmonicity of a harmonic structure incorporated thereinto.

12. The music audio signal generating system according to claim 1, further comprising a pitch manipulating section configured to manipulate pitch parameters relating to pitches and a duration manipulating section configured to manipulate duration parameters relating to durations, wherein the pitch parameters and the duration parameters are included in a plurality of parameters to be analyzed by the separated audio signal analyzing and storing section.

20. The music audio signal generating system according to claim 1, further comprising a musical score manipulating section configured to generate an audio signal of musical instrument sounds generated by the musical instrument of the first or second kind when a musical score is played with the musical instrument of the first or second kind, by utilizing the plurality of parameters for each tone stored in the separated audio signal analyzing and storing section.

21. The music audio signal generating system according to claim 20, wherein the musical score manipulating section is configured to create pitch parameters relating to pitches, duration parameters relating to durations, and timbre parameters relating to timbres among parameters constructing a harmonic model such that the created parameters may be suitable to each tone in a musical structure of another musical score.

2. A music audio signal generating system comprising:

a signal extracting and storing section configured to extract a separated audio signal including only an audio signal of musical instrument sounds generated by a musical instrument of a first kind from a music audio signal including the audio signal of the musical instrument sounds generated by the musical instrument of the first kind and store the separated audio signal for each tone of the musical instrument sounds, and also store a residual audio signal;

a separated audio signal analyzing and storing section configured to analyze a plurality of parameters for each tone including at least harmonic peak parameters indicating relative amplitudes of n-th order harmonic components and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components and then store the plurality of parameters in order to represent the separated audio signal for each tone using a harmonic model that is formulated by the plurality of parameters;

a replacement parameter storing section configured to store harmonic peak parameters indicating relative amplitudes of n-th order harmonic components of a plurality of tones generated by a musical instrument of a second kind and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components, the harmonic peak parameters and the power envelop parameters being created from an audio signal of musical instrument sounds generated by the musical instrument of the second kind that is different from the musical instrument of the first kind, and required to represent, using the harmonic model, audio signals of the plurality of tones generated by the musical instrument of the second kind and corresponding to all of the tones included in the separated audio signal;

a replaced parameter creating and storing section configured to create replaced harmonic peak parameters by replacing a plurality of harmonic peaks included in the harmonic peak parameters, which are stored in the separated audio signal analyzing and storing section and indicate the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind, with harmonic peaks included in the harmonic peak parameters, which are stored in the replacement parameter storing section and indicate the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind, and then store the replaced harmonic peak parameters thus created, and also configured to create replaced power envelope parameters by replacing the power envelope parameters, which are stored in the separated audio signal analyzing and storing section and indicate the temporal power envelopes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind, with the power envelope parameters, which are stored in the replacement parameter storing section and indicate the temporal power envelopes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind, and then store the replaced power envelope parameters thus created;

a synthesized separated audio signal generating section configured to generate a synthesized separated audio signal for each tone using parameters other than the harmonic peak parameters and the power envelope parameters, which are stored in the separated audio signal analyzing and storing section, as well as the replaced harmonic peak parameters and the replaced power envelope parameters stored in the replaced parameter creating and storing section; and

a signal adding section configured to add the synthesized separated audio signal and the residual audio signal to output a music audio signal including the audio signal of music instrument sounds generated by the musical instrument of the second kind.

4. The music audio signal generating system according to claim 2, wherein:

the separated audio signal analyzing and storing section further has a function of storing an inharmonic component distribution parameter indicating the distribution of inharmonic components of each of the tones of a plurality of kinds generated by the musical instrument of the first kind;

the replacement parameter storing section further has a function of storing an inharmonic component distribution parameter indicating the distribution of inharmonic components of each of the tones of a plurality of kinds included in the audio signal of the musical instrument sounds generated by the musical instrument of the second kind;

the replaced parameter creating and storing section further has a function of creating a replaced inharmonic component distribution parameter indicating the distribution of inharmonic components of each tone by replacing the inharmonic component distribution parameter, which is stored in the separated audio signal analyzing and storing section, for each tone included in the musical instrument sounds generated by the musical instrument of the first kind with the inharmonic component distribution parameter, which is stored in the replacement parameter storing section, for each tone included in the musical instrument sounds generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind, and then storing the replaced inharmonic component distribution parameter thus created; and

the synthesized separated audio signal generating section generates a synthesized separated audio signal for each tone using parameters other than the harmonic peak parameter, the power envelope parameter, and the inharmonic component distribution parameter, which are stored in the separated audio signal analyzing and storing section, as well as the replaced harmonic peak parameter, the replaced power envelope parameter, and the replaced inharmonic component distribution parameter that are stored in the replaced parameter creating and storing section.

5. The music audio signal generating system according to claim 2, wherein:

the replacement parameter storing section comprises;

a parameter analyzing and storing section configured to analyze and store at least harmonic peak parameters for tones of a plurality of kinds that are obtained from an audio signal of musical instrument sounds generated by the musical instrument of the second kind, the harmonic peak parameters indicating relative amplitudes of n-th order harmonic components for each tone and required to represent a separated audio signal for each tone using the harmonic model, and also configured to store power envelope parameters indicating temporal power envelopes of the n-th order harmonic components for each of tones of the plurality of kinds;

a parameter interpolation creating and storing section configured to create the harmonic peak parameters by an interpolation method for tones other than the tones of the plurality of kinds among the tones generated by the musical instrument of the second kind and corresponding to all of the tones included in the separated audio signal, based on the harmonic peak parameters and the power envelope parameters that are stored in the parameter analyzing and storing section, the harmonic peak parameters being required to represent the tones other than the tones of the plurality of kinds using the harmonic model, and then store the harmonic peak parameters thus created; and

the parameter analyzing and storing section stores the power envelope parameters indicating temporal power envelopes of the n-th order harmonic components, which are obtained by analysis, as representative power envelope parameters.

6. The music audio signal generating system according to claim 2, wherein:

the replacement parameter storing section comprises;

a parameter analyzing and storing section configured to analyze and store at least harmonic peak parameters indicating relative amplitudes of n-th order harmonic components of each of the tones of a plurality of kinds and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components; and

a parameter interpolation creating and storing section configured to create the harmonic peak parameters and the power envelope parameters by an interpolation method for tones other than the tones of the plurality of kinds among the tones generated by the musical instrument of the second kind and corresponding to all of the tones included in the separated audio signal, based on the harmonic peak parameters and the power envelope parameters that are stored in the parameter analyzing and storing section, the harmonic peak parameters and the power envelope parameters being required to represent an audio signal of the tones other than the tones of the plurality of kinds using the harmonic model, and then store the harmonic peak parameters and the power envelope parameters thus created.

7. The music audio signal generating system according to claim 5, wherein:

the replacement parameter storing section further comprises a function generating and storing section configured to store the harmonic peak parameters for each tone generated by the music instrument of the second kind as pitch-dependent feature functions, based on data stored in the parameter analyzing and storing section and the parameter interpolation creating and storing section; and

the replaced parameter creating and storing section is configured to acquire a plurality of peaks included in the harmonic peak parameters for each tone generated by the music instrument of the second kind from the pitch-dependent feature functions.

3. A music audio signal generating system comprising:

a signal extracting and storing section configured to extract a separated audio signal including only an audio signal of musical instrument sounds generated by a musical instrument of a first kind from a music audio signal including the audio signal of the musical instrument sounds generated by the musical instrument of the first kind, and store the separated audio signal for each tone of the musical instrument sounds, and also store a residual audio signal;

a separated audio signal analyzing and storing section configured to analyze a plurality of parameters for each tone including at least harmonic peak parameters indicating relative amplitudes of n-th order harmonic components and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components and then store the plurality of parameters in order to represent the separated audio signal for each tone using a harmonic model that is formulated by the plurality of parameters;

a replacement parameter storing section configured to store harmonic peak parameters indicating relative amplitudes of n-th order harmonic components of a plurality of tones generated by a musical instrument of a second kind and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components, the harmonic peak parameters and the power envelop parameters being created from an audio signal of musical instrument sounds generated by the musical instrument of the second kind that is different from the musical instrument of the first kind, and required to represent, using the harmonic model, audio signals of the plurality of tones generated by the musical instrument of the second kind and corresponding to all of the tones included in the music separated audio signal;

a musical instrument category determining section configured to determine whether or not the musical instrument of the first kind and the musical instrument of the second kind belong to the same category of musical instruments;

a replaced parameter creating and storing section configured to create replaced harmonic peak parameters by replacing a plurality of harmonic peaks included in the harmonic peak parameters, which are stored in the separated audio signal analyzing and storing section and indicate the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind, with harmonic peaks included in the harmonic peak parameters, which are stored in the replacement parameter storing section and indicate the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind, and then store the replaced harmonic peak parameters thus created, and also configured to create replaced power envelope parameters by replacing the power envelope parameters, which are stored in the separated audio signal analyzing and storing section and indicate the temporal power envelopes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind, with the power envelope parameters, which are stored in the replacement parameter storing section and indicate the temporal power envelopes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind, and then store the replaced power envelope parameters thus created;

a synthesized separated audio signal generating section configured to generate a synthesized separated audio signal for each tone, using parameters other than the harmonic peak parameters, which are stored in the separated audio signal analyzing and storing section, and the replaced harmonic peak parameters stored in the replaced parameter creating and storing section if the music instrument category determining section determines that the musical instrument of the first kind and the musical instrument of the second kind belong to the same category, or using parameters other than the harmonic peak parameters and the power envelope parameters, which are stored in the separated audio signal analyzing and storing section, as well as the replaced harmonic peak parameters and the replaced power envelope parameters stored in the replaced parameter creating and storing section if the music instrument category determining section determines that the musical instrument of the first kind and the musical instrument of the second kind belong to different categories; and

a signal adding section configured to add the synthesized separated audio signal and the residual audio signal to output a music audio signal including the audio signal of music instrument sounds generated by the musical instrument of the second kind.

13. A music audio signal generating method implemented in a computer to cause the computer to execute the steps of:

extracting a separated audio signal including only an audio signal of each tone included in musical instrument sounds generated by a musical instrument of a first kind from a music audio signal including the audio signal of the musical instrument sounds generated by the musical instrument of the first kind, and also extracting a residual audio signal;

analyzing a plurality of parameters for each tone including at least harmonic peak parameters indicating relative amplitudes of n-th order harmonic components and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components in order to represent the separated audio signal for each tone using a harmonic model that is formulated by the plurality of parameters;

creating harmonic peak parameters indicating relative amplitudes of n-th order harmonic components of each tone generated by a musical instrument of a second kind based on an audio signal of musical instrument sounds generated by the musical instrument of the second kind that is different from the musical instrument of the first kind, wherein the harmonic peak parameters are required to represent, using the harmonic model, audio signals of a plurality of tones generated by the musical instrument of the second kind and corresponding to all of the tones included in the separated audio signal;

creating replaced harmonic peak parameters by replacing a plurality of harmonic peaks included in the harmonic peak parameters indicating the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind with a plurality of harmonic peaks included in the harmonic peak parameters indicating the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind;

generating a synthesized separated audio signal for each tone using parameters other than the harmonic peak parameters and the replaced harmonic peak parameters; and

adding the synthesized separated audio signal and the residual audio signal to output a music audio signal including the audio signal of music instrument sounds generated by the musical instrument of the second kind.

14. A music audio signal generating method implemented in a computer to cause the computer to execute the steps of:

extracting a separated audio signal including only an audio signal of each tone included in musical instrument sounds generated by a musical instrument of a first kind from a music audio signal including the audio signal of the musical instrument sounds generated by the musical instrument of the first kind, and also extracting a residual audio signal;

analyzing a plurality of parameters for each tone including at least harmonic peak parameters indicating relative amplitudes of n-th order harmonic components and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components in order to represent the separated audio signal for each tone using a harmonic model that is formulated by the plurality of parameters;

creating harmonic peak parameters indicating relative amplitudes of n-th order harmonic components of each tone generated by a musical instrument of a second kind and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components based on an audio signal of musical instrument sounds generated by the musical instrument of the second kind that is different from the musical instrument of the first kind, wherein the harmonic peak parameters and the power envelope parameters are required to represent, using the harmonic model, audio signals of the tones generated by the musical instrument of the second kind and corresponding to all of the tones included in the separated audio signal;

creating replaced harmonic peak parameters by replacing a plurality of harmonic peaks included in the harmonic peak parameters indicating the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind with a plurality of harmonic peaks included in the harmonic peak parameters indicating the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind, and also creating replaced power envelope parameters by replacing a feature region for the power envelope parameters indicating the temporal power envelopes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind with a feature region for the power envelope parameters indicating the temporal power envelopes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind;

generating a synthesized separated audio signal for each tone using parameters other than the harmonic peak parameters and the power envelope parameters as well as the replaced harmonic peak parameters and the replaced power envelope parameters; and

adding the synthesized separated audio signal and the residual audio signal to output a music audio signal including the audio signal of music instrument sounds generated by the musical instrument of the second kind.

15. A music audio signal generating method implemented in a computer to cause the computer to execute the steps of:

extracting a separated audio signal including only an audio signal of each tone included in musical instrument sounds generated by a musical instrument of a first kind from a music audio signal including the audio signal of the musical instrument sounds generated by the musical instrument of the first kind, and also extracting a residual audio signal;

analyzing a plurality of parameters for each tone including at least harmonic peak parameters indicating relative amplitudes of n-th order harmonic components and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components in order to represent the separated audio signal for each tone using a harmonic model that is formulated by the plurality of parameters;

creating harmonic peak parameters indicating relative amplitudes of n-th order harmonic components of each tone generated by a musical instrument of a second kind and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components based on an audio signal of musical instrument sounds generated by the musical instrument of the second kind that is different from the musical instrument of the first kind, wherein the harmonic peak parameters and the power envelope parameters are required to represent, using the harmonic model, audio signals of the tones generated by the musical instrument of the second kind and corresponding to all of the tones included in the music separated audio signal;

determining whether or not the musical instrument of the first kind and the musical instrument of the second kind belong to the same category of musical instruments;

creating replaced harmonic peak parameters by replacing a plurality of harmonic peaks included in the harmonic peak parameters indicating the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind with a plurality of harmonic peaks included in the harmonic peak parameters stored in the replacement parameter storing section and indicating the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind, and also creating replaced power envelope parameters by replacing a feature region for the power envelope parameters indicating the temporal power envelopes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind with a feature region for the power envelope parameters indicating the temporal power envelopes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind;

generating a synthesized separated audio signal for each tone using parameters other than the harmonic peak parameters and the replaced harmonic peak parameters if the music instrument category determining section determines that the musical instrument of the first kind and the musical instrument of the second kind belong to the same category, or using parameters other than the harmonic peak parameters and the power envelope parameters as well as the replaced harmonic peak parameters and the replaced power envelope parameters if the music instrument category determining section determines that the musical instrument of the first kind and the musical instrument of the second kind belong to different categories; and

adding the synthesized separated audio signal and the residual audio signal to output a music audio signal including the audio signal of music instrument sounds generated by the musical instrument of the second kind.

16. A computer program for music audio signal generation installed in a computer to cause the computer to execute the steps of:

extracting a separated audio signal including only an audio signal of each tone included in musical instrument sounds generated by a musical instrument of a first kind from a music audio signal including the audio signal of the musical instrument sounds generated by the musical instrument of the first kind, and also extracting a residual audio signal;

analyzing a plurality of parameters for each tone including at least harmonic peak parameters indicating relative amplitudes of n-th order harmonic components and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components in order to represent the separated audio signal for each tone using a harmonic model that is formulated by the plurality of parameters;

creating harmonic peak parameters indicating relative amplitudes of n-th order harmonic components of each tone generated by a musical instrument of a second kind based on an audio signal of musical instrument sounds generated by the musical instrument of the second kind that is different from the musical instrument of the first kind, wherein the harmonic peak parameters are required to represent, using the harmonic model, audio signals of the tones generated by the musical instrument of the second kind and corresponding to all of the tones included in the music separated audio signal;

creating replaced harmonic peak parameters by replacing a plurality of harmonic peaks included in the harmonic peak parameters indicating the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind with a plurality of harmonic peaks included in the harmonic peak parameters indicating the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind;

generating a synthesized separated audio signal for each tone using parameters other than the harmonic peak parameters and the replaced harmonic peak parameters; and

adding the synthesized separated audio signal and the residual audio signal to output a music audio signal including the audio signal of music instrument sounds generated by the musical instrument of the second kind.

19. A computer readable recording medium recorded with the computer program for music audio signal generation according to claim 16.

17. A computer program for music audio signal generation installed in a computer to cause the computer to execute the steps of:

extracting a separated audio signal including only an audio signal of each tone included in musical instrument sounds generated by a musical instrument of a first kind from a music audio signal including the audio signal of the musical instrument sounds generated by the musical instrument of the first kind, and also extracting a residual audio signal;

analyzing a plurality of parameters for each tone including at least harmonic peak parameters indicating relative amplitudes of n-th order harmonic components and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components in order to represent the separated audio signal for each tone using a harmonic model that is formulated by the plurality of parameters;

creating harmonic peak parameters indicating relative amplitudes of n-th order harmonic components of each tone generated by a musical instrument of a second kind and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components based on an audio signal of musical instrument sounds generated by the musical instrument of the second kind that is different from the musical instrument of the first kind, wherein the harmonic peak parameters and the power envelope parameters are required to represent, using the harmonic model, audio signals of the tones generated by the musical instrument of the second kind and corresponding to all of the tones included in the separated audio signal;

creating replaced harmonic peak parameters by replacing a plurality of harmonic peaks included in the harmonic peak parameters indicating the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind with a plurality of harmonic peaks included in the harmonic peak parameters indicating the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind, and also creating replaced power envelope parameters by replacing a feature region for the power envelope parameters indicating the temporal power envelopes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind with a feature region for the power envelope parameters indicating the temporal power envelopes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind;

generating a synthesized separated audio signal for each tone using parameters other than the harmonic peak parameters and the power envelope parameters as well as the replaced harmonic peak parameters and the replaced power envelope parameters; and

adding the synthesized separated audio signal and the residual audio signal to output a music audio signal including the audio signal of music instrument sounds generated by the musical instrument of the second kind.

18. A computer program for music audio signal generation installed in a computer to cause the computer to execute the steps of:

extracting a separated audio signal including only an audio signal of each tone included in musical instrument sounds generated by a musical instrument of a first kind from a music audio signal including the audio signal of the musical instrument sounds generated by the musical instrument of the first kind, and also extracting a residual audio signal;

analyzing a plurality of parameters for each tone including at least harmonic peak parameters indicating relative amplitudes of n-th order harmonic components and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components in order to represent the separated audio signal for each tone using a harmonic model that is formulated by the plurality of parameters;

creating harmonic peak parameters indicating relative amplitudes of n-th order harmonic components of each tone generated by a musical instrument of a second kind and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components based on an audio signal of musical instrument sounds generated by the musical instrument of the second kind that is different from the musical instrument of the first kind, wherein the harmonic peak parameters and the power envelope parameters are required to represent, using the harmonic model, audio signals of the tones generated by the musical instrument of the second kind and corresponding to all of the tones included in the separated audio signal;

determining whether or not the musical instrument of the first kind and the musical instrument of the second kind belong to the same category of musical instruments;

creating replaced harmonic peak parameters by replacing a plurality of harmonic peaks included in the harmonic peak parameters indicating the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind with a plurality of harmonic peaks included in the harmonic peak parameters and indicating the relative amplitudes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind, and also creating replaced power envelope parameters by replacing a feature region for the power envelope parameters indicating the temporal power envelopes of the n-th order harmonic components of each tone generated by the musical instrument of the first kind with a feature region for the power envelope parameters indicating the temporal power envelopes of the n-th order harmonic components of each tone generated by the musical instrument of the second kind and corresponding to each tone generated by the musical instrument of the first kind;

generating a synthesized separated audio signal for each tone using parameters other than the harmonic peak parameters and the replaced harmonic peak parameters if the music instrument category determining section determines that the musical instrument of the first kind and the musical instrument of the second kind belong to the same category, or using parameters other than the harmonic peak parameters and the power envelope parameters as well as the replaced harmonic peak parameters and the replaced power envelope parameters if the music instrument category determining section determines that the musical instrument of the first kind and the musical instrument of the second kind belong to different categories; and

adding the synthesized separated audio signal and the residual audio signal to output a music audio signal including the audio signal of music instrument sounds generated by the musical instrument of the second kind.

22. A music audio signal generating system comprising:

a signal extracting and storing section configured to extract a separated audio signal including only an audio signal of musical instrument sounds generated by a musical instrument when a performer plays a musical score with the musical instrument, from a music audio signal including the audio signal of the musical instrument sounds and store the separated audio signal thus extracted for each tone included in the musical instrument sounds;

a separated audio signal analyzing and storing section configured to analyze a plurality of parameters for each tone including at least harmonic peak parameters indicating relative amplitudes of n-th order harmonic components and power envelope parameters indicating temporal power envelopes of the n-th order harmonic components in order to represent the separated audio signal for each tone using a harmonic model that is formulated by the plurality of parameters and store the parameters thus created; and

a musical score manipulating section configured to generate an audio signal of musical instrument sounds generated when the performer plays another musical score different from the musical score with the musical instrument by utilizing the plurality of parameters for each tone stored in the separated audio signal analyzing and storing section.