Electronic musical apparatus for synthesizing vocal sounds using format sound synthesis techniques

• US 5,703,311 A
• Filed: 07/29/1996
• Issued: 12/30/1997
• Est. Priority Date: 08/03/1995
• Status: Expired due to Term

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Yamaha Corporation
• Inventors: Ohta, Shinichi
• Primary Examiner(s): Shoop, Jr., William M.
• Assistant Examiner(s): DONELS, JEFFREY
• Application Number:
  US08/687,976
• Time in Patent Office: 519 Days
• Field of Search: 84/600, 84/622, 84/659, 395/2.16-2.19, 395/2.87
• US Class Current: 84/622
• CPC Class Codes: G10H 2250/455 Gensound singing voices, i.e. generation of human voices for musical applications, vocal singing sounds or intelligible words at a desired pitch or with desired vocal effects, e.g. by phoneme synthesis formant synthesis G10H2250/481; parcor synthesis G10H2250/505; modulation effects G10H2210/195; ensemble effects G10H2210/245; speech synthesis in general G10L13/00
• CPC Class Codes: G10H 2250/481 Formant synthesis, i.e. simulating the human speech production mechanism by exciting formant resonators, e.g. mimicking vocal tract filtering as in LPC synthesis vocoders, wherein musical instruments may be used as excitation signal to the time-varying filter estimated from a singer's speech gensound singing voices G10H2250/455; parcor synthesis G10H2250/505; effect Helmholtz G10H2210/275
• CPC Class Codes: G10H 7/10 using coefficients or parameters stored in a memory, e.g. Fourier coefficients G10H7/12 takes precedence
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 25/15 the extracted parameters being formant information

Abstract

An electronic musical apparatus is designed to sing a song based on performance data which indicate a melody originally played by a musical instrument. Herein, the apparatus contains a formant tone generator and a data memory which stores a plurality of formant data, lyric data and melody data. Formant synthesis method is employed for voice synthesis to generate voices based on the plurality of formant data selectively designated by the lyric data so that the voices are sequentially generated in accordance with words of a song. Thus, the song is automatically swung by sequentially generating the voices in accordance with a melody which is designated by. the melody data; and the voice synthesis is controlled such that generation of the voices temporarily stopped at timings of pausing for breath. Moreover, the data memory can store formant parameters with respect to each phoneme, so that the formant tone generator can gradually shift sounding thereof from a first phoneme (e.g., a consonant) to a second phoneme (e.g., a vowel). Herein, formant parameters, regarding the first phoneme, are supplied to the formant tone generator in a pre-interpolation time between a first phoneme sounding-start-time and an interpolation start time. Then, Interpolation is effected on the formant parameters to achieve gradual shifting of the sounding. A pace for the shifting of the sounding from the first phoneme to the second phoneme can be changed on demand.

Claims

1. An electronic musical apparatus comprising:

storage means for storing formant data and performance data, the performance data including lyric data corresponding to words of a lyric of a song to be sung and melody data corresponding to a melody of the song, the lyric data including breath data, the melody data including sounding duration data;

voice synthesis means for synthesizing voices corresponding to the words of the lyric, based on the formant data and the performance data which designate sounds to be generated in accordance with the melody of the song; and

control means for controlling the voice synthesis means such that the sounds are generated in accordance with the performance data, the control means accessing the storage means to sequentially read out the performance data in accordance with the melody to supply the voice synthesizing means with the corresponding formant data, wherein the control means controls a sounding duration of the sounds in accordance with at least one of the breath data and the sounding duration data.

2. An electronic musical apparatus comprising:

storage means for storing words of a lyric of a song to be sung as well as breath information indicating timings of pausing for breath;

voice synthesis means for automatically synthesizing voices, corresponding to the words of the lyric, based on performance data which indicate a melody of the song; and

control means for controlling the voice synthesis means based on the breath information such that the voices are generated in accordance with the melody but generation of the voices is temporarily stopped at the timings of pausing for breath.

3. An electronic musical apparatus according to claim 2 wherein the performance data are stored in the storage means.

4. An electronic musical apparatus comprising:

storage means for storing a plurality of formant data, lyric data and melody data with respect to a song to be sung, wherein the plurality of formant data respectively correspond to syllables of a language by which the song is sung whilst the lyric data designate words of a lyric of the song as well as timings of pausing for breath;

formant synthesis means for synthesizing voices based on the plurality of formant data selectively designated by the lyric data so that the voices are sequentially generated in accordance with the words of the lyric and are collected to sing the song in accordance with a melody of the song designated by the melody data; and

breath control means for controlling the formant synthesis means such that the voices are generated but generation of the voices is temporarily stopped at the timings of pausing for breath.

5. An electronic musical apparatus according to claim 4 wherein the formant synthesis means consists of a plurality of tone-generator channels, each of which consists of vowel formant generating sections and consonant formant generating sections which selectively cooperate with each other to form a voice corresponding to a syllable of the language.

6. An electronic musical instrument according to claim 4 wherein the plurality of formant data respectively correspond to 50 vocal sounds of the Japanese syllabary.

7. A formant parameter creating method, applicable to an electronic musical apparatus employing a formant tone generator which operates based on formant parameters, comprising the steps of:

outputting formant parameters, corresponding to a first phoneme, to the formant tone generator in a duration which is determined in advance;

starting interpolation on the formant parameters after a lapse of the duration, wherein the interpolation is effected to shift sounding of the formant tone generator from the first phoneme to a second phoneme; and

sequentially outputting results of the interpolation to the formant tone generator,

whereby the formant tone generator synthesizes formant-related sound based on the first and second phonemes.

8. A formant parameter creating method, applicable to an electronic musical apparatus employing a formant tone generator which operates based on formant parameters, comprising the steps of:

multiplying a sounding time of a first phoneme by an interpolation dead rate, which is determined in advance, so as to calculate a pre-interpolation time between a first phoneme sounding-start-time and an interpolation start time;

outputting formant parameters, corresponding to the first phoneme, to the formant tone generator during the pre-interpolation time;

detecting a lapse of the pre-interpolation time;

starting interpolation on the formant parameters after the lapse of the pre-interpolation time, wherein the interpolation is effected to shift sounding of the formant tone generator from the first phoneme to a second phoneme; and

sequentially outputting results of the interpolation to the formant tone generator,

whereby the formant tone generator synthesizes formant-related sound based on the first and second phonemes.

9. A formant parameter creating method according to claim 8 wherein the interpolation dead rate is a constant which is commonly used for each phoneme.

10. A formant parameter creating method according to claim 8 wherein the interpolation dead rate is determined with respect to each phoneme.

11. A formant parameter creating method according to claim 8 wherein an interpolation-dead-rate adjusting coefficient is further used such that the sounding time of the first phoneme is multiplied by the interpolation dead rate and the interpolation-dead-rate adjusting coefficient so as to calculate a pre-interpolation time between the first phoneme sounding-start-time and the interpolation start time, wherein the interpolation-dead-rate adjusting coefficient is determined with respect to each phoneme.

12. A formant parameter creating method, applicable to an electronic musical apparatus employing a formant tone generator which operates based on formant parameters, comprising the steps of:

multiplying a sounding time of a first phoneme by an interpolation dead rate, which is determined in advance, so as to calculate a pre-interpolation time between a first phoneme sounding-start-time and an interpolation start time;

subtracting the pre-interpolation time from the sounding time of the first phoneme so as to calculate an interpolation time;

outputting formant parameters, corresponding to the first phoneme, to the formant tone generator during the pre-interpolation time;

detecting a lapse of the pre-interpolation time;

starting interpolation, using the interpolation time, on the formant parameters after the lapse of the pre-interpolation time, wherein the interpolation is effected to shift sounding of the formant tone generator from the first phoneme to a second phoneme;

sequentially outputting results of the interpolation to the formant tone generator;

detecting a lapse of the interpolation time; and

outputting formant parameters, regarding the second phoneme, to the formant tone generator,

whereby the formant tone generator synthesizes formant-related sound based on the first and second phonemes.

13. A formant parameter creating method, applicable to an electronic musical apparatus employing a formant tone generator which operates based on formant parameters, comprising the steps of:

multiplying a sounding time of a first phoneme by an interpolation dead rate, which is determined in advance, so as to calculate a pre-interpolation time between a first phoneme sounding-start-time and an interpolation start time;

subtracting a sum of the pre-interpolation time and a sounding time of a second phoneme from the sounding time of the first phoneme so as to calculate an interpolation time;

outputting formant parameters, corresponding to the first phoneme, to the formant tone generator during the pre-interpolation time;

detecting a lapse of the pre-interpolation time;

starting interpolation, using a sum of the interpolation time and the sounding time of the second phoneme, on the formant parameters after the lapse of the pre-interpolation time, wherein the interpolation is effected to shift sounding of the formant tone generator from the first phoneme to the second phoneme;

sequentially outputting results of the interpolation to the formant tone generator;

detecting a lapse of the interpolation time; and

outputting formant parameters, regarding the second phoneme, to the formant tone generator after the lapse of the interpolation time,

whereby the formant tone generator synthesizes formant-related sound based on the first and second phonemes.

14. A formant parameter creating method, applicable to an electronic musical apparatus employing a formant tone generator which operates based on formant parameters, comprising the steps of:

performing first interpolation at an initial stage of a sounding time of a first phoneme such that formant parameters of the first phoneme are shifted to formant parameters of a second phoneme at a first pace;

sequentially outputting results of the first interpolation to the formant tone generator;

performing second interpolation at a latter stage of the sounding time of the first phoneme such that formant parameters of the first phoneme are shifted to the formant parameters of the second phoneme at a second pace, wherein said first pace is slower than said second pace; and

sequentially outputting results of the second interpolation to the formant tone generator,

whereby the formant tone generator synthesizes formant-related sound based on the first and second phonemes.

15. A storage device storing a plurality of formant data, lyric data and melody data with respect to a song to be sung, wherein the plurality of formant data respectively correspond to syllables of a language by which the song is sung whilst the lyric data designate words of a lyric of the song as well as timings of pausing for breath, the storage device further storing programs which cause an electronic musical apparatus to execute a lyric performance method comprising the steps of:

synthesizing voices based on the plurality of formant data selectively designated by the lyric data so that the voices are sequentially generated in accordance with the words of the lyric and are collected to sing the song in accordance with a melody of the song designated by the melody data; and

controlling the voices such that the voices are generated but generation of the voices is temporarily stopped at the timings of pausing for breath.

16. A storage device storing programs and formant parameters which cause an electronic musical apparatus, employing a formant tone generator, to execute a formant parameter creating method comprising the steps of:

outputting formant parameters, corresponding to a first phoneme, to the formant tone generator in a duration which is determined in advance;

starting interpolation on the formant parameters after a lapse of the duration, wherein the interpolation is effected to shift sounding of the formant tone generator from the first phoneme to a second phoneme; and

sequentially outputting results of the interpolation to the formant tone generator,

whereby the formant tone generator synthesizes formant-related sound based on the first and second phonemes.

17. A storage device according to claim 16 wherein the first phoneme corresponds to a consonant whilst the second phoneme corresponds to a vowel.