Singing voice synthesizing method, singing voice synthesizing device, program, recording medium, and robot

• US 20060185504A1
• Filed: 03/19/2004
• Published: 08/24/2006
• Est. Priority Date: 03/20/2003
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kobayashi, Kenichiro
• Granted Patent: US 7,189,915 B2
• Time in Patent Office: Days
• US Class Current: 84/645
• CPC Class Codes: G10H 1/0066 using a MIDI interface
• CPC Class Codes: G10H 2230/055 Spint toy, i.e. specifically designed for children, e.g. adapted for smaller fingers or simplified in some way; Musical instrument-shaped game input interfaces with simplified control features
• CPC Class Codes: G10H 2250/455 Gensound singing voices, i.e. generation of human voices for musical applications, vocal singing sounds or intelligible words at a desired pitch or with desired vocal effects, e.g. by phoneme synthesis
• CPC Class Codes: G10H 7/002 using a common processing for different operations or calculations, and a set of microinstructions (programme) to control the sequence thereof

Abstract

A singing voice synthesizing method for synthesizing the singing voice from performance data is disclosed. The input performance data are analyzed as the musical information of the pitch and the length of sounds and the lyric (S2 and S3). A track as the subject of the lyric is selected from the analyzed musical information (S5). A note the singing voice is allocated to is selected from the track (S6). The length of the note is changed to suit to a song being sung (S7). The voice quality suited to the singing is selected based on e.g. the track name/sequence name (S8) and singing voice data is prepared (S9). The singing voice is generated based on the singing voice data (S10).

Claims

1. A method for synthesizing the singing voice comprising

an analyzing step of analyzing performance data as the musical information of the pitch and the length of the sound and the lyric;

and

a singing voice generating step of generating the singing voice based on the musical information analyzed;

said singing voice generating step deciding on the type of said singing voice based on the information on the type of the sound included in the musical information analyzed.

2. The method for synthesizing the singing voice according to claim 1 wherein said performance data is performance data of a MIDI file.

3. The method for synthesizing the singing voice according to claim 2 wherein said singing voice generating step decides on the type of the singing voice based on the name of the musical instrument or the name of the track/name of the sequence contained in a track in the performance data of said MIDI file.

4. The method for synthesizing the singing voice according to claim 2 wherein said singing voice generating step allocates the time as from the timing of the note-on until the timing of the note-off of each sound of the singing voice as one sound of the singing voice, said timing of the note-on being the timing reference for the beginning of each sound of the singing voice.

5. The method for synthesizing the singing voice according to claim 4 wherein, with the timing of the note-on in said performance data of said MIDI file being the timing reference for the beginning of each sound of the singing voice, said singing voice generating step discontinuing the first sound of said singing voice in case there is a note-on of a second sound before note-off of said first note as a note superposed on said first note, even before the note-off of said first sound, said singing voice generating step causing enunciation of said second sound of the singing voice at a timing of the note-on of said second note.

6. The method for synthesizing the singing voice according to claim 5 wherein, if there is no superposition between said first and second notes in said performance data of said MIDI file, said singing voice generating step attenuates the sound volume of said first sound to render the break point from the second sound of the singing voice clear, said singing voice generating step not attenuating the sound volume in case there is superposition between said first and second notes and piecing said first and second notes together to express the slur in a musical air.

7. The method for synthesizing the singing voice according to claim 5 wherein, if there is no superposition between said first and second notes, but there is only a sound break interval between said first and second notes shorter than a pre-specified time, said singing voice generating step shifts the timing of the end of said first sound to the timing of the beginning of said second sound to piece the first and second notes together.

8. The method for synthesizing the singing voice according to claim 4 wherein, if there are plural notes having the same note-on timingin the performance data of said MIDI file, said singing voice generating step selects the note of the highest pitch as the sound of the song.

9. The method for synthesizing the singing voice according to claim 4 wherein, if there are plural notes having the same note-on timingin the performance data of said MIDI file, said singing voice generating step selects the note of the lowest pitch as the sound of the song.

10. The method for synthesizing the singing voice according to claim 4 wherein, if there are plural notes having the same note-on timingin the performance data of said MIDI file, said singing voice generating step selects the note of the maximum sound volume as the sound of the song.

11. The method for synthesizing the singing voice according to claim 4 wherein, if there are plural notes having the same note-on timingin the performance data of said MIDI file, said singing voice generating step treats the notes as separate voice parts and imparts the same lyric to the voice parts to generate the singing voice of different values of the pitch.

12. The method for synthesizing the singing voice according to claim 4 wherein, if the time length as from the note-on until note-off is shorter than a prescribed value, said singing voice generating step does not treat the note as the subject of singing.

13. The method for synthesizing the singing voice according to claim 4 wherein the time length as from the note-on until note-off is expanded a preset ratio to generate the singing voice.

14. The method for synthesizing the singing voice according to claim 13 wherein the data of said preset ratio used for changing the time as from the note-on until note-off is provided in a form associated with the names of the musical instruments.

15. The method for synthesizing the singing voice according to claim 4 wherein said singing voice generating step adds preset time to the time as from the note-on until note-off in said performance data of said MIDI file to generate the singing voice.

16. The method for synthesizing the singing voice according to claim 15 wherein the preset data for addition for changing the time as from the note-on until note-off is provided in a form associated with the names of the musical instruments.

17. The method for synthesizing the singing voice according to claim 4 wherein said singing voice generating step changes the time as from the note-on until note-off, and wherein said data for changing said time is set by an operator.

18. The method for synthesizing the singing voice according to claim 2 wherein said singing voice generating step sets the type of the singing voice from one name of the musical instrument to the next.

19. The method for synthesizing the singing voice according to claim 2 wherein, if the designation of the musical instrument is changed by a patch in the performance data of said MIDI file, said singing voice generating step changes the type of the singing voice even in the same track.

20. An apparatus for synthesizing the singing voice comprising

analyzing means for analyzing performance data as the musical information of the pitch and the length of the sound and the lyric;

and

singing voice generating means for generating the singing voice based on the musical information analyzed;

said singing voice generating means deciding on the type of said singing voice based on the information on the type of the sound included in the musical information analyzed.

21. The apparatus for synthesizing the singing voice according to claim 20 wherein said performance data is performance data of a MIDI file.

22. The apparatus for synthesizing the singing voice according to claim 21 wherein said singing voice generating step decides on the type of the singing voice based on the name of the musical instrument or the name of the track/name of the sequence contained in a track in the performance data of said MIDI file.

23. The apparatus for synthesizing the singing voice according to claim 21 wherein said singing voice generating means allocates the time as from the timing of the note-on until the timing of the note-off of each sound of the singing voice as one sound of the singing voice, said timing of the note-on in the performance data of the MIDI file being the timing reference for the beginning of each sound of the singing voice.

24. A program for having a computer execute preset processing, said program comprising

an analyzing step of analyzing performance data as the musical information of the pitch and the length of the sound and the lyric;

and

a singing voice generating step of generating the singing voice based on the musical information analyzed;

said singing voice generating step deciding on the type of said singing voice based on the information on the type of the sound included in the musical information analyzed.

25. The program according to claim 24 where said performance data is performance data of a MIDI file.

26. A computer-readable recording medium having recorded thereon a program for having a computer execute preset processing, said program comprising

an analyzing step of analyzing input performance data as the musical information of the pitch and the length of the sound and the lyric;

and

a singing voice generating step of generating the singing voice based on the musical information analyzed;

said singing voice generating step deciding on the type of said singing voice based on the information on the type of the sound included in the musical information analyzed.

27. The recording medium according to claim 26 where said performance data is performance data of a MIDI file.

28. An autonomous robot apparatus for performing movements based on the input information supplied, comprising

analyzing means for analyzing performance data as the musical information of the pitch and the length of the sound and the lyric;

and

singing voice generating means for generating the singing voice based on the musical information analyzed;

said singing voice generating means deciding on the type of said singing voice based on the information on the type of the sound included in the musical information analyzed.

29. The robot apparatus for synthesizing the singing voice according to claim 28 wherein said performance data is performance data of a MIDI file.