Formant converting apparatus modifying singing voice to emulate model voice

• US 5,750,912 A
• Filed: 01/16/1997
• Issued: 05/12/1998
• Est. Priority Date: 01/18/1996
• Status: Expired due to Term

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: Matsumoto, Shuichi
• Primary Examiner(s): Shoop, Jr., William M.
• Assistant Examiner(s): DONELS, JEFFREY
• Application Number:
  US08/784,815
• Time in Patent Office: 481 Days
• Field of Search: 84/600, 84/609, 84/610, 84/634, 84/636, 395/2.18, 395/2.7, 395/2.76, 434/307 A
• US Class Current: 84/609
• CPC Class Codes: G10H 1/366 with means for modifying or correcting the external signal, e.g. pitch correction, reverberation, changing a singer's voice
• CPC Class Codes: G10H 2220/011 Lyrics displays, e.g. for karaoke applications
• CPC Class Codes: G10H 2250/031 Spectrum envelope processing
• CPC Class Codes: G10H 2250/481 Formant synthesis, i.e. simulating the human speech production mechanism by exciting formant resonators, e.g. mimicking vocal tract filtering as in LPC synthesis vocoders, wherein musical instruments may be used as excitation signal to the time-varying filter estimated from a singer's speech gensound singing voices G10H2250/455; parcor synthesis G10H2250/505; effect Helmholtz G10H2210/275
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 2021/0135 Voice conversion or morphing
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence

Abstract

In a voice modifying apparatus for modifying a singing voice to emulate a model voice, a microphone collects the singing voice created by a singer. An analyzer sequentially analyzes the collected singing voice to extract therefrom actual formant data representing resonance characteristics of a singer'"'"'s own vocal organ which is physically activated to create the singing voice. A sequencer operates in synchronization with progression of the singing voice for sequentially providing reference formant data which indicates a vocal quality of the model voice and which is arranged to match with the progression of the singing voice. A comparator sequentially compares the actual formant data and the reference formant data with each other to detect a difference therebetween during the progression of the singing voice. An equalizer modifies frequency characteristics of the collected singing voice according to the detected difference so as to emulate the vocal quality of the model voice.

Claims

1. A voice modifying apparatus for modifying a singing voice to emulate a model voice, comprising:

an input section that collects the singing voice created by a singer;

an analyzing section that sequentially analyzes the collected singing voice to extract therefrom actual formant data representing resonance characteristics of a singer'"'"'s own vocal organ which is physically activated to create the singing voice;

a sequencer section that operates in synchronization with progression of the singing voice for sequentially providing reference formant data which indicates a vocal quality of the model voice and which is arranged to match with the progression of the singing voice;

a comparing section that sequentially compares the actual formant data and the reference formant data with each other to detect a difference therebetween during the progression of the singing voice; and

a modifying section that modifies frequency characteristics of the collected singing voice according to the detected difference so as to emulate the vocal quality of the model voice.

2. A voice modifying apparatus according to claim 1, wherein the sequencer section comprises a memory that stores a time-sequential pattern of the reference formant data provisionally sampled from a model singing sound of the model voice, and a sequencer that retrieves the time-sequential pattern of the reference formant data from the memory in synchronization with the progression of the singing voice.

3. A voice modifying apparatus according to claim 1, wherein the sequencer section comprises a memory that stores a set of formant data elements provisionally sampled from vowel components of the model voice, and a sequencer that sequentially retrieves the formant data elements from the memory in correspondence to vowel components contained in the singing voice so as to form the reference formant data in synchronization with the progression of the singing voice.

4. A voice modifying apparatus according to claim 3, wherein the memory further stores word data which indicates a sequence of phonemes to be voiced by the singer to create the singing voice and sequence data which indicates timings at which each of the phonemes is to be voiced, and wherein the sequencer analyzes the word data and the sequence data to identify each of the vowel components contained in the singing voice so that the sequencer can retrieve the formant data element corresponding to the identified vowel component.

5. A voice modifying apparatus according to claim 1, wherein the sequencer section comprises a memory that provisionally records a model singing sound of the model voice, and a sequencer that sequentially processes the recorded model singing sound to extract therefrom the reference formant data.

6. A voice modifying apparatus according to claim 1, wherein the analyzing section includes an envelope generator that provides the actual formant data in the form of a first envelope of a frequency spectrum of the singing voice, the sequencer section includes another envelope generator that provides the reference formant data in the form of a second envelope of a frequency spectrum of the model voice, the comparing section includes a comparator that differentially processing the first envelope and the second envelope with each other to detect an envelope difference therebetween, and the modifying section comprises an equalizer that modifies the frequency characteristics of the collected singing voice based on the detected envelope difference so as to equalize the frequency characteristics of the collected singing voice to those of the model voice.

7. A karaoke apparatus for producing a karaoke music to accompany a singing voice while modifying the singing voice to emulate a model voice, comprising:

a tone generating section that generates the karaoke music according to karaoke data;

an input section that collects the singing voice created by a karaoke player along with the karaoke music;

an analyzing section that sequentially analyzes the collected singing voice to extract therefrom actual formant data representing resonance characteristics of a karaoke player'"'"'s own vocal organ which is physically activated to create the singing voice;

a sequencer section that operates in synchronization with progression of the karaoke music for sequentially providing reference formant data which indicates a vocal quality of the model voice and which is arranged according to the karaoke data in matching with the progression of the singing voice;

a comparing section that sequentially compares the actual formant data and the reference formant data with each other to detect a difference therebetween;

a modifying section that modifies frequency characteristics of the collected singing voice according to the detected difference so as to emulate the vocal quality of the model voice; and

a mixer section that mixes the modified singing voice to the generated karaoke music in real time basis.

8. A karaoke apparatus according to claim 7, wherein the sequencer section comprises a memory that stores a set of formant data elements provisionally sampled from vowel components of the model voice, and a sequencer that sequentially retrieves the formant data elements from the memory in correspondence to vowel components contained in the singing voice so as to form the reference formant data in synchronization with the progression of the karaoke music.

9. A karaoke apparatus according to claim 8, wherein the memory further stores the karaoke data containing lyric word data which indicates a sequence of phonemes to be voiced by the karaoke player to create the singing voice and containing sequence data which indicates timings at which each of the phonemes is to be voiced, and wherein the sequencer analyzes the lyric word data and the sequence data to identify each of the vowel components contained in the singing voice so that the sequencer can retrieve the formant data element corresponding to the identified vowel component.

10. A karaoke apparatus according to claim 7, further comprising a requesting section that requests a desired one of the karaoke music which is originally sung by a professional singer so that the sequencer section provides the reference formant data which indicates a specific vocal quality of the model voice of the professional singer.

11. A method for modifying a singing voice to emulate a model voice, comprising the steps of:

collecting the singing voice created by a singer;

sequentially analyzing the collected singing voice to extract therefrom actual formant data representing resonance characteristics of a singer'"'"'s own vocal organ which is physically activated to create the singing voice;

sequentially providing in synchronization with progression of the singing voice reference formant data which indicates a vocal quality of the model voice and which is arranged to match with the progression of the singing voice;

sequentially comparing the actual formant data and the reference formant data with each other to detect a difference therebetween during the progression of the singing voice; and

modifying frequency characteristics of the collected singing voice according to the detected difference so as to emulate the vocal quality of the model voice.

12. The method according to claim 11, wherein the step of sequentially providing comprises supplying a memory with a time-sequential pattern of the reference formant data provisionally sampled from a model singing sound of the model voice, and retrieving the time-sequential pattern of the reference formant data from the memory in synchronization with the progression of the singing voice.

13. The method according to claim 11, wherein the step of sequentially providing comprises supplying a memory with a set of formant data elements provisionally sampled from vowel components of the model voice, and sequentially retrieving the formant data elements from the memory in correspondence to vowel components contained in the singing voice so as to form the reference formant data in synchronization with the progression of the singing voice.

14. The method according to claim 13, wherein the step of supplying further comprises supplying the memory with word data which indicates a sequence of phonemes to be voiced by the singer to create the singing voice and sequence data which indicates timings at which each of the phonemes is to be voiced, and the step of retrieving further comprises analyzing the word data and the sequence data to identify each of the vowel components contained in the singing voice so as to retrieve the formant data element corresponding to the identified vowel component.

15. The method according to claim 11, wherein the step of sequentially providing comprises recording a model singing sound of the model voice in a memory, and sequentially processing the recorded model singing sound to extract therefrom the reference formant data.

16. The method according to claim 11, wherein the step of sequentially analyzing comprises providing the actual formant data in the form of a first envelope of a frequency spectrum of the singing voice, the step of sequentially providing comprises providing the reference formant data in the form of a second envelope of a frequency spectrum of the model voice, the step of sequentially comparing comprises differentially processing the first envelope and the second envelope with each other to detect an envelope difference therebetween, and the step of modifying comprises modifying the frequency characteristics of the collected singing voice based on the detected envelope difference so as to equalize the frequency characteristics of the collected singing voice to those of the model voice.

17. A method for producing a karaoke music to accompany a singing voice while modifying the singing voice to emulate a model voice, comprising the steps of:

generating the karaoke music according to karaoke data;

collecting the singing voice created by a karaoke player along with the karaoke music;

sequentially analyzing the collected singing voice to extract therefrom actual formant data representing resonance characteristics of a karaoke player'"'"'s own vocal organ which is physically activated to create the singing voice;

sequentially providing in synchronization with progression of the karaoke music reference formant data which indicates a vocal quality of the model voice and which is arranged according to the karaoke data in matching with the progression of the singing voice;

sequentially comparing the actual formant data and the reference formant data with each other to detect a difference therebetween;

modifying frequency characteristics of the collected singing voice according to the detected difference so as to emulate the vocal quality of the model voice; and

mixing the modified singing voice to the generated karaoke music in real time basis.

18. The method according to claim 17, wherein the step of sequentially providing comprises supplying a memory with a set of formant data elements provisionally sampled from vowel components of the model voice, and sequentially retrieving the formant data elements from the memory in correspondence to vowel components contained in the singing voice so as to form the reference formant data in synchronization with the progression of the karaoke music.

19. The method according to claim 18, wherein the step of supplying further comprises supplying the memory with the karaoke data containing lyric word data which indicates a sequence of phonemes to be voiced by the karaoke player to create the singing voice and containing sequence data which indicates timings at which each of the phonemes is to be voiced, and wherein the step of sequentially retrieving comprises analyzing the lyric word data and the sequence data to identify each of the vowel components contained in the singing voice to thereby retrieve the formant data element corresponding to the identified vowel component.

20. The method according to claim 17, further comprising the step of requesting a desired one of the karaoke music which is originally sung by a professional singer so that the step of sequentially providing provides the reference formant data which indicates a specific vocal quality of the model voice of the professional singer.