Electronic apparatus for simulating singing of song

• US 4,731,847 A
• Filed: 04/26/1982
• Issued: 03/15/1988
• Est. Priority Date: 04/26/1982
• Status: Expired due to Term

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Lin, Kun-Shan, Frantz, Gene A., Lybrook, Gilbert A.
• Primary Examiner(s): Kemeny, Emanuel S.
• Application Number:
  US06/372,257
• Time in Patent Office: 2,150 Days
• Field of Search: 381/51-53, 84/1.01, 84/1.03, 364/513.5
• US Class Current: 704/260
• CPC Class Codes: G10H 2250/455 Gensound singing voices, i.e. generation of human voices for musical applications, vocal singing sounds or intelligible words at a desired pitch or with desired vocal effects, e.g. by phoneme synthesis formant synthesis G10H2250/481; parcor synthesis G10H2250/505; modulation effects G10H2210/195; ensemble effects G10H2210/245; speech synthesis in general G10L13/00
• CPC Class Codes: G10H 2250/601 Compressed representations of spectral envelopes, e.g. LPC [linear predictive coding], LAR [log area ratios], LSP [line spectral pairs], reflection coefficients
• CPC Class Codes: G10H 5/005 Voice controlled instruments
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers

Abstract

An electronic apparatus in which the operator inputs both the textual material and a sequence of pitches which upon synthesization simulates singing qualities. The operator inputs a textual material, typically through a keyboard arrangement, and also a sequence of pitches as the tune of the desired song. The text is broken into syllable components which are matched to each note of the tune. The syllables are used to generate control parameters for the synthesizer from their allophonic components. The invention allows the entry of text and a pitch sequence so as to simulate electronically the singing of a tune.

Claims

1. An electronic sound synthesis apparatus for simulating the vocal singing of a song, said apparatus comprising:

operator input means for selectively introducing a sequence of textual information representative of human sounds and for establishing a sequence of pitch information;

memory means storing digital data therein representative of at least portions of words in a human language from which the lyrics of a song may be synthesized, said memory means further including a storage portion in which digital data representative of a plurality of pitches is stored from which the tune of a song may be synthesized;

control means operably coupled to said operator input means and said memory means for forming a sequence of synthesis control data in response to the accessing of digital data representative of at least portions of words and the accessing of digital data representative of a selected sequence of pitches defining a tune, said control means including correlation means for combining the sequences of digital data from said memory means respectively representative of the lyrics and the tune of the song in a manner producing said sequence of synthesis control data;

synthesizer means operably associated with said memory means and said control means for receiving said sequence of synthesis control data as produced by said correlation means and providing an analog output signal representative of the song as produced by the lyrics and tune; and

audio means coupled to said synthesizer means for converting said analog output signal into an audible song comprising the lyrics and the tune in a correlated relationship.

2. An electronic sound synthesis apparatus as set forth in claim 1, wherein said operator input means is further effective for establishing duration information corresponding to each of the pitches included in the sequence of pitch information;

the storage portion of said memory means in which digital data representative of a plurality of pitches is stored further storing digital data representative of a plurality of different durations to which any one of the plurality of pitches may correspond from which the tune of the song may be synthesized; and

said sequence of synthesis control data being formed by said control means in further response to the accessing of digital data representative of selected durations corresponding respectively to the individual pitches included in the selected sequence of pitches defining a tune such that the duration information corresponding to each of the pitches included in the sequence of pitches is included in said sequence of synthesis control data produced by said correlation means.

3. An electronic sound synthesis apparatus as set forth in claim 1, wherein said operator input means comprises keyboard means for selectively introducing at least textual information.

4. An electronic sound synthesis apparatus as set forth in claim 3, wherein said keyboard means includes a first keyboard including a plurality of keys respectively representative of letters of the alphabet and adapted to be selectively actuated by an operator in the introduction of the sequence of textual information, and a second keyboard including a plurality of keys respectively representative of individual pitch-defining musical notes and adapted to be selectively actuated by the operator in establishing the sequence of pitch information.

5. An electronic sound synthesis apparatus as set forth in claim 4, wherein said second keyboard is arranged in the form of a piano-like keyboard.

6. An electronic sound synthesis apparatus as set forth in claim 1, wherein said storage portion included in said memory means in which digital data representative of a plurality of pitches is stored comprises a tune library in which a plurality of predetermined tunes as defined by respective selective arrangements of pluralities of pitch sequences are stored;

said operator input means including a keyboard having a plurality of keys for selective actuation by an operator so as to identify respective predetermined tunes as stored in said tune library of said memory means; and

said control means accessing digital data representative of a selected sequence of pitches defining said tune from said tune library as identified by the selective key actuation of said keyboard by the operator such that said correlation means of said control means is effective for combining the sequence of digital data from said memory means representative of the lyrics with the digital data from said tune library of said memory means representative of the selected tune in producing said sequence of synthesis control data.

7. An electronic sound synthesis apparatus as set forth in claim 1, further including

means operably coupled to said operator input means for receiving said sequence of textual information therefrom and establishing a sequence of syllables corresponding to said sequence of textual information;

said correlation means of said control means matching each syllable from said sequence of syllables with a corresponding pitch from said sequence of pitches in combining the sequences of digital data from said memory means respectively representative of the lyrics and the tune of the song for producing said sequence of synthesis control data.

8. An electronic sound synthesis apparatus as set forth in claim 7, wherein said means for establishing said sequence of syllables from said sequence of textual information includes means for forming a sequence of allophones as digital signals identifying the respective allophone subset variants of each of the recognized phonemes in a given spoken language as modified by the speech environment in which the particular phoneme occurs from said sequence of textual information, and

means for grouping the allophones in the sequence of allophones into said sequence of syllables.

9. An electronic sound synthesis apparatus as set forth in claim 2, further including

allophone rule means having a plurality of allophonic signals corresponding to digital characters representative of textual information, wherein the allophonic signals are determinative of the respective allophone subset variants of each of the recognized phonemes in a given spoken language as modified by the speech environment in which the particular phoneme occurs;

allophone rules processor means having an input for receiving the sequence of textual information from said operator input means and operably coupled to said allophone rule means for searching the allophone rule means to provide an allophonic signal output corresponding to the digital characters representative of the sequence of textual information from the allophonic signals of said allophone rule means;

syllable extraction means coupled to said allophone rules processor means for receiving said allophonic signal output therefrom and grouping the allophones into a sequence of syllables corresponding to said allophonic signal output; and

said control means combining each syllable of said sequence of syllables with digital data corresponding to an associated pitch and duration in forming said sequence of synthesis control data.

10. An electronic sound synthesis apparatus as set forth in claim 9, further including

allophone library means in which digital signals representative of allophone-defining speech parameters identifying the respective allophone subset variants of each of the recognized phonemes in a given spoken language as modified by the speech environment in which the particular phoneme occurs are stored, said allophone library means being operably coupled to said control means and providing digital signals representative of the particular allophone-defining speech parameters corresponding to the sequence of syllables;

and

the digital data corresponding to respective pitches and their associated durations being provided in the form of digital signals designating pitch and duration parameters and being combined by said control means with said digital signals representative of the particular allophone-defining speech parameters corresponding to the sequence of syllables in forming said sequence of synthesis control data.

11. An electronic sound synthesis apparatus as set forth in claim 10, wherein said digital signals representative of the particular allophone-defining speech parameters corresponding to the sequence of syllables and said digital signals designating pitch and duration parameters are linear predictive coding parameters such that said sequence of synthesis control data is in the form of linear predictive coding digital signal parameters;

and

said synthesizer means being a linear predictive coding synthesizer.

12. An electronic sound synthesis apparatus for simulating the vocal singing of a song, said apparatus comprising:

operator input means for selectively introducing a sequence of textual information representative of human sounds and for establishing a sequence of pitch information;

memory means storing digital data therein representative of at least portions of words in a human language from which the lyrics of a song may be synthesized;

pitch determination means operably associated with said operator input means and responsive to the establishment of the sequence of pitch information for providing digital data representative of the sequence of pitches from which the tune of a song may be synthesized;

control means operably coupled to said operator input means, said memory means and said pitch determination means for forming a sequence of synthesis control data in response to the accessing of digital data representative of at least portions of words and the accessing of digital data representative of the sequence of pitches defining a tune, said control means including correlation means for combining the sequences of digital data from said memory means and said pitch determination means respectively representative of the lyrics and the tune of the song in a manner producing said sequence of synthesis control data;

synthesizer means operably associated with said memory means and said control means for receiving said sequence of synthesis control data as produced by said correlation means and providing an analog output signal representative of the song as produced by the lyrics and tune; and

audio means coupled to said synthesizer means for converting said analog output signal into an audible song comprising the lyrics and the tune in a correlated relationship.

13. An electronic sound synthesis apparatus as set forth in claim 12, wherein said operator input means includes keyboard means for selectively introducing at least textual information.

14. An electronic sound synthesis apparatus as set forth in claim 12, wherein said operator input means is further effective for establishing duration information corresponding to each of the pitches included in the sequence of pitch information;

said pitch determination means being further responsive to the establishment of the respective durations corresponding to individual pitches included in the sequence of pitch information for providing digital data representative of the respective durations for each of the pitches included in the sequence of pitches from which the tune of the song may be synthesized; and

said digital data representative of the duration information for each of the pitches included in the sequence of pitches being incorporated into said sequence of synthesis control data as produced by said correlation means of said control means.

15. An electronic sound synthesis apparatus as set forth in claim 14, wherein said operator input means at least includes a microphone for receiving an operator input as an operator-generated sequence of tones, said microphone generating an electrical analog output signal in response to said operator-generated sequence of tones;

and

said pitch determination means comprising pitch extractor means operably associated with said microphone for acting upon said electrical analog output signal therefrom to identify the sequence of pitches and durations associated therewith corresponding to the operator-generated sequence of tones and providing digital data representative of the sequence of pitches and associated durations from which the tune of the song may be synthesized.

16. An electronic sound synthesis apparatus as set forth in claim 15, wherein said operator input means further includes a keyboard having a plurality of keys respectively representative of letters of the alphabet and adapted to be selectively actuated by an operator in the introduction of the sequence of textual information.

17. An electronic sound synthesis apparatus as set forth in claim 12, further including

means operably coupled to said operator input means for receiving said sequence of textual information therefrom and establishing a sequence of syllables corresponding to said sequence of textual information;

said correlation means of said control means matching each syllable from said sequence of syllables with a corresponding pitch from said sequence of pitches in combining the sequences of digital data from said memory means and said pitch determination means respectively representative of the lyrics and the tune of the song for producing said sequence of synthesis control data.

18. An electronic sound synthesis apparatus as set forth in claim 17, wherein said means for establishing said sequence of syllables from said sequence of textual information includes means for forming a sequence of allophones as digital signals identifying the respective allophone subset variants of each of the recognized phonemes in a given spoken language as modified by the speech environment in which the particular phoneme occurs from said sequence of textual information, and

means for grouping the allophones in the sequence of allophones into said sequence of syllables.

19. An electronic sound synthesis apparatus as set forth in claim 14, further including

allophone rule means having a plurality of allophonic signals corresponding to digital characters representative of textual information, wherein the allophonic signals are determinative of the respective allophone subset variants of each of the recognized phonemes in a given spoken language as modified by the speech environment in which the particular phoneme occurs;

allophone rules processor means having an input for receiving the sequence of textual information from said operator input means and operably coupled to said allophone rule means for searching the allophone rule means to provide an allophonic signal output corresponding to the digital characters representative of the sequence of textual information from the allophonic signals of said allophone rule means;

syllable extraction means coupled to said allophone rules processor means for receiving said allophonic signal output therefrom and grouping the allophones into a sequence of syllables corresponding to said allophonic signal output; and

said control means combining each syllable of said sequence of syllables with digital data corresponding to an associated pitch and duration in forming said sequence of synthesis control data.

20. An electronic sound synthesis apparatus as set forth in claim 19, further including

allophone library means in which digital signals representative of allophone-defining speech parameters identifying the respective allophone subset variants of each of the recognized phonemes in a given spoken language as modified by the speech environment in which the particular phoneme occurs are stored, said allophone library means being operably coupled to said control means and providing digital signals representative of the particular allophone-defining speech parameters corresponding to the sequence of syllables;

and

the digital data corresponding to respective pitches and their associated durations being provided in the form of digital signals designating pitch and duration parameters and being combined by said control means with said digital signals representative of the particular allophone-defining speech parameters corresponding to the sequence of syllables in forming said sequence of synthesis control data.

21. An electronic sound synthesis apparatus as set forth in claim 20, wherein said digital signals representative of the particular allophone-defining speech parameters corresponding to the sequence of syllables and said digital signals designating pitch and duration parameters are linear predictive coding parameters such that said sequence of synthesis control data is in the form of linear predictive coding digital signal parameters;

and

said synthesizer means being a linear predictive coding synthesizer.