Prosody generation using syllable-centered polynomial representation of pitch contours

• US 8,886,539 B2
• Filed: 03/17/2014
• Issued: 11/11/2014
• Est. Priority Date: 12/03/2012
• Status: Active Grant

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Chengjun Julian Chen
• Inventors: Chen, Chengjun Julian
• Primary Examiner(s): ALBERTALLI, BRIAN LOUIS
• Application Number:
  US14/216,611
• Publication Number:
  US 20140195242A1
• Time in Patent Office: 239 Days
• Field of Search: None
• US Class Current: 704/268
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 13/0335 Pitch control
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation

Abstract

The present invention discloses a parametrical representation of prosody based on polynomial expansion coefficients of the pitch contour near the center of each syllable. The said syllable pitch expansion coefficients are generated from a recorded speech database, read from a number of sentences by a reference speaker. By correlating the stress level and context information of each syllable in the text with the polynomial expansion coefficients of the corresponding spoken syllable, a correlation database is formed. To generate prosody for an input text, stress level and context information of each syllable in the text is identified. The prosody is generated by using the said correlation database to find the best set of pitch parameters for each syllable. By adding to global pitch contours and using interpolation formulas, complete pitch contour for the input text is generated. Duration and intensity profile are generated using a similar procedure.

Claims

1. A method for building databases for prosody generation in speech synthesis using one or more processors comprising:

A) compile a text corpus of sentences containing all the prosody phenomena of interest;

B) for each phrase in each said sentence, identify the phrase type;

C) segment each sentence into syllables, identify the property and context information of each said syllable;

D) read the sentences by a reference speaker to make a recording of voice signals;

E) segment the voice signals of each sentence into syllables, each said syllable is aligned with a syllable in the text;

F) identify the voiced section in each syllable of the voice recording;

G) calculate pitch values in the said voiced section;

H) generate a polynomial expansion of the pitch contour of each said voiced section in each syllable by least-squares fitting, comprising the use of Gegenbauer polynomials, which at least have a constant term representing the average pitch of the said syllable;

I) for all phrases of a given type, generate a polynomial expansion of the values of said average pitch of all syllables in the said phrases using least-squares fitting, to generate an average global pitch contour of the given phrase type;

J) form a set of syllable pitch parameters for each said syllable by subtracting the value of the global pitch profile at that point from the value of the average pitch of the said syllable together with the rest of polynomial expansion coefficients for the said syllable;

K) correlate the syllable pitch parameters with the property and context information of the said syllable from an analysis of the text to form a database of syllable pitch parameters;

L) correlate the intensity and duration parameters of a syllable to the property and context information of the said syllable from an analysis of the text to form a database of intensity and duration.

2. The pitch values in claim 1 are expressed as a linear function of the logarithm of the pitch period, comprising the use of MIDI unit.

3. The property and context information of the said syllable in claim 1 comprises the stress level of the said syllable in a word, the emphasis level, part of speech, grammatical identity of the said word in the phrase, and the similar information of neighboring syllables and words.

4. For tone languages, the property and context information in claim 1 comprises the tone and stress level of the said syllable in a word, the emphasis level, part of speech, grammatical identity of the said word in the phrase, and the similar information of neighboring syllables and words.

5. The type of phrase in claim 1 comprises declarative, interrogative, exclamatory, or intermediate phrase.

6. A method for generating prosody in speech synthesis from an input sentence using the said databases in claim 1 comprising:

A) for each phrase in the said input sentence, identify the phrase type;

B) segment each sentence into syllables, identify the property and context information of each said syllable;

C) based on the said phrase type, retrieving a global phrase pitch profile from the global pitch profiles database for each said phrase;

D) finding the syllable pitch parameters for each said syllable using the property and context information of each said syllable and the database of syllable pitch parameters;

E) for each said syllable, adding the pitch value in the global pitch contour at the time of the said syllable to the constant term of the said syllable pitch parameters;

F) calculating pitch values for the entire sentence using polynomial interpolation;

G) finding the intensity and duration parameters for each said syllable using the property and context information of each said syllable and the database of intensity and duration parameters;

H) output the said pitch contour and said intensity and duration parameters for the entire sentence as prosody parameters for speech synthesis.

7. The pitch values in claim 6 are expressed as a linear function of the logarithm of the pitch period, comprising the use of MIDI unit.

8. The property and context information in claim 6 comprises the stress level of the said syllable in a word, the emphasis level, part of speech, grammatical identity of the said word in the phrase, and the similar information of neighboring syllables and words.

9. For tone languages, the property and context information in claim 6 comprises the tone and stress level of the said syllable in a word, the emphasis level, part of speech, grammatical identity of the said word in the phrase, and the similar information of neighboring syllables and words.

10. The type of phrase in claim 6 comprises declarative, interrogative, exclamatory, or intermediate phrase.

11. The recording of voice signals in claim 1 includes simultaneous electroglottograph signals, the voiced sections are identified by the existence of the electroglottograph signals, and the pitch values are calculated from the electroglottograph signals.