Speech synthesis system, speech synthesis program product, and speech synthesis method

• US 8,370,149 B2
• Filed: 08/15/2008
• Issued: 02/05/2013
• Est. Priority Date: 09/07/2007
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Tachibana, Ryuki, Nishimura, Masafumi
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Sirjani, Fariba
• Application Number:
  US12/192,510
• Publication Number:
  US 20090070115A1
• Time in Patent Office: 1,635 Days
• Field of Search: 704/258, 704/260
• US Class Current: 704/260
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/07 Concatenation rules
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation

Abstract

Waveform concatenation speech synthesis with high sound quality. Prosody with both high accuracy and high sound quality is achieved by performing a two-path search including a speech segment search and a prosody modification value search. An accurate accent is secured by evaluating the consistency of the prosody by using a statistical model of prosody variations (the slope of fundamental frequency) for both of two paths of the speech segment selection and the modification value search. In the prosody modification value search, a prosody modification value sequence that minimizes a modified prosody cost is searched for. This allows a search for a modification value sequence that can increase the likelihood of absolute values or variations of the prosody to the statistical model as high as possible with minimum modification values.

Claims

1. A speech synthesis system for synthesizing speech from text, the system comprising:

a speech segment database configured to store a plurality of speech segments;

means for determining a first speech segment sequence corresponding to an input text, by selecting speech segments from the speech segment database-according to a first cost calculated based at least in part on a statistical model of prosody variations;

means for determining prosody modification values for the first speech segment sequence, after the first speech segment sequence is selected, by using a second cost calculated based at least in part on the statistical model of prosody variations, wherein the first cost is different from the second cost; and

means for applying the determined prosody modification values to the first speech segment sequence to produce a second speech segment sequence whose prosodic characteristics are different from prosodic characteristics of the first speech segment sequence,

wherein the second cost includes at least a prosody modification cost, the system further comprising means for increasing the prosody modification cost of continuous speech segments having a slope likelihood greater than a given value before determining the prosody modification values in response to detection of the continuous speech segments in the first speech segment sequence.

2. The speech synthesis system according to claim 1, wherein the first cost for determining the first speech segment sequence includes a spectrum continuity cost, a duration error cost, a volume error cost, an absolute frequency likelihood cost, a frequency slope likelihood cost, and a frequency linear approximation error cost.

3. The speech synthesis system according to claim 1, wherein the second cost for determining the prosody modification values includes an absolute frequency likelihood cost, a frequency slope likelihood cost, a frequency linear approximation error cost, and a prosody modification cost.

4. The speech synthesis system according to claim 1, wherein the statistical model uses a decision tree and Gaussian mixture models.

5. At least one computer-readable storage device encoded with a speech synthesis program which causes a system for synthesizing speech from text to perform:

determining a first speech segment sequence corresponding to an input text, by selecting speech segments from the speech segment database according to a first cost calculated based at least in part on a statistical model of prosody variations;

determining prosody modification values for the first speech segment sequence, after the first speech segment sequence is selected, by using a second cost calculated based at least in part on the statistical model of prosody variations, wherein the first cost is different from the second cost; and

applying the determined prosody modification values to the first speech segment sequence to produce a second speech segment sequence whose prosodic characteristics are different from prosodic characteristics of the first speech segment sequence,

wherein the second cost includes at least a prosody modification cost, the program further causing the system to perform the step of increasing the prosody modification cost of continuous speech segments having a slope likelihood greater than a given value in the first speech segment sequence before determining the prosody modification values in response to detection of the continuous speech segments.

6. The at least one computer readable storage device of claim 5, wherein the first cost for determining the first speech segment sequence includes a spectrum continuity cost, a duration error cost, a volume error cost, an absolute frequency likelihood cost, a frequency slope likelihood cost, and a frequency linear approximation error cost.

7. The at least one computer readable storage device of claim 5, wherein the second cost for determining the prosody modification values includes an absolute frequency likelihood cost, the frequency slope likelihood cost, a frequency linear approximation error cost, and a prosody modification cost.

8. The at least one computer readable storage device of claim 5, wherein the statistical model uses a decision tree and a Gaussian mixture model.

9. A speech synthesis method for synthesizing speech from text by computer processing, the method comprising:

determining a first speech segment sequence corresponding to an input text by selecting speech segments from a speech segment database-according to a first cost calculated based at least in part on statistical model of prosody variations;

determining prosody modification values for the first speech segment sequence, after the first speech segment sequence is selected, by using a second cost calculated based at least in part on the statistical model of prosody variations, wherein the first cost is different from the second cost; and

applying the determined prosody modification values to the first speech segment sequence to produce a second speech segment sequence whose prosodic characteristics are different from prosodic characteristics of the first speech segment sequence,

wherein the second cost includes at least a prosody modification cost, the method further comprising increasing the prosody modification cost of continuous speech segments having a slope likelihood greater than a given value in the first speech segment sequence before determining the prosody modification values in response to detection of the continuous speech segments.

10. The speech synthesis method according to claim 9, wherein the first cost for determining the first speech segment sequence includes a spectrum continuity cost, a duration error cost, a volume error cost, an absolute frequency likelihood cost, a frequency slope likelihood cost, and a frequency linear approximation error cost.

11. The speech synthesis method according to claim 9, wherein the second cost for determining the prosody modification values includes an absolute frequency likelihood cost, a frequency slope likelihood cost, a frequency linear approximation error cost, and a prosody modification cost.

12. A speech synthesis method according to claim 9, wherein the statistical model uses a decision tree and a Gaussian mixture model.

13. A speech synthesis system for synthesizing speech from text, the system comprising:

at least one processor configured to;

select a first speech segment sequence corresponding to an input text from a speech segment database by using a first cost value calculated based at least in part on a statistical model of prosody variations;

determine prosody modification values for the first speech segment sequence, after the first speech segment sequence is selected, by using a second cost value calculated based at least in part on the statistical model of prosody variations, wherein the first cost value is different from the second cost value; and

apply the determined prosody modification values to the first speech segment sequence to produce a second speech segment sequence whose prosodic characteristics are different from prosodic characteristics of the first speech segment sequence,

wherein the second cost includes at least a prosody modification cost, and the at least one processor is further configured to increase the prosody modification cost of continuous speech segments having a slope likelihood greater than a given value in the first speech segment sequence before determining the prosody modification values in response to detection of the continuous speech segments.

14. The system of claim 13, wherein the first cost for determining the first speech segment sequence includes a spectrum continuity cost, a duration error cost, a volume error cost, an absolute frequency likelihood cost, a frequency slope likelihood cost, and a frequency linear approximation error cost.

15. The system of claim 13, wherein the second cost for determining the prosody modification values includes an absolute frequency likelihood cost, a frequency slope likelihood cost, a frequency linear approximation error cost, and a prosody modification cost.