Duration prediction modeling in speech synthesis

• US 7,840,408 B2
• Filed: 10/19/2006
• Issued: 11/23/2010
• Est. Priority Date: 10/20/2005
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Hao, Jie, Yi, Lifu
• Primary Examiner(s): Lerner; Martin
• Application Number:
  US11/551,025
• Publication Number:
  US 20070129948A1
• Time in Patent Office: 1,496 Days
• Field of Search: 704/258, 704/260, 704/266, 704/267
• US Class Current: 704/258
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation
• CPC Class Codes: G10L 15/148 Duration modelling in HMMs, e.g. semi HMM, segmental models or transition probabilities

Abstract

The present invention provides a method and apparatus for training a duration prediction model, method and apparatus for duration prediction, method and apparatus for speech synthesis. Said method for training a duration prediction model, comprising: generating an initial duration prediction model with a plurality of attributes related to duration prediction and at least part of possible attribute combinations of said plurality of attributes, in which each of said plurality of attributes and said attribute combinations is included as an item; calculating importance of each said item in said duration prediction model; deleting the item having the lowest importance calculated; re-generating a duration prediction model with the remaining items; determining whether said re-generated duration prediction model is an optimal model; and repeating said step of calculating importance and the following steps, if said duration prediction model is determined as not optimal model.

Claims

1. A method for training a duration prediction model, comprising:

generating an initial duration prediction model with a plurality of attributes related to duration prediction and at least part of possible attribute combinations of said plurality of attributes, in which each of said plurality of attributes and said attribute combinations is included as an item;

calculating importance of each item in said duration prediction model with a computer processor;

deleting the item having a lowest importance calculated;

re-generating a duration prediction model with remaining items;

determining whether said re-generated duration prediction model is an optimal model; and

repeating said step of calculating importance and the steps following said step of calculating importance with a newly re-generated duration prediction model, if said duration prediction model is determined as not an optimal model.

2. The method for training a duration prediction model according to claim 1, wherein said plurality of attributes related to duration prediction include:

attributes of language type and speech type.

3. The method for training a duration prediction model according to claim 1, wherein said plurality of attributes related to duration prediction include:

any selected from current phoneme, another phoneme in a same syllable, neighboring phoneme in a previous syllable, neighboring phoneme in a next syllable, tone of a current syllable, tone of the previous syllable, tone of the next syllable, part of speech, distance to a next pause, distance to a previous pause, phoneme position in a lexical word, length of a current, previous and next lexical word, number of syllables in the lexical word, syllable position in a sentence, and number of lexical words in the sentence.

4. The method for training a duration prediction model according to claim 1, wherein said duration prediction model is a Generalized Linear Model (GLM).

5. The method for training a duration prediction model according to claim 1, wherein said at least part of possible attribute combinations of said plurality of attributes include all 2nd order attribute combinations of said plurality of attributes related to duration prediction.

6. The method for training a duration prediction model according to claim 1, wherein said step of calculating importance of each item in said duration prediction model includes calculating the importance of each said item with F-test.

7. The method for training a duration prediction model according to claim 1, wherein said step of determining whether said re-generated duration prediction model is an optimal model includes determining whether said re-generated duration prediction model is an optimal model based on Bayes Information Criterion (BIC).

8. The method for training a duration prediction model according to claim 7, wherein said step of determining whether said re-generated duration prediction model is an optimal model includes

calculating based on an equation

BIC=N log(SSE/N)+p log N,

wherein SSE represents sum square of prediction errors and N represents a number of training sample;

and

determining said re-generated duration prediction model as an optimal model, when the BIC is a minimum.

9. The method for training a duration prediction model according to any one of the preceding claims, wherein said plurality of attributes related to duration prediction further include speaking rate.

10. The method for training a duration prediction model according to claim 1, further comprising:

obtaining corresponding values of said plurality of attributes related to duration prediction; and

calculating a duration based on said duration prediction model and said corresponding values of said plurality of attributes related to duration prediction.

11. The method for training a duration prediction model according to claim 10, wherein said plurality of attributes related to duration prediction include speaking rate.

12. The method for training a duration prediction model according to any one of claims 10-11, further comprising:

performing speech synthesis based on the durations predicted.

13. An apparatus for training a duration prediction model, comprising:

an initial model generator configured to generate an initial duration prediction model with a plurality of attributes related to duration prediction and at least part of possible attribute combinations of said plurality of attributes, in which each of said plurality of attributes and said attribute combinations is included as an item;

an importance calculator configured to calculate importance of each said item in said duration prediction model;

an item deleting unit configured to delete the item having a lowest importance calculated;

a model re-generator configured to re-generate a duration prediction model with remaining items after a deletion of said item deleting unit; and

an optimization determining unit configured to determine whether said duration prediction model re-generated by said model re-generator is an optimal model.

14. The apparatus for training a duration prediction model according to claim 13, wherein said plurality of attributes related to duration prediction include:

attributes of language type and speech type.

15. The apparatus for training a duration prediction model according to claim 13, wherein said plurality of attributes related to duration prediction include:

any selected from current phoneme, another phoneme in a same syllable, neighboring phoneme in a previous syllable, neighboring phoneme in a next syllable, tone of a current syllable, tone of the previous syllable, tone of the next syllable, part of speech, distance to a next pause, distance to a previous pause, phoneme position in a lexical word, length of a current, previous and next lexical word, number of syllables in the lexical word, syllable position in a sentence, and number of lexical words in the sentence.

16. The apparatus for training a duration prediction model according to claim 13, wherein said at least part of possible attribute combinations of said plurality of attributes include all 2nd order attribute combinations of said plurality of attributes related to duration prediction.

17. The apparatus for training a duration prediction model according to claim 13, wherein said importance calculator is configured to calculate the importance of each said item with F-test.

18. The apparatus for training a duration prediction model according to claim 13, wherein said optimization determining unit is configured to determine whether said re-generated duration prediction model is an optimal model based on Bayes Information Criterion (BIC).

19. The apparatus for training a duration prediction model according to any one of claims 13-18, wherein said plurality of attributes related to duration prediction further include speaking rate.

20. The apparatus for training a duration prediction model according to claim 13, further comprising:

an attribute obtaining unit configured to obtain corresponding values of said plurality of attributes related to duration prediction; and

a duration calculator configured to calculate a duration based on said duration prediction model and said corresponding values of said plurality of attributes related to duration prediction.

21. The apparatus for training a duration prediction model according to claim 20, wherein said plurality of attributes related to duration prediction include speaking rate.

22. The apparatus for training a duration prediction model according to any one of claims 20-21, wherein the apparatus for training a duration prediction model is included in an apparatus for speech synthesis configured to perform speech synthesis based on the durations predicted.