METHOD AND APPARATUS FOR TRAINING A PROSODY STATISTIC MODEL AND PROSODY PARSING, METHOD AND SYSTEM FOR TEXT TO SPEECH SYNTHESIS

• US 20070129938A1
• Filed: 10/06/2006
• Published: 06/07/2007
• Est. Priority Date: 10/09/2005
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation), Toshiba Digital Solutions Corporation (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Wang, Haifeng, Li, Guohua
• Granted Patent: US 8,024,174 B2
• Time in Patent Office: Days
• US Class Current: 704/10
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models

Abstract

The present invention provides a method and apparatus for training a prosody statistic model and prosody parsing, a method and system for text to speech synthesis. Said method for training a prosody statistic model with a raw corpus that includes a plurality of sentences with punctuation, comprising: transforming said plurality of sentences in said raw corpus into a plurality of token sequences respectively; counting a frequency for each adjacent token pair occurring in said plurality of token sequences and frequencies of punctuation that represents a pause occurring at associated positions of said each token pair; calculating pause probabilities at said associated positions of said each token pair; and constructing said prosody statistic model based on said token pairs and said pause probabilities at associated positions thereof. With the present invention a prosody statistic model can be trained from a raw corpus without manually prosody parsing tags. And the prosody statistic model can be used in the prosody parsing and further voice synthesis.

Claims

1. A method for training a prosody statistic model with a raw corpus that includes a plurality of sentences with punctuation, comprising:

transforming said plurality of sentences in said raw corpus into a plurality of token sequences respectively;

counting a frequency for each adjacent token pair occurring in said plurality of token sequences and frequencies of punctuation that represents a pause occurring at associated positions of said each token pair;

calculating pause probabilities at said associated positions of said each token pair; and

constructing said prosody statistic model based on said token pairs and said pause probabilities at associated positions thereof.

2. The method for training a prosody statistic model according to claim 1, wherein said associated positions of said each token pair include:

before, after and amid said token pair.

3. The method for training a prosody statistic model according to claim 2, wherein said step of calculating pause probabilities at said associated positions of said each token pair comprises:

calculating a pause probability before said token pair based on the frequency of said token pair and the frequency of punctuation representing a pause occurring before said token pair;

calculating a pause probability after said token pair based on the frequency of said token pair and the frequency of punctuation representing a pause occurring after said token pair;

calculating a pause probability amid said token pair based on the frequency of said token pair and the frequency of punctuation representing a pause occurring amid said token pair.

4. The method for training a prosody statistic model according to claim 3, wherein said step of calculating a pause probability before said token pair comprises:

calculating the pause probability before said token pair Prob_ptt with

${\mathrm{Prob}}_{\mathrm{ptt}}=\frac{{\mathrm{Count}}_{\mathrm{ptt}}}{{\mathrm{Count}}_{\mathrm{tt}}}$

wherein Count_tt represents the frequency of said token pair, Count_ptt represents the frequency of punctuation representing a pause occurring before said token pair.

5. The method for training a prosody statistic model according to claim 3, wherein said step of calculating a pause probability after said token pair comprises:

calculating the pause probability after said token pair Prob_ttp with

${\mathrm{Prob}}_{\mathrm{ttp}}=\frac{{\mathrm{Count}}_{\mathrm{ttp}}}{{\mathrm{Count}}_{\mathrm{tt}}}$

wherein Count_tt represents the frequency of said token pair, Count_ttp represents the frequency of punctuation representing a pause occurring after said token pair.

6. The method for training a prosody statistic model according to claim 3, wherein said step of calculating a pause probability amid said token pair comprises:

calculating the pause probability amid said token pair Prob_tpt with

${\mathrm{Prob}}_{\mathrm{tpt}}=\frac{{\mathrm{Count}}_{\mathrm{tpt}}}{{\mathrm{Count}}_{\mathrm{tt}}}$

wherein Count_tt represents the frequency of said token pair, Count_tpt represents the frequency of punctuation representing a pause occurring amid said token pair.

7. The method for training a prosody statistic model according to claim 1, wherein said step of calculating pause probabilities at said associated positions of said each token pair further comprises:

performing smoothing process on said calculated pause probabilities.

8. The method for training a prosody statistic model according to claim 1, wherein said step of calculating pause probabilities at said associated positions of said each token pair further comprises:

performing cutoff process on said calculated pause probabilities.

9. The method for training a prosody statistic model according to claim 1, wherein said token is in a form of any one selected from character, word, word plus part of speech and phrase.

10. The method for training a prosody statistic model according to claim 1, wherein said prosody statistic model includes a token translation table that stores each of the tokens involved in the prosody statistic model and an token ID corresponding to said token.

11. A method of prosody parsing, comprising:

training a prosody statistic model by using the method of any one of claims 1˜

10;

inputting a text for prosody parsing, which includes at least one sentence;

transforming the sentence into a token sequence;

calculating a pause weight for each pause position in said token sequence based on said prosody statistic model; and

selecting at least one pause positions to insert a pause tag according to said calculated pause weight for each pause position.

12. The method of prosody parsing according to claim 11, wherein said token is in a form of any one selected from character, word, word plus part of speech and phrase.

13. The method of prosody parsing according to claim 11, wherein said prosody statistic model includes a plurality of token pairs and pause probabilities at associated positions of each of said plurality of token pairs, said step of calculating a pause weight for each pause position in said token sequence comprises:

calculating a pause weight for each pause position in said token sequence according to the pause probabilities at associated positions of corresponding token pairs in the prosody statistic model.

14. The method of prosody parsing according to claim 13, wherein the associated positions of each token pair includes before, after and amid said token pair, said step of calculating a pause weight for each pause position in said token sequence comprises:

calculating the pause weight for each pause position with weight=−

log(Prob_ttp×

Prob_tpt×

Prob_ptt)

wherein Prob_ttp represents the pause probability after the token pair that is located before said pause position, Prob_ptt represents the pause probability before the token pair that is located after said pause position, and Prob_tpt represents the pause probability amid the token pair that is located across said pause position.

15. The method of prosody parsing according to claim 13, wherein a pause position is not set after the first token of said token sequence or before the last token of said token sequence.

16. The method of prosody parsing according to claim 11, wherein said step of selecting at least one pause positions to insert a pause tag is performed on condition that:

the pause weight of a pause position is higher than a predetermined threshold.

17. The method of prosody parsing according to claim 11, wherein said step of selecting at least one pause positions to insert a pause tag is performed on condition that:

the pause weight of a pause position is higher than a predetermined threshold; and

the number of tokens between two selected pause positions is larger than another predetermined threshold.

18. A method of text to speech synthesis, comprising:

Inserting at least one pause tag into the text by using the method of any one of claims 11˜

17; and

Performing voice synthesis according to the text inserted with said at least one pause tag.

19. An apparatus for training a prosody statistic model with a raw corpus that includes a plurality of sentences with punctuation, comprising:

a tokenization unit configured to transform said plurality of sentences in said raw corpus into a plurality of token sequences respectively;

a counter configured to count a frequency for each adjacent token pair occurring in said plurality of token sequences and frequencies of punctuation that represents a pause occurring at associated positions of said each token pair;

a pause probability calculator configured to calculate pause probabilities at said associated positions of said each token pair; and

a prosody statistic model constructor configured to construct said prosody statistic model based on said token pairs and said pause probabilities at associated positions thereof.

20. The apparatus for training a prosody statistic model according to claim 19, wherein said associated positions of said each token pair include:

before, after and amid said token pair.

21. The apparatus for training a prosody statistic model according to claim 20, wherein said pause probability calculator is configured to calculate a pause probability before a token pair based on the frequency of said token pair and the frequency of punctuation representing a pause occurring before said token pair, to calculate a pause probability after said token pair based on the frequency of said token pair and the frequency of punctuation representing a pause occurring after said token pair, and to calculate a pause probability amid said token pair based on the frequency of said token pair and the frequency of punctuation representing a pause occurring amid said token pair.

22. The apparatus for training a prosody statistic model according to claim 21, wherein said pause probability calculator is configured to calculate the pause probability before said token pair Prob_ptt with

${\mathrm{Prob}}_{\mathrm{ptt}}=\frac{{\mathrm{Count}}_{\mathrm{ptt}}}{{\mathrm{Count}}_{\mathrm{tt}}}$

wherein Count_tt represents the frequency of said token pair, Count_ptt represents the frequency of punctuation representing a pause occurring before said token pair.

23. The apparatus for training a prosody statistic model according to claim 21, wherein said pause probability calculator is configured to calculate the pause probability after said token pair Prob_ttp with

${\mathrm{Prob}}_{\mathrm{ttp}}=\frac{{\mathrm{Count}}_{\mathrm{ttp}}}{{\mathrm{Count}}_{\mathrm{tt}}}$

wherein Count_tt represents the frequency of said token pair, Count_ttp represents the frequency of punctuation representing a pause occurring after said token pair.

24. The apparatus for training a prosody statistic model according to claim 21, wherein said pause probability calculator is configured to calculate the pause probability amid said token pair Prob_tpt with

${\mathrm{Prob}}_{\mathrm{tpt}}=\frac{{\mathrm{Count}}_{\mathrm{tpt}}}{{\mathrm{Count}}_{\mathrm{tt}}}$

wherein Count_tt represents the frequency of said token pair, Count_tpt represents the frequency of punctuation representing a pause occurring amid said token pair.

25. The apparatus for training a prosody statistic model according to claim 19, further comprising:

a smoothing unit configured to perform smoothing process on said calculated pause probabilities.

26. The apparatus for training a prosody statistic model according to claim 19, further comprising:

a cutoff unit configured to perform cutoff process on said calculated pause probabilities.

27. The apparatus for training a prosody statistic model according to claim 19, wherein said token is in a form of any one selected from character, word, word plus part of speech and phrase.

28. An apparatus for prosody parsing, comprising:

a text input unit configured to input a text for prosody parsing, which includes at least one sentence;

a tokenization unit configured to transform the sentence into a token sequence;

a pause weight calculator configured to calculate a pause weight for each pause position in said token sequence based on a prosody statistic model that is trained from a raw corpus and includes a plurality of token pairs and pause probabilities at associated positions of each said plurality of token pairs; and

a pause tag setting unit configured to select at least one pause positions to insert a pause tag according to said calculated pause weight for each pause position.

29. The apparatus for prosody parsing according to claim 28, wherein said token is in a form of any one selected from character, word, word plus part of speech and phrase.

30. The apparatus for prosody parsing according to claim 28, wherein said pause weight calculator is configured to calculate a pause weight for each pause position in said token sequence according to the pause probabilities at associated positions of corresponding token pairs in the prosody statistic model.

31. The apparatus for prosody parsing according to claim 30, wherein the associated positions of each token pair includes before, after and amid said token pair, said pause weight calculator is configured to calculate the pause weight for each pause position with weight=−

log(Prob_ttp×

Prob_tpt×

Prob_ptt)

wherein Prob_ttp represents the pause probability after the token pair that is located before said pause position, Prob_ptt represents the pause probability before the token pair that is located after said pause position, and Prob_tpt represents the pause probability amid the token pair that is located across said pause position.

32. The apparatus for prosody parsing according to claim 30, wherein said pause tag setting unit is configured not to set a pause position after the first token of said token sequence or before the last token of said token sequence.

33. The apparatus for prosody parsing according to claim 28, wherein said pause tag setting unit is configured to selecting at least one pause positions to insert a pause tag on condition that:

the pause weight of a pause position is higher than a predetermined threshold.

34. The apparatus for prosody parsing according to claim 28, wherein said pause tag setting unit is configured to selecting at least one pause positions to insert a pause tag on condition that:

the pause weight of a pause position is higher than a predetermined threshold; and

the number of tokens between two selected pause positions is larger than another predetermined threshold.

35. A system for text to speech synthesis, comprising:

an apparatus for prosody parsing according to any one of claims 28˜

34; and

a voice synthesizer configured to perform voice synthesis according to the text inserted with said at least one pause tag.

36. The system for text to speech synthesis according to claim 35, further comprising:

an apparatus for training a prosody statistic model according to any one of claims 28˜

34.