Fundamental frequency pattern generation apparatus and fundamental frequency pattern generation method

• US 8,478,595 B2
• Filed: 09/05/2008
• Issued: 07/02/2013
• Est. Priority Date: 09/10/2007
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Mizutani, Nobuaki
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Sirjani, Fariba
• Application Number:
  US12/205,626
• Publication Number:
  US 20090070116A1
• Time in Patent Office: 1,761 Days
• Field of Search: 704258-278
• US Class Current: 704/260
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation

Abstract

A fundamental frequency pattern generation apparatus includes a first storage including representative vectors each corresponding to a prosodic control unit and having a section for changing the number of phonemes, a second storage unit including a rule to select a vector corresponding to an input context, a selection unit configured to select a vector from the representative vectors by applying the rule to the context and output the selected vector, a calculation unit configured to calculate an expansion/contraction ratio of the section of the selected vector in a time-axis direction based on a designated value for a specific feature amount related to a length of a fundamental frequency pattern to be generated, the designated value of the feature amount being required of the fundamental frequency pattern to be generated, and an expansion/contraction unit configured to expand/contract the selected vector based on the expansion/contraction ratio to generate the fundamental frequency pattern.

Claims

1. A fundamental frequency pattern generation apparatus comprising:

a computer apparatus comprising a non-transitory computer readable storage medium and a processor;

a first storage unit comprising the non-transitory computer readable storage medium storing a plurality of representative vectors each corresponding to a prosodic control unit and having a first section including a plurality of sample points and a section except for the first section, wherein the first section is a section of the representative vector, which starts with one of an accent nucleus phoneme, an accent nucleus succeeding adjacent phoneme, and an accent nucleus succeeding second phoneme and ends with one of a prosodic control unit end phoneme, a prosodic control unit end preceding adjacent phoneme, and prosodic control unit end preceding second phoneme;

a second storage unit comprising the non-transitory computer readable storage medium storing a rule to select a representative vector corresponding to an input context;

a selection unit configured to select the representative vector corresponding to the input context from the plurality of representative vectors by applying the rule to the input context and output the selected representative vector;

a calculation unit comprising the processor configured to calculate, using a mapping function, an expansion/contraction ratio for a number of phonemes included in the first section of the selected representative vector based on first designated values for a number of phonemes included in a first portion of a fundamental frequency pattern to be generated from the first section of the selected representative vector, the first designated values being required for the fundamental frequency pattern to be generated, such that the number of the phonemes included in the first section of the selected representative vector equals the first designated value, and

an expansion/contraction unit comprising the processor configured to expand/contract the number of the phonemes included in the first section of the selected representative vector based on the expansion/contraction ratio, and then to expand/contract each of the phoneme durations of the phonemes included in all sections of the selected representative vector after the number of the phonemes included in the first section are expanded/contracted, based on second designated values corresponding to phoneme durations of all phonemes included in all portions of the fundamental frequency pattern, the second designated values being required for the fundamental frequency pattern to be generated, such that the phoneme durations of the phonemes included in all sections of the selected representative vector after the number of the phonemes included in the first section are expanded/contracted equal the second designated values corresponding to the phoneme durations, to generate the fundamental frequency pattern.

2. The apparatus according to claim 1, wherein the calculation unit calculates one of an expansion/contraction ratio sequence which monotonically increases from a start of the first section and then monotonically decreases to an end of the first section, and an expansion/contraction ratio sequence which monotonically decreases from the start of the first section and then monotonically increases to the end of the first section.

3. The apparatus according to claim 1, wherein the section except the first section of the representative vector is a second section from a prosodic control unit start phoneme to one of an accent nucleus preceding adjacent phoneme, an accent nucleus phoneme, and an accent nucleus succeeding adjacent phoneme, and wherein the representative vector includes the second section and the first section following to the second section.

4. The apparatus according to claim 1, wherein the section except the first section of the representative vector includes a second section from a prosodic control unit start phoneme to one of an accent nucleus preceding adjacent phoneme, an accent nucleus phoneme, and an accent nucleus succeeding adjacent phoneme, and a third section from a succeeding adjacent phoneme to the first section to a prosodic control unit end phoneme, and wherein the representative vector includes the second section, the first section following to the second section, and the third section following to the second section.

5. The apparatus according to claim 1, wherein the prosodic control unit is at least one of a sentence unit, a breath group unit, an accent phrase unit, a morpheme unit, a word unit, a mora unit, a syllable unit, a phoneme unit, a semi-phoneme unit, a unit obtained by dividing one phoneme into a plurality of parts, and a unit formed by combining two or more of them.

6. The apparatus according to claim 1, wherein the context contains language information about the prosodic control unit, which is obtained by analyzing a text.

7. The apparatus according to claim 1, wherein the context contains a value of an arbitrary attribute.

8. The apparatus according to claim 7, wherein the attribute is at least one of information about prominence, information about an utterance style, information representing an intention, and information representing a mental attitude.

9. The apparatus according to claim 1, wherein the phoneme is at least one of a mora, syllable, phoneme, semi-phoneme, and a unit obtained by dividing one phoneme into a plurality of parts.

10. The apparatus according to claim 1, wherein the representative vector is at least one of a fundamental frequency pattern extracted from natural voice, an approximated fundamental frequency pattern obtained by approximating the fundamental frequency pattern, an quantized fundamental frequency pattern obtained by quantizing the fundamental frequency pattern extracted from the natural voice, and an approximated quantized fundamental frequency pattern obtained by approximating the quantized fundamental frequency pattern.

11. The apparatus according to claim 1, wherein the first and second designated values are values obtained from the input context.

12. The apparatus according to claim 1, wherein the first and second designated values are values obtained from input information different from the input context.

13. A fundamental frequency pattern generation apparatus comprising:

a computer apparatus comprising a non-transitory computer readable storage medium and a processor;

a first storage unit comprising the non-transitory computer readable storage medium storing a plurality of representative vectors each corresponding to a prosodic control unit and having a first section and a section except the first section, wherein the first section is a section of the representative vector, which starts with one of an accent nucleus phoneme, an accent nucleus succeeding adjacent phoneme, and an accent nucleus succeeding second phoneme and ends with one of a prosodic control unit end phoneme, a prosodic control unit end preceding adjacent phoneme, and a prosodic control unit end preceding second phoneme;

a second storage unit comprising the non-transitory computer readable storage medium storing a rule to select a representative vector corresponding to an input context;

a selection unit configured to select the representative vector corresponding to the input context from the plurality of representative vectors by applying the rule to the input context and output the selected representative vector;

a calculation unit comprising the processor configured to calculate an expansion/contraction ratio for number of phonemes included in the first section of the selected representative vector, based on a first designated value for a number of phonemes included in a first portion of a fundamental frequency pattern to be generated from the first section of the selected representative vector, the first designated value being required for the fundamental frequency pattern to be generated, such that the number of the phonemes included in the first section of the selected representative vector equals the first designated value; and

an expansion/contraction unit comprising the processor configured to expand/contract the number of the phonemes included in the first section of the selected representative vector based on the expansion/contraction ratio and then to expand/contract each of phoneme durations of the phonemes included in all sections of the selected representative vector after the number of the phonemes included in the first section are expanded/contracted, based on second designated values corresponding to phoneme durations of all phonemes included in all portions of the fundamental frequency pattern, the second designated values being required for the fundamental frequency pattern to be generated, such that the phoneme durations of the phonemes included in all sections of the selected representative vector after the number of the phonemes included in the first section are expanded/contracted equal the second designated values corresponding to the phoneme durations, to generate the fundamental frequency pattern.

14. The apparatus according to claim 13, wherein the section except the first section of the representative vector is a second section from a prosodic control unit start phoneme to one of an accent nucleus preceding adjacent phoneme, an accent nucleus phoneme, and an accent nucleus succeeding adjacent phoneme and wherein the representative vector includes the second section and the first section following to the second section.

15. The apparatus according to claim 13, wherein the section except the first section of the representative vector includes a second section from a prosodic control unit start phoneme to one of an accent nucleus preceding adjacent phoneme, an accent nucleus phoneme, and an accent nucleus succeeding adjacent phoneme, and a third section from a succeeding adjacent phoneme to the first section to a prosodic control unit end phoneme, and wherein the representative vector includes the second section, and the first section following to the second section, and the third section following to the first section.

16. The apparatus according to claim 13, wherein the prosodic control unit is at least one of a sentence unit, a breath group unit, an accent phrase unit, a morpheme unit, a word unit, a mora unit, a syllable unit, a phoneme unit, a semi-phoneme unit, a unit obtained by dividing one phoneme into a plurality of parts, and a unit formed by combining two or more of them.

17. The apparatus according to claim 13, wherein the context contains language information about the prosodic control unit, which is obtained by analyzing a text.

18. The apparatus according to claim 13, wherein the context contains a value of an arbitrary attribute.

19. The apparatus according to claim 18, wherein the attribute is at least one of information about prominence, information about an utterance style, information representing an intention, and information representing a mental attitude.

20. The apparatus according to claim 13, wherein the phoneme is at least one of a mora, syllable, phoneme, semi-phoneme, and a unit obtained by dividing one phoneme into a plurality of parts.

21. The apparatus according to claim 13, wherein the representative vector is at least one of a fundamental frequency pattern extracted from natural voice, an approximated fundamental frequency pattern obtained by approximating the fundamental frequency pattern, an quantized fundamental frequency pattern obtained by quantizing the fundamental frequency pattern extracted from the natural voice, and an approximated quantized fundamental frequency pattern obtained by approximating the quantized fundamental frequency pattern.

22. The apparatus according to claim 13, wherein the first and second designated values are values obtained from the input context.

23. The apparatus according to claim 13, wherein the first and second designated values are values obtained from input information different from the input context.

24. The apparatus according to claim 13, wherein the non-transitory computer readable storage medium comprises a device selected from the group consisting of an internal memory of the computer apparatus, an external memory of the computer apparatus, a hard disk of the computer apparatus and a storage medium readable by the computer apparatus.

25. The apparatus according to claim 24, wherein the storage medium is selected from the group consisting of a CD-R, CD-RW, DVD-RAM, and DVD-R.

26. A fundamental frequency pattern generation method comprising:

storing in advance a plurality of representative vectors each corresponding to a prosodic control unit and having a first section and a section except the first section, wherein the first section is a section of the representative vector, which starts with one of an accent nucleus phoneme, an accent nucleus succeeding adjacent phoneme, and an accent nucleus succeeding second phoneme and ends with one of a prosodic control unit end phoneme, a prosodic control unit end preceding adjacent phoneme, and a prosodic control unit end preceding second phoneme;

storing in advance a rule to select a representative vector corresponding to an input context;

selecting, via a computer processor, the representative vector corresponding to the input context from the plurality of representative vectors by applying the rule to the input context and output the selected representative vector;

calculating, via the computer processor, an expansion/contraction ratio for number of phonemes included in the first section of the selected representative vector, based on a designated value for number of phonemes included in a first portion of a fundamental frequency pattern to be generated from the first section of the selected representative vector, the designated value being required for the fundamental frequency pattern to be generated, such that the number of the phonemes included in the first section of the selected representative vector equals the designated value; and

expanding/contracting, via the computer processor, the number of the phonemes included in the first section of the selected representative vector based on the expansion/contraction ratio, and then expanding/contracting each of phoneme durations of the phonemes included in all sections of the selected representative vector after the number of the phonemes included in the first section are expanded/contracted, based on designated values corresponding to phoneme durations of all phonemes included in all portions of the fundamental frequency pattern, the designated values being required for the fundamental frequency pattern to be generated, such that the phoneme durations of the phonemes included in all sections of the selected representative vector after the number of the phonemes included in the first section are expanded/contracted equal the designated values corresponding to the phoneme durations, to generate the fundamental frequency pattern.

27. A non-transitory computer readable storage medium storing instructions of a computer program which when executed by a computer results in performance of steps comprising:

storing in advance a plurality of representative vectors each corresponding to a prosodic control unit and having a first section and a section except the first section, wherein the first section is a section of the representative vector, which starts with one of an accent nucleus phoneme, an accent nucleus succeeding adjacent phoneme, and an accent nucleus succeeding second phoneme and ends with one of a prosodic control unit end phoneme, a prosodic control unit end preceding adjacent phoneme, and a prosodic control unit end preceding second phoneme;

storing in advance a rule to select a representative vector corresponding to an input context;

selecting the representative vector corresponding to the input context from the plurality of representative vectors by applying the rule to the input context and output the selected representative vector;

calculating an expansion/contraction ratio for number of phonemes included in the first section of the selected representative vector, based on a designated value for number of phonemes included in a first portion of a fundamental frequency pattern to be generated from the first section of the selected representative vector, the designated value being required for the fundamental frequency pattern to be generated, such that the number of the phonemes included in the first section of the selected representative vector equals the designated value; and

expanding/contracting the number of the phonemes included in the first section of the selected representative vector based on the expansion/contraction ratio, and then expanding/contracting each of phoneme durations of the phonemes included in all sections of the selected representative vector after the number of the phonemes included in the first section are expanded/contracted, based on designated values corresponding to phoneme durations of all phonemes included in all portions of the fundamental frequency pattern, the designated values being required for the fundamental frequency pattern to be generated, such that the phoneme durations of the phonemes included in all sections of the selected representative vector after the number of the phonemes included in the first section are expanded/contracted equal the designated values corresponding to the phoneme durations, to generate the fundamental frequency pattern.

28. A fundamental frequency pattern generation method comprising:

storing, in non-transitory storage medium, a plurality of representative vectors each corresponding to a prosodic control unit and having a first section and a section except the first section, wherein the first section is a section of a representative vector;

storing, in non-transitory storage medium, a rule to select a representative vector corresponding to an input context;

selecting, via a computer processor, the representative vector corresponding to the input context from the plurality of representative vectors by applying the rule to the input context and output the selected representative vector;

calculating, via the computer processor, an expansion/contraction ratio for a number of phonemes included in the first section of the selected representative vector based on the selected representative vector such that the number of the phonemes included in the first section of the selected representative vector equals the designated value; and

expanding/contracting, via the computer processor, first the number of the phonemes included in the first section of the selected representative vector based on the expansion/contraction ratio and then each of phoneme durations of the phonemes.

29. A fundamental frequency pattern generation method comprising:

preparing in advance a first storage unit to store a plurality of representative vectors each corresponding to a prosodic control unit and having a first section including a plurality of sample points and a section except for the first section, wherein the first section is a section of the representative vector, which starts with one of an accent nucleus phoneme, an accent nucleus succeeding adjacent phoneme, and an accent nucleus succeeding second phoneme and ends with one of a prosodic control unit end phoneme, a prosodic control unit end preceding adjacent phoneme, and prosodic control unit end preceding second phoneme,

preparing in advance a second storage unit to store a rule to select a representative vector corresponding to an input context,

selecting, via a computer processor, the representative vector corresponding to the input context from the plurality of representative vectors by applying the rule to the input context and outputting the selected representative vector;

calculating, using a mapping function on the computer processor, an expansion/contraction ratio for a number of phonemes included in the first section of the selected representative vector, based on a designated value for a number of phonemes included in a first portion of a fundamental frequency pattern to be generated from the first section of the selected representative vector, the designated value being required for the fundamental frequency pattern to be generated, such that the number of the phonemes included in the first section of the selected representative vector equals the designated value; and

expanding/contracting, via the computer processor, the number of the phonemes included in the first section of the selected representative vector based on the expansion/contraction ratio, and then expanding/contracting each of the phoneme durations of the phonemes included in all sections of the selected representative vector after the number of the phonemes included in the first section are expanded/contracted, based on designated values corresponding to phoneme durations of all phonemes included in all portions of the fundamental frequency pattern, the designated values being required for the fundamental frequency pattern to be generated, such that the phoneme durations of the phonemes included in all sections of the selected representative vector after the number of the phonemes included in the first section are expanded/contracted equal the designated values corresponding to the phoneme durations, to generate the fundamental frequency pattern.

30. A non-transitory computer readable storage medium storing instructions of a computer program which when executed by a computer results in performance of steps comprising:

preparing in advance a first storage unit to store a plurality of representative vectors each corresponding to a prosodic control unit and having a first section including a plurality of sample points and a section except for the first section, wherein the first section is a section of the representative vector, which starts with one of an accent nucleus phoneme, an accent nucleus succeeding adjacent phoneme, and an accent nucleus succeeding second phoneme and ends with one of a prosodic control unit end phoneme, a prosodic control unit end preceding adjacent phoneme, and prosodic control unit end preceding second phoneme,

preparing in advance a second storage unit to store a rule to select a representative vector corresponding to an input context,

selecting the representative vector corresponding to the input context from the plurality of representative vectors by applying the rule to the input context and outputting the selected representative vector;

calculating, using a mapping function on the computer processor, an expansion/contraction ratio for a number of phonemes included in the first section of the selected representative vector, a designated value for a number of phonemes included in a first portion of a fundamental frequency pattern to be generated from the first section of the selected representative vector, the designated value being required for the fundamental frequency pattern to be generated, such that the number of the phonemes included in the first section of the selected representative vector equals the designated value; and

expanding/contracting, via the computer processor, the number of the phonemes included in the first section of the selected representative vector based on the expansion/contraction ratio, and then expanding/contracting each of the phoneme durations of the phonemes included in all sections of the selected representative vector after the number of the phonemes included in the first section are expanded/contracted, based on designated values corresponding to phoneme durations of all phonemes included in all portions of the fundamental frequency pattern, the designated values being required for the fundamental frequency pattern to be generated, such that the phoneme durations of the phonemes included in all sections of the selected representative vector after the number of the phonemes included in the first section are expanded/contracted equal the designated values corresponding to the phoneme durations, to generate the fundamental frequency pattern.