Prosodic control rule generation method and apparatus, and speech synthesis method and apparatus

• US 7,761,301 B2
• Filed: 10/20/2006
• Issued: 07/20/2010
• Est. Priority Date: 10/20/2005
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Xu, Dawei
• Primary Examiner(s): Sked; Matthew J
• Application Number:
  US11/583,969
• Publication Number:
  US 20070094030A1
• Time in Patent Office: 1,369 Days
• Field of Search: None
• US Class Current: 704/260
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation

Abstract

A prosodic control rule generation method includes dividing an input text into language units, estimating a punctuation mark incidence at a boundary between language units in the input text, the punctuation mark incidence indicating a degree that a punctuation mark occurs at the boundary, based on attribute information items of a plurality of language units adjacent to the boundary, and generating a prosodic control rule for speech synthesis including a condition for the punctuation mark incidence based on a plurality of learning data items each concerning prosody and including the punctuation mark incidence.

Claims

1. A computer implemented prosodic control rule generation method executed on a suitably programmed computer, the method including:

dividing an input text into language units;

estimating a punctuation mark incidence at a boundary between the language units, the punctuation mark incidence indicating a degree that a punctuation mark occurs at the boundary, based on attribute information items of a plurality of language units adjacent to the boundary; and

generating, by the computer, a prosodic control rule for speech synthesis including a condition for the punctuation mark incidence based on a plurality of learning data items each concerning prosody and including the punctuation mark incidence.

2. The computer implemented prosodic control rule generation method according to claim 1, wherein each of the learning data items further includes a word class of each of the language units, and

generating the prosodic control rule generates the prosodic control rule including conditions for the punctuation mark incidence between the language units and the word classes of the language units.

3. The computer implemented prosodic control rule generation method according to claim 1, wherein the estimating estimates the punctuation mark incidence at the boundary between a “

j−

1”

(j is a positive integer)- and “

j”

-th language units from the beginning of the input text, based on each of “

I+1”

language unit sequences each including I language units starting with a “

j−

i”

(i=0, 1, . . . , I, I is a positive integer equal to or larger than

     1)-th language unit.

4. The computer implemented prosodic control rule generation method according to claim 3, wherein the punctuation mark incidence at the boundary between the “

j−

1”

-th language unit and the “

j”

-th language unit is a weighted average of “

I+1”

punctuation mark incidences at the boundary between the “

j−

1”

-th language unit and the “

j”

-th language unit, the “

I+1”

punctuation mark incidences each being estimated from an arrangement of word classes in respective “

I+1”

language unit sequences.

5. A computer implemented prosodic control rule generation method executed on a suitably programmed computer, the method including:

dividing an input text into language units;

estimating a punctuation mark incidence at a boundary between language units in the input text, the punctuation mark incidence indicating a degree that a punctuation mark occurs at the boundary, based on attribute information items of a plurality of language units adjacent to the boundary;

generating a plurality of learning data items each concerning prosodic boundary between the language units and including the punctuation mark incidence between the language units; and

generating, by the computer, a prosodic boundary estimation rule for determining a type of a prosodic boundary and including a condition for the punctuation mark incidence between the language units based on the learning data items concerning prosodic boundary.

6. The computer implemented prosodic control rule generation method according to claim 5, wherein the type of the prosodic boundary is one of a prosodic word boundary, a prosodic phrase boundary, a breath group boundary, and a language unit boundary which is not the prosodic word boundary, the prosodic phrase boundary, or the breath group boundary.

7. The computer implemented prosodic control rule generation method according to claim 5, further including:

generating a plurality of learning data items each concerning prosody and including the type of the prosodic boundary between language units; and

generating a prosodic control rule for speech synthesis including a condition for the type of the prosodic boundary based on the learning data items concerning prosody.

8. The computer implemented prosodic control rule generation method according to claim 5, wherein the estimating estimates the punctuation mark incidence at the boundary between a “

j−

1”

(j is a positive integer)- and “

j”

-th language units from the beginning of the input text, based on each of “

I+1”

language unit sequences each including I language units starting with a “

j−

i”

(i=0, 1, . . . , I, I is a positive integer equal to or larger than

     1)-th language unit.

9. The computer implemented prosodic control rule generation method according to claim 8, wherein the punctuation mark incidence at the boundary between the “

j−

1”

-th language unit and the “

j”

-th language unit is a weighted average of “

I+1”

punctuation mark incidences at the boundary between the “

j−

1”

-th language unit and the “

j”

-th language unit, the “

I+1”

punctuation mark incidences each being estimated from an arrangement of word classes in respective “

I+1”

language unit sequences.

10. A computer implemented speech synthesis method executed on a suitably programmed computer, the method comprising:

dividing an input text into language units;

estimating a punctuation mark incidence at a boundary between language units in the input text, the punctuation mark incidence indicating a degree that a punctuation mark occurs at the boundary, based on attribute information items of a plurality of language units adjacent to the boundary;

selecting a prosodic control rule for speech synthesis based on the punctuation mark incidence; and

synthesizing, by the computer, a speech corresponding to the input text using the selected prosodic control rule.

11. The computer implemented speech synthesis method according to claim 10, wherein the selecting selects, from a plurality of prosodic control rules for speech synthesis each including a condition for the punctuation mark incidence between the language units, the prosodic control rule whose condition meets the punctuation mark incidence between the language units estimated.

12. The computer implemented speech synthesis method according to claim 11, wherein the prosodic control rules are generated based on a plurality of learning data items each concerning prosody and including the punctuation mark incidence between language units.

13. A computer implemented speech synthesis method executed on a suitably programmed computer comprising:

dividing an input text into language units;

estimating a punctuation mark incidence at a boundary between language units in the input text, the punctuation mark incidence indicating a degree that a punctuation mark occurs at the boundary, based on attribute information items of a plurality of language units adjacent to the boundary;

determining a type of a prosodic boundary between the language units based on the punctuation mark incidence between the language units estimated;

selecting a prosodic control rule for speech synthesis based on the type of the prosodic boundary between the language units determined; and

synthesizing, by the computer, a speech corresponding to the input text using the prosodic control rule selected.

14. The computer implemented speech synthesis method according to claim 13, wherein the determining the type includes:

selecting, from a group of a plurality of prosodic boundary estimation rules each including a condition for the punctuation mark incidence between the language units in order to determine the type of the prosodic boundary between the language units, the prosodic boundary estimation rule whose condition meets the punctuation mark incidence between the language units estimated, and determining the type of the prosodic boundary between the language units types based on the prosodic boundary estimation rule selected.

15. The computer implemented speech synthesis method according to claim 14, wherein the prosodic boundary estimation rules are generated based on a plurality of learning data items each concerning the boundary between the language units and including the punctuation mark incidence between the language units.

16. The computer implemented speech synthesis method according to claim 13, wherein the selecting selects, from a plurality of prosodic control rules for speech synthesis each including a condition for a type of the prosodic boundary between the language units, the prosodic control rule whose condition meets the type determined.

17. The computer implemented speech synthesis method according to claim 16, wherein the prosodic control rules are generated based on a plurality of learning data items each concerning prosody and including the type of the prosodic boundary between the language units.

18. The computer implemented speech synthesis method according to claim 13, wherein the determining the type includes:

selecting, from a plurality of groups each including a plurality of prosodic boundary estimation rules each including a condition for the punctuation mark incidence between the language units in order to determine the type of the prosodic boundary between the language units, a plurality of prosodic boundary estimation rules whose conditions meet the punctuation mark incidence between the language units estimated respectively, and determining the type of the prosodic boundary according to a majority decision rule among the prosodic boundary estimation rules selected.

19. A prosodic control rule generation apparatus including:

a dividing unit configured to divide an input text into language units;

an estimation unit configured to estimate a punctuation mark incidence at a boundary between the language units, the punctuation mark incidence indicating a degree that a punctuation mark occurs at the boundary, based on attribute information items of a plurality of language units adjacent to the boundary; and

a generation unit configured to generate a prosodic control rule for speech synthesis including a condition for the punctuation mark incidence based on a plurality of learning data items each concerning prosody and including the punctuation mark incidence.

20. A prosodic control rule generation apparatus including:

a dividing unit configured to divide an input text into language units;

an estimation unit configured to estimate a punctuation mark incidence at a boundary between language units in the input text, the punctuation mark incidence indicating a degree that a punctuation mark occurs at the boundary, based on attribute information items of a plurality of language units adjacent to the boundary;

a first generation unit configured to generate a plurality of learning data items each concerning prosodic boundary between the language units and including the punctuation mark incidence between the language units; and

a second generation unit configured to generate a prosodic boundary estimation rule for determining a type of a prosodic boundary and including a condition for the punctuation mark incidence between the language units based on the learning data items concerning prosodic boundary.

21. The prosodic control rule generation apparatus according to claim 20, further including:

a third generation unit configured to generate a plurality of learning data items each concerning prosody and including the type of the prosodic boundary between language units; and

a fourth generation unit configured to generate a prosodic control rule for speech synthesis including a condition for the type of the prosodic boundary based on the learning data items concerning prosody.

22. A speech synthesis apparatus comprising:

a dividing unit configured to divide an input text into language units;

an estimation unit configured to estimate a punctuation mark incidence at a boundary between language units in the input text, the punctuation mark incidence indicating a degree that a punctuation mark occurs at the boundary, based on attribute information items of a plurality of language units adjacent to the boundary;

a selecting unit configured to select a prosodic control rule for speech synthesis based on the punctuation mark incidence; and

a synthesizing unit configured to synthesize a speech corresponding to the input text using the selected prosodic control rule.

23. The speech synthesis apparatus according to claim 22, further comprising:

a memory to store a plurality of prosodic control rules for speech synthesis each including a condition for the punctuation mark incidence between the language units; and

wherein the selecting unit selects, from the prosodic control rules for speech synthesis, the prosodic control rule whose condition meets the punctuation mark incidence between the language units estimated.

24. A speech synthesis apparatus comprising:

a dividing unit configured to divide an input text into language units;

an estimation unit configured to estimate a punctuation mark incidence at a boundary between language units in the input text, the punctuation mark incidence indicating a degree that a punctuation mark occurs at the boundary, based on attribute information items of a plurality of language units adjacent to the boundary;

a determination unit configured to determine a type of a prosodic boundary between the language units based on the punctuation mark incidence between the language units estimated;

a selecting unit configured to select a prosodic control rule for speech synthesis based on the type of the prosodic boundary between the language units determined; and

a synthesizing unit configured to synthesize a speech corresponding to the input text using the prosodic control rule selected.

25. The speech synthesis apparatus according to claim 24, further comprising:

a first memory to store a group of a plurality of prosodic boundary estimation rules each including a condition for the punctuation mark incidence between the language units in order to determine the type of the prosodic boundary between the language units; and

wherein

the determination unit selects, from the group of a plurality of prosodic boundary estimation rules, the prosodic boundary estimation rule whose condition meets the punctuation mark incidence between the language units estimated, and determines the type of the prosodic boundary between the language units based on the prosodic boundary estimation rule selected.

26. The speech synthesis apparatus according to claim 24, further comprising:

a second memory to store a plurality of prosodic control rules for speech synthesis each including a condition for a type of the prosodic boundary between the language units; and

wherein

the selecting unit selects, from the prosodic control rules for speech synthesis, the prosodic control rule whose condition meets the type determined.

27. The speech synthesis apparatus according to claim 24, further comprising:

a first memory to store a plurality of groups each including a plurality of prosodic boundary estimation rules each including a condition for the punctuation mark incidence between the language units in order to determine the type of the prosodic boundary between the language units; and

wherein

the determination unit selects, from the groups, a plurality of prosodic boundary estimation rules whose conditions meet the punctuation mark incidence between the language units estimated respectively, and determines the type of the prosodic boundary according to a majority decision rule among the prosodic boundary estimation rules selected.