Speech synthesizing method and apparatus using prosody control

• US 7,054,815 B2
• Filed: 03/27/2001
• Issued: 05/30/2006
• Est. Priority Date: 03/31/2000
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Yamada, Masayuki, Komori, Yasuhiro
• Primary Examiner(s): HARPER, V PAUL
• Application Number:
  US09/818,886
• Publication Number:
  US 20010037202A1
• Time in Patent Office: 1,890 Days
• Field of Search: 704/258, 704/267
• US Class Current: 704/267
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 13/06 Elementary speech units used in speech synthesisers; Concatenation rules
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation

Abstract

A speech synthesizing apparatus extracts small speech segments from a speech waveform as a prosody control target and adds inhibition information for inhibiting a predetermined prosody change process to a selected small speech segment in executing prosody control. Prosody control is performed by performing a predetermined prosody change process by using small speech segments of the extracted small speech segments other than small speech segments to which inhibition information is added. This makes it possible to prevent a deterioration in synthesized speech due to waveform editing operation.

Claims

1. A speech synthesizing method comprising:

an extraction step of extracting a plurality of speech segments from a speech waveform;

an adding step of adding limitation information for inhibiting execution of predetermined processing to a selected speech segment of the plurality of speech segments;

a prosody control step of processing the plurality of speech segments to control prosody of the speech waveform, wherein the prosody control step inhibits execution of the predetermined processing for a speech segment to which the limitation information is added; and

a synthesizing step of obtaining synthesized speech by using the speech waveform for which prosody control is performed in the prosody control step.

2. The method according to claim 1, wherein

the predetermined processing includes deletion of a speech segment, and

in the prosody control step, deletion of the speech segment to which the limitation information is added is inhibited when reduction of an utterance time of synthesized speech is performed as the prosody control.

3. The method according to claim 1, wherein

the predetermined processing includes repetition of a speech segment, and

in the prosody control step, repetition of a speech segment to which the limitation information is added is inhibited when prolongation of a time of synthesized speech is performed as the prosody control.

4. The method according to claim 1, wherein

the predetermined processing includes a change in an interval of a speech segment, and

in the prosody control step, a change in an interval of a speech segment to which the limitation information is added is inhibited when making a change in a fundamental frequency of synthesized speech as the prosody control.

5. The method according to claim 1, wherein

a storage unit in which a plurality of window functions arranged along a time axis and limitation information corresponding to at least one of the window functions are stored is used,

in the extraction step, speech segments are extracted from a speech waveform by using the plurality of window functions, and

in the prosody control step, when limitation information is made to correspond to a window function, a speech segment extracted by using the window function is selected and the limitation is imposed on the speech segment on the basis of the limitation information.

6. The method according to claim 1, wherein in the adding step, the limitation information is added to a speech segment corresponding to a specific position on a speech waveform.

7. The method according to claim 6, wherein the specific position includes a boundary between a voiced sound portion and an unvoiced sound portion.

8. The method according to claim 6, wherein the specific position includes a phoneme boundary.

9. The method according to claim 6, wherein the specific position is a predetermined range including a plosive, and the predetermined range includes a plurality of speech segments.

10. The method according to claim 1, wherein the speech waveform comprises the plurality of speech segments;

and

wherein the prosody control step do not execute the predetermined processing to the speech segments in case that the limitation information is effective.

11. A speech synthesizing apparatus comprising:

an extraction unit configured to extract a plurality of speech segments from a speech waveform;

an adding unit configured to add limitation information for inhibiting execution of predetermined processing to a selected speech segment of the plurality of speech segments;

a prosody control unit configured to process the plurality of speech segments to control prosody of the speech waveform, wherein the prosody control step inhibits execution of the predetermined processing for a speech segment to which the limitation information is added; and

a synthesizing unit configured to obtain synthesized speech by using the speech waveform for which prosody control is performed by said prosody control unit.

12. The apparatus according to claim 11, wherein

the predetermined processing includes deletion of a speech segment, and

said prosody control unit inhibits deletion of the speech segment to which the limitation information is added when reduction of an utterance time of synthesized speech is performed as the prosody control.

13. The apparatus according to claim 11, wherein the predetermined processing includes repetition of a speech segment, and

said prosody control unit inhibits repetition of a speech segment to which the limitation information is added when prolongation of a time of synthesized speech is performed as the prosody control.

14. The apparatus according to claim 11, wherein

the predetermined processing includes a change in an interval of a speech segment, and

said prosody control unit inhibits a change in an interval of a speech segment to which the limitation information is added when making a change in a fundamental frequency of synthesized speech as the prosody control.

15. The apparatus according to claim 11, further comprising a storage unit in which a plurality of window functions arranged along a time axis and limitation information corresponding to at least one of the window functions are stored,

wherein said extraction unit extracts speech segments from a speech waveform by using the plurality of window functions, and

said prosody control unit, when limitation information is made to correspond to a window function, selects a speech segment extracted by using the window function and imposes the limitation on the basis of the limitation information.

16. The apparatus according to claim 11, wherein said adding unit adds the limitation information to a speech segment corresponding to a specific position on a speech waveform.

17. The apparatus according to claim 16, wherein the specific position includes a boundary between a voiced sound portion and an unvoiced sound portion.

18. The apparatus according to claim 16, wherein the specific position includes a phoneme boundary.

19. The apparatus according to claim 16, wherein the specific position is a predetermined range including a plosive, and the predetermined range includes a plurality of speech segments.

20. The apparatus according to claim 11, wherein the speech waveform comprises the plurality of speech segments;

and

wherein the prosody control unit do not execute the predetermined processing to the speech segments in case that the limitation information is effective.

21. A control program for making a computer implement a speech synthesizing method comprising:

an extraction step of extracting a plurality of speech segments from a speech waveform;

an adding step of adding limitation information for inhibiting execution of predetermined processing to a selected speech segment of the plurality of speech segments;

a prosody control step of processing the plurality of speech segments to control prosody of the speech waveform, wherein the prosody control step inhibits execution of the predetermined processing for a speech segment to which the limitation information is added; and

a synthesizing step of obtaining synthesized speech by using the speech waveform for which prosody control is performed in the prosody control step.

22. A storage medium storing a control program for making a computer implement a speech synthesizing method comprising;

an extraction step of extracting a plurality of speech segments from a speech waveform;

an adding step of adding limitation information for inhibiting execution of predetermined processing to selected speech segment of the plurality of speech segments;

a prosody control step of processing the plurality of speech segments to control prosody of the speech waveform, wherein the prosody control step inhibits execution of the predetermined processing for a speech segment to which the limitation information is added; and

a synthesizing step of obtaining synthesized speech by using the speech waveform for which prosody control is performed in the prosody control step.

23. A speech synthesizing method comprising:

an extraction step of extracting a plurality of speech segments from a speech waveform;

a prosody control step of processing the plurality of speech segments to control prosody of the speech waveform, wherein the prosody control step inhibits execution of the predetermined processing for a speech segment based on the limitation information corresponding to the speech waveform; and

a synthesizing step of obtaining synthesized speech by using the speech waveform for which prosody control is performed in the prosody control step.

24. The method according to claim 23, wherein the speech waveform comprises the plurality of speech segments;

and

wherein the prosody control step do not execute the predetermined processing to the speech segments in case that the limitation information is effective.

25. The method according to claim 24, wherein the limitation information is effective for a speech segment corresponding to a specific position on a speech waveform.

26. The method according to claim 25, wherein specific position includes a boundary between a voiced sound portion and an unvoiced sound portion.

27. The method according to claim 25, wherein specific position includes a phoneme boundary.

28. The method according to claim 25, wherein the specific position includes a plosive.

29. The method according to claim 23, wherein the predetermined processing includes deletion of a speech segment, and in the prosody control step, deletion of the speech segment is inhibited in case that prolongation of a time of synthesized speech is performed as the prosody control.

30. The method according to claim 23, wherein the predetermined processing includes repetition of a speech segment, and in the prosody control step, repetition of a speech segment is inhibited in case that prolongation of a time of synthesized speech is performed as the prosody.

31. The method according to claim 23, wherein the predetermined processing includes a change in an interval of a speech segment, and in the prosody control step, a change in an interval of a speech segment is inhibited in case that making a change in a fundamental frequency of synthesized speech as the prosody control.

32. A speech synthesizing apparatus comprising:

an extraction unit configured to extract a plurality of speech segments from a speech waveform;

a prosody control unit configured to process the plurality of speech segments to control prosody of the speech waveform, wherein the prosody control step inhibits execution of the predetermined processing for a speech segment based on the limitation information corresponding to the speech waveform; and

a synthesizing unit configured to obtain synthesized speech by using the speech waveform for which prosody control is performed by said prosody control unit.

33. The apparatus according to claim 32, wherein the speech waveform comprises the plurality of speech segments;

and

wherein the prosody control unit do not execute the predetermined processing to the speech segments in case that the limitation information is effective.

34. A control program for making a computer implement a speech synthesizing method comprising:

an extraction step of extracting a plurality of speech segments from a speech waveform;

a prosody control step of processing the plurality of speech segments to control prosody of the speech waveform, wherein the prosody control step inhibits execution of the predetermined processing for a speech segment based on the limitation information corresponding to the speech waveform; and

a synthesizing step of obtaining synthesized speech by using the speech waveform for which prosody control is performed in the prosody control step.

35. A storage medium storing a control program for making a computer implement a speech synthesizing method comprising:

an extraction step of extracting a plurality of speech segments from a speech waveform;

a prosody control step of processing the plurality of speech segments to control prosody of the speech waveform, wherein the prosody control step inhibits execution of the predetermined processing for a speech segment based on the limitation information corresponding to the speech waveform; and

a synthesizing step of obtaining synthesized speech by using the speech waveform for which prosody control is performed in the prosody control step.