SPEECH PROCESSING APPARATUS AND PROGRAM

• US 20090177474A1
• Filed: 09/18/2008
• Published: 07/09/2009
• Est. Priority Date: 01/09/2008
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Morita, Masahiro, Kagoshima, Takehiko
• Granted Patent: US 8,195,464 B2
• Time in Patent Office: Days
• US Class Current: 704/260
• CPC Class Codes: G10L 13/07 Concatenation rules

Abstract

A speech synthesizer includes a periodic component fusing unit and an aperiodic component fusing unit, and fuses periodic components and aperiodic components of a plurality of speech units for each segment, which are selected by a unit selector, by a periodic component fusing unit and an aperiodic component fusing unit, respectively. The speech synthesizer is further provided with an adder, so that the adder adds, edits, and concatenates the periodic components and the aperiodic components of the fused speech units to generate a speech waveform.

Claims

1. A speech processing apparatus for carrying out text-to-speech synthesis, comprising:

an input unit to which a plurality of segments obtained by delimiting a phonological sequence corresponding to a target speech in units of synthesis and prosodic information on the respective segments corresponding to the target speech are entered;

a unit selector configured to select a plurality of first speech units from a group of speech units on the basis of the prosodic information for each of the plurality of segments;

a decomposer configured to decompose each of the plurality of first speech units into periodic components and aperiodic components for each of the plurality of segments;

a periodic component fusing unit configured to generate a second speech unit by fusing the periodic components of the plurality of first speech units for each of the plurality of segments;

an aperiodic component fusing unit configured to generate a third speech unit by fusing the aperiodic components of the plurality of first speech units for each of the plurality of segments; and

a generator configured to generate a synthesized speech by adding speech waveforms obtained respectively from the second speech unit and the third speech unit generated for each of the plurality of segments and concatenating the same among the segments.

4. The apparatus according to claim 1, wherein the generator includes:

an adder configured to generate a fourth speech unit by adding the second speech unit and the third speech unit for each of the plurality of segments; and

a concatenator configured to generate the synthesized speech by concatenating the speech waveforms obtained from the fourth speech units among the segments.

6. The apparatus according to claim 1, wherein the generator includes:

a first concatenator configured to concatenate the speech waveforms obtained from the second speech units among the segments to generate the speech waveform of the periodic components;

a second concatenator configured to concatenate the speech waveforms obtained from the third speech units among the segments to generate the speech waveform of the aperiodic components; and

an adder configured to add the periodic component waveform and the aperiodic component waveform to generate the synthesized waveform.

8. The apparatus according to claim 1, wherein the aperiodic component fusing unit includes:

a first generator configured to generate a set of fused spectrum parameters which represents spectrum characteristics of the plurality of aperiodic components of first speech units for each of the plurality of segments;

a second generator configured to generate a fused power envelope which represents the temporal change of the power of the plurality of aperiodic components; and

an output unit configured to output the set of fused spectrum parameters and the fused power envelope as the third speech unit, and

wherein the generator generates the speech waveform of the third speech unit from the set of fused spectrum parameters and the fused power envelope, and adds the speech waveform with the one obtained from the second speech unit for each of the plurality of segments.

10. The apparatus according to claim 1, wherein the aperiodic component fusing unit includes:

an analyzer configured to carry out linear prediction analysis for the aperiodic component waveforms of the plurality of first speech units and obtain a first set of linear prediction coefficients and a first linear prediction residual waveform respectively for each of the plurality of segments;

a first fusing unit configured to fuse the plurality of first sets of linear prediction coefficients and generate a second set of linear prediction coefficients;

a first extractor configured to extract a residual power envelope indicating the temporal change of the power of the respective first linear prediction residual waveform for each of the plurality of first linear prediction residual waveforms;

a second extractor configured to fuse the plurality of residual power envelopes to generate a second residual power envelope; and

an output unit configured to output the second set of linear prediction coefficients and the second residual power envelope as the third speech unit, and

wherein the generator generates the speech waveform of the third speech unit using the second set of linear prediction coefficients and the second residual power envelope.

12. The apparatus according to claim 1, wherein the aperiodic component fusing unit includes:

an analyzer configured to carry out linear prediction analysis for the aperiodic component waveforms of the plurality of first speech units and obtain a first set of linear prediction coefficients and a first linear prediction residual waveform respectively for each of the plurality of segments;

a second fusing unit configured to carry out the linear prediction analysis on the second aperiodic component waveform obtained by concatenating the aperiodic component waveforms of the plurality of first speech units to generate the second set of linear prediction coefficients;

a third extractor configured to extract the residual power envelope indicating the temporal change of the power of the respective first linear prediction residual waveform for each of the plurality of first linear prediction residual waveforms;

a fourth extractor configured to fuse the plurality of residual power envelopes to generate a second residual power envelope; and

an output unit configured to output the second set of linear prediction coefficients and the second residual power envelope as information relating to the third speech unit, and

wherein the generator generates the speech waveform of the third speech waveform using the second set of linear prediction coefficients and the second residual power envelope.

2. A speech processing apparatus for carrying out text-to-speech synthesis, comprising:

an input unit to which a plurality of segments obtained by delimiting a phonological sequence corresponding to a target speech in units of synthesis and prosodic information on the respective segments corresponding to the target speech are entered;

an environment storage configured to store speech-units'"'"' environments of a plurality of speech units;

a unit storage configured to store periodic components and aperiodic components of each of the speech units(, which were decomposed from the waveform data of each of the speech units);

an environment selector configured to select the unit environments of a plurality of first speech units from the environment storage on the basis of the prosodic information for each of the plurality of segments;

a periodic component fusing unit configured to extract the periodic components of the first speech units corresponding to the selected unit environments of the plurality of first speech units from the unit storage and fuse the periodic components to generate the second speech unit for each of the plurality of segments;

an aperiodic component fusing unit configured to extract the aperiodic components of the first speech units corresponding to the unit environments of the plurality of first speech units from the unit storage and fuse the aperiodic components to generate a third speech unit for each of the plurality of segments; and

a generator configured to generate a synthesized speech by adding speech waveforms obtained respectively from the second speech units and the third speech units of the plurality of segments and concatenating the same among the segments.

3. The apparatus according to claim 2, wherein the unit environment of the first speech units selected by the environment storage is the same or different between the periodic components and the aperiodic components.

5. The apparatus according to claim 2, wherein the generator includes:

an adder configured to generate the fourth speech unit by adding the second speech unit and the third speech unit for each of the plurality of segments; and

a concatenator configured to generate the synthesized speech by concatenating the speech waveforms obtained from the fourth speech units among the segments.

7. The apparatus according to claim 2, wherein the generator includes:

a first concatenator configured to concatenate the speech waveforms obtained from the second speech units among the segments to generate the speech waveform of the periodic components;

a second concatenator configured to concatenate the speech waveforms obtained from the third speech units among the segments to generate the speech waveform of the aperiodic components; and

an adder configured to add the periodic component waveform and the aperiodic component waveform to generate the synthesized waveform.

9. The apparatus according to claim 2, wherein the aperiodic component fusing unit includes:

a first generator configured to generate a set of fused spectrum parameters which represents spectrum characteristics of the plurality of aperiodic components of first speech units for each of the plurality of segments;

a second generator configured to generate a fused power envelope which represents the temporal change of the powers of the plurality of aperiodic components; and

an output unit configured to output the set of fused spectrum parameters and the fused power envelope as the third speech unit, and

wherein the generator generates the speech waveform of the third speech unit from the set of fused spectrum parameters and the fused power envelope, and adds the speech waveform with the one obtained from the second speech unit for each of the plurality of segments.

11. The apparatus according to claim 2, wherein the aperiodic component fusing unit includes:

an analyzer configured to carry out linear prediction analysis for the aperiodic component waveforms of the plurality of first speech units and obtain a first set of linear prediction coefficients and a first linear prediction residual waveform respectively for each of the plurality of segments;

a first fusing unit configured to fuse the plurality of first sets of linear prediction coefficients and generate a second set of linear prediction coefficients;

a first extractor configured to extract a residual power envelope indicating the temporal change of the power of the respective first linear prediction residual waveform for each of the plurality of first linear prediction residual waveforms;

a second extractor configured to fuse the plurality of residual power envelopes to generate a second residual power envelope; and

an output unit configured to output the second set of linear prediction coefficients and the second residual power envelope as the third speech unit, and

wherein the generator generates the speech waveform of the third speech unit using the second set of linear prediction coefficients and the second residual power envelope.

13. The apparatus according to claim 2, wherein the aperiodic component fusing unit includes:

an analyzer configured to carry out linear prediction analysis for the aperiodic component waveforms of the plurality of first speech units and obtain a first set of linear prediction coefficients and a first linear prediction residual waveform respectively for each of the plurality of segments;

a second fusing unit configured to carry out the linear prediction analysis on the second aperiodic component waveform obtained by concatenating the aperiodic component waveforms of the plurality of first speech units to generate the second set of linear prediction coefficients;

a third extractor configured to extract the residual power envelope indicating the temporal change of the power of the respective first linear prediction residual waveform for each of the plurality of first linear prediction residual waveform;

a fourth extractor configured to fuse the plurality of residual power envelopes to generate a second residual power envelope; and

an output unit configured to output the second set of linear prediction coefficients and the second residual power envelope as information relating to the third speech unit, and

wherein the generator generates the speech waveform of the third speech waveform using the second set of linear prediction coefficients and the second residual power envelope.

14. A speech processing apparatus for creating a storage for storing a plurality of speech units used for text-to-speech synthesis comprising:

an input unit to which a plurality of segments obtained by delimiting a phonological sequence corresponding to a target speech in units of synthesis and prosodic information on the respective segments corresponding to the target speech are entered;

a unit selector configured to select a plurality of first speech units from a group of the speech units on the basis of the prosodic information for each of the plurality of segments;

a decomposer configured to decompose each of the plurality of first speech units into periodic components and aperiodic components for each of the plurality of segments;

a periodic component fusing unit configured to generate a second speech unit by fusing the periodic components of the plurality of first speech units for each of the plurality of segments;

an aperiodic component fusing unit configured to generate a third speech unit by fusing the aperiodic components of the plurality of first speech units for each of the plurality of segments; and

the storage configured to store the plurality of second speech units and the plurality of third speech units.

16. The apparatus according to claim 14, wherein the storage extracts and stores the second speech units and the third speech units of a specified amount from the plurality of second speech units and the plurality of third speech units on the basis of the frequency of appearance of the speech units or the quantity of characteristics of the speech units.

15. A speech processing apparatus for creating a storage configured to store a plurality of speech units used for text-to-speech synthesis comprising:

a unit storage configured to store periodic components and aperiodic components of each of the speech units(, which were decomposed from the waveform data of each of the speech units);

an input unit to which a plurality of segments obtained by delimiting a phonological sequence corresponding to a target speech in units of synthesis and prosodic information on the respective segments corresponding to the target speech are entered;

a component selector configured to select the periodic components and the aperiodic components of the plurality of first speech units from the unit storage on the basis of the prosodic information for each of the plurality of segments;

a periodic component fusing unit configured to generate a second speech unit by fusing the periodic components of the plurality of first speech units for each of the plurality of segments;

an aperiodic component fusing unit configured to generate a third speech unit by fusing the aperiodic components of the plurality of first speech units for each of the plurality of segments; and

the storage configured to store the plurality of second speech units and the plurality of third speech units.

17. The apparatus according to claim 15, wherein the storage extracts and stores the second speech units and the third speech units of a specified amount from the plurality of second speech units and the plurality of third speech units on the basis of the frequency of appearance of the speech units or the quantity of characteristics of the speech units.

18. A speech processing program product configured to carry out text-to-speech synthesis and stored in a computer readable medium, the computer realizing the functions of:

accepting a plurality of segments obtained by delimiting a phonological sequence corresponding to a target speech in units of synthesis and prosodic information on the respective segments corresponding to the target speech;

selecting a plurality of first speech units from a group of speech units on the basis of the prosodic information for each of the plurality of segments;

decomposing each of the plurality of first speech units into periodic components and aperiodic components for each of the plurality of segments;

generating a second speech unit by fusing the periodic components of the plurality of first speech units for each of the plurality of segments;

generating a third speech unit by fusing the aperiodic components of the plurality of first speech units for each of the plurality of segments; and

generating a synthesized speech by adding speech waveform obtained respectively from the second speech unit and the third speech unit generated for each of the plurality of segments and concatenating the same among the segments.

19. A speech processing program product configured to carry out text-to-speech synthesis and stored in a computer readable medium, the computer comprising:

an environment storage configured to store unit environments of a plurality of speech units;

a unit storage configured to store periodic components and aperiodic components of each of the speech units (which were decomposed from the waveform data of each of the speech units);

the computer realizing the functions of;

accepting a plurality of segments obtained by delimiting a phonological sequence corresponding to a target speech in units of synthesis and prosodic information on the respective segments corresponding to the target speech;

selecting the unit environments of a plurality of first speech units from the environment storage on the basis of the prosodic information for each of the plurality of segments;

extracting the periodic components of the first speech units corresponding to the selected unit environments of the plurality of first speech units from the unit storage and fusing the periodic components individually to generate the second speech unit for each of the plurality of segments;

extracting the aperiodic components of the first speech units corresponding to the selected unit environments of the plurality of first speech units from the unit storage and fusing the aperiodic components individually to generate third speech unit for each of the plurality of segments; and

generating a synthesized speech by adding speech waveform obtained respectively from the second speech unit and the third speech unit for each of the plurality of segments and concatenating the same among the segments.

20. A speech processing program product for creating a storage configured to store a plurality of speech units used for text-to-speech synthesis stored in a computer readable medium, the computer realizing the functions of:

accepting a plurality of segments obtained by delimiting a phonological sequence corresponding to a target speech in units of synthesis and prosodic information on the respective segments corresponding to the target speech;

selecting a plurality of first speech units from a group of the speech units on the basis of the prosodic information for each of the plurality of segments;

decomposing each of the plurality of first speech units into periodic components and aperiodic components for each of the plurality of segments;

generating a second speech unit by fusing the periodic components of the plurality of first speech units for each of the plurality of segments;

generating a third speech unit by fusing the aperiodic components of the plurality of first speech units for each of the plurality of segments; and

storing the plurality of second speech units and the plurality of third speech units in the storage.

21. A speech processing program product for creating a storage configured to store a plurality of speech units used for text-to-speech synthesis stored in a computer readable medium, the computer comprising:

a unit storage configured to store periodic components and aperiodic components of each of the plurality of speech units(, which were decomposed from the waveform data of each of the speech units);

the computer realizing the functions of;

accepting a plurality of segments obtained by delimiting a phonological sequence corresponding to a target speech in units of synthesis and prosodic information on the respective segments corresponding to the target speech;

selecting the periodic components and the aperiodic components of the plurality of first speech units from the unit storage on the basis of the prosodic information for each of the plurality of segments;

generating a second speech unit by fusing the periodic components of the plurality of first speech units for each of the plurality of segments;

generating a third speech unit by fusing the aperiodic components of the plurality of first speech units for each of the plurality of segments; and

storing the plurality of second speech units and the plurality of third speech units in the storage.