Speech synthesizing apparatus, method, and program

• US 8,630,857 B2
• Filed: 02/15/2008
• Issued: 01/14/2014
• Est. Priority Date: 02/20/2007
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Kato, Masanori, Kondo, Reishi, Mitsui, Yasuyuki
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US12/527,802
• Publication Number:
  US 20100076768A1
• Time in Patent Office: 2,160 Days
• Field of Search: 704/260, 704/258, 704/207, 704/267, 704/266, 704/205, 704/220, 704/268, 704/270.1
• US Class Current: 704/266
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 13/06 Elementary speech units used in speech synthesisers; Concatenation rules

Abstract

Disclosed is a speech synthesizing apparatus including a segment selection unit that selects a segment suited to a target segment environment from candidate segments, includes a prosody change amount calculation unit that calculates prosody change amount of each candidate segment based on prosody information of candidate segments and the target segment environment, a selection criterion calculation unit that calculates a selection criterion based on the prosody change amount, a candidate selection unit that narrows down selection candidates based on the prosody change amount and the selection criterion, and an optimum segment search unit than searches for an optimum segment from among the narrowed-down candidate segments.

Claims

1. A speech synthesizing apparatus comprising:

a storage unit that stores speech segments; and

a segment selection unit that selects a segment suited to a target segment environment from among a plurality of candidate segments selected from the storage unit, wherein

the segment selection unit performs control to exclude, from the candidate segment which is a candidate of the selection, a segment having a prosody change amount less than a selection criterion that is determined based on a prosody change amount of the candidate segments.

2. The speech synthesizing apparatus according to claim 1, wherein the segment selection unit comprises:

a prosody change amount calculation unit that calculates a prosody change amount of each candidate segment, based on prosody information of the target segment environment and the candidate segments;

a selection criterion calculation unit that calculates a selection criterion, based on the prosody change amount;

a candidate selection unit that narrows down selection candidates, based on the prosody change amount and the selection criterion; and

an optimum segment search unit that searches for an optimum segment from among the narrowed-down candidate segments;

wherein the candidate selection unit excludes, from selection candidates, a segment having a prosody change amount less than the selection criterion, and excludes the segment from a target of search for an optimum segment by the optimum segment search unit.

3. The speech synthesizing apparatus according to claim 2, wherein the selection criterion calculation unit comprises:

a cost calculation unit that calculates a cost of each candidate segment based on the target segment environment and a segment environment of the candidate segments; and

calculates the selection criterion based on the cost.

4. The speech synthesizing apparatus according to claim 1, wherein the segment selection unit comprises:

an optimum segment search unit that searches for optimum segments based on the target segment environment and a segment environment of the candidate segments;

a prosody change amount calculation unit that calculates a prosody change amount of each candidate segment, based on prosody information of the target segment environment and the candidate segments;

a selection criterion calculation unit that calculates a selection criterion based on the prosody change amount; and

a decision unit that decides, in a case where, among the optimum segments, there exists a segment having a prosody change amount less than the selection criterion, that re-execution of search for an optimum segment is necessary;

wherein in a case where the decision unit decides that the re-execution of the search for an optimum segment is necessary, the optimum segment search unit re-executes the search for an optimum segment.

5. The speech synthesizing apparatus according to claim 4, wherein the prosody change amount calculation unit calculates the prosody change amount for only the optimum segments.

6. The speech synthesizing apparatus according to claim 4, wherein the optimum segment search unit excludes segments that do not satisfy the selection criterion from candidates, and re-executes search for optimum segments.

7. The speech synthesizing apparatus according to claim 1, wherein the segment selection unit comprises:

a prosody change amount calculation unit that calculates a prosody change amount of each candidate segment, based on prosody information of the target segment environment and the candidate segments;

a selection criterion calculation unit that calculates a selection criterion from the prosody change amount;

a unit cost calculation unit that calculates a unit cost of each candidate segment based on the target segment environment and a segment environment of the candidate segments; and

an optimum segment search unit that searches for an optimum segment from among candidate segments based on the unit cost;

wherein the unit cost calculation unit assigns a penalty to a unit cost of a segment having a prosody change amount less than the selection criterion.

8. The speech synthesizing apparatus according to claim 7, wherein the unit cost calculation unit determines the penalty, the penalty being made larger in accordance with increase in a difference between the prosody change amount and the selection criterion.

9. The speech synthesizing apparatus according to claim 2, wherein the selection criterion calculation unit determines the selection criterion based on an average value of the prosody change amount.

10. The speech synthesizing apparatus according to claim 2, wherein the selection criterion calculation unit determines the selection criterion based on a value obtained by smoothing the prosody change amount in a time domain.

15. The speech synthesizing apparatus according to claim 2, wherein a selection criterion used by the candidate selection unit is determined in advance, or is input from outside the speech synthesizing apparatus, and there is no necessity to compute a selection criterion based on the prosody change amount by the selection criterion calculation unit.

16. The speech synthesizing apparatus according to claim 1, further comprising, in addition to the segment selection unit:

a language processing unit that generates a language processing result including a symbol sequence representing a reading from text, and morphological part of speech, conjugation, and accent information;

a prosody generation unit that generates prosody information of synthesized speech generated based on the language processing result;

a prosody control unit that generates a waveform having a prosody generated by the prosody generation unit, from speech segments selected by the segment selection unit;

a waveform connection unit that concatenates speech segments output by the prosody control unit, to output the result as synthesized speech; and

a speech segment information storage unit that stores speech segments divided into synthesis units and attribute information of each speech segment;

wherein the segment selection unit comprises;

a unit cost calculation unit that receives the language processing result generated by the language processing unit, and prosody information generated by the prosody generation unit, generates the target segment environment for each synthesis unit, selects, as candidate segments, a plurality of speech segments matching information designated by the target segment environment, from the speech segment information storage unit, and, calculates a unit cost of each candidate segment, based on segment environment of the candidate segments and the target segment environment;

a prosody change amount calculation unit that calculates prosody change amount of each candidate segment, based on the prosody information, the unit cost of a plurality of candidate segments, and attribute information of each speech segment from the speech segment information storage unit;

a selection criterion calculation unit that calculates a selection criterion for candidates necessary for narrowing down candidate segments, based on prosody change amount of each of the candidate segments;

a candidate selection unit that narrows down candidate segments, based on the selection criterion from the selection criterion calculation unit, the prosody change amount from the prosody change amount calculation unit, and the unit cost and information of each candidate segment from the unit cost calculation unit, and excludes, from candidates, a segment of which the prosody change amount is small compared to others, based on the selection criterion, from among candidate segments of which the unit cost is relatively low, and outputs information of the narrowed-down and selected candidate segments and unit cost thereof;

a concatenation cost calculation unit that calculates concatenation cost of each of the candidate segments, based on information of each of the candidate segments, and attribute information of each speech segment from the speech segment information storage unit; and

an optimum segment search unit that obtains, based on information of the candidate segments, the unit cost, and the concatenation cost, an optimum segment sequence, which is a speech segment sequence in which an objective function related to the unit cost and the concatenation cost is optimized, to be provided to the prosody control unit.

17. The speech synthesizing apparatus according to claim 1, further comprising, in addition to the segment selection unit:

a language processing unit that generates a language processing result including a symbol sequence representing a of synthesized speech generated based on the language processing reading from text, and morphological part of speech, conjugation, and accent information;

a prosody generation unit that generates prosody information result;

a prosody control unit that generates a waveform having a prosody generated by the prosody generation unit, from speech segments selected by the segment selection unit;

a waveform connection unit that concatenates speech segments output by the prosody control unit, to output the result as synthesized speech; and

a speech segment information storage unit that stores speech segments divided into synthesis units and attribute information of each speech segment;

wherein the segment selection unit comprises;

a unit cost calculation unit that receives the language processing result generated by the language processing unit, and the prosody information generated by the prosody generation unit, generates the target segment environment for each synthesis unit, selects, as candidate segments, a plurality of speech segments matching information designated by the target segment environment, from the speech segment information storage unit, and, calculates a unit cost of each candidate segment, based on a segment environment of the candidate segments and the target segment environment;

a concatenation cost calculation unit that calculates concatenation cost of each of the candidate segments, based on information of each of the candidate segments, and attribute information of each speech segment from the speech segment information storage unit;

a candidate selection unit that narrows down candidate segments, based on information of each of the candidate segments, the unit cost and the concatenation cost, and outputs information of the narrowed-down and selected candidate segments and unit cost thereof;

an optimum segment search unit that obtains, based on information of the candidate segments, the unit cost, and the concatenation cost, an optimum segment sequence, which is a speech segment sequence in which an objective function related to the unit cost and the concatenation cost is optimized, to be provided to the prosody control unit;

a prosody change amount calculation unit that calculates prosody change amount of optimum segments in question, based on each segment of the optimum segment sequence output from the optimum segment search unit, the prosody information from the prosody generation unit, and attribute information of the optimum segments from the speech segment information storage unit;

a selection criterion calculation unit that calculates a selection criterion necessary for distinguishing existence of a segment whose prosody change amount is particularly small in comparison to others, based on prosody change amount of each optimum segment from the prosody change amount calculation unit; and

a decision unit that decides whether or not there exists a segment whose prosody change amount is particularly small in comparison to others, based on optimum segments from the optimum segment search unit, prosody change amount of each segment from the prosody change amount calculation unit, and a selection criterion supplied from the selection criterion calculation unit,

in a case where it is decided that there exists a segment whose prosody change amount is particularly small in comparison to others, supplies a segment whose prosody change amount is particularly small to the candidate selection unit, the candidate selection unit re-executing search of candidate segments, and in a case where it is decided that there does not exist a segment whose prosody change amount is particularly small in comparison to others, or in a case where the number of times the search is repeated exceeds an upper limit, and supplies optimum segments to the prosody control unit;

wherein the candidate selection unit excludes, a segment supplied from the decision unit, from among the candidate segments supplied from the concatenation cost calculation unit, and supplies a candidate segment that is not excluded, and unit cost and concatenation cost of the candidate segment to the optimum segment search unit.

18. The speech synthesizing apparatus according to claim 1, further comprising, in addition to the segment selection unit:

a language processing unit that generates a language processing result including a symbol sequence representing a reading from text, and morphological part of speech, conjugation, and accent information;

a prosody generation unit that generates prosody information of synthesized speech generated based on the language processing result;

a prosody control unit that generates a waveform having a prosody generated by the prosody generation unit, from speech segments selected by the segment selection unit;

a waveform connection unit that concatenates speech segments output by the prosody control unit, to output the concatenated as synthesized speech; and

a speech segment information storage unit that stores speech segments divided into synthesis units and attribute information of each speech segment;

wherein the segment selection unit comprises;

a unit cost calculation unit that receives the language processing result generated by the language processing unit, and the prosody information generated by the prosody generation unit, generates the target segment environment for each synthesis unit, selects, as candidate segments, a plurality of speech segments matching information designated by the target segment environment, from the speech segment information storage unit, and, calculates a unit cost of each candidate segment, based on a segment environment of the candidate segments and the target segment environment;

a prosody change amount calculation unit that calculates prosody change amount of each candidate segment, based on the prosody information, the unit cost of each of the plurality of candidate segments, and attribute information of each speech segment from the speech segment information storage unit;

a selection criterion calculation unit that calculates a selection criterion for candidates necessary for narrowing down candidate segments, based on prosody change amount of each of the candidate segments;

a unit cost correcting unit that corrects a unit cost of a candidate segment of which the prosody change amount is small in comparison to other segments, based on the selection criterion from the selection criterion calculation unit, the prosody change amount of candidate segments supplied from the prosody change amount calculation unit, and the unit cost and information of each candidate segment supplied from the unit cost calculation unit;

a concatenation cost calculation unit that calculates concatenation cost of each candidate segment, based on information of each of the candidate segments, and the attribute information of each speech segment from the speech segment information storage unit; and

an optimum segment search unit that obtains, based on information of the candidate segments, the unit cost, and the concatenation cost, an optimum segment sequence, which is a speech segment sequence in which an objective function related to the unit cost and the concatenation cost is optimized, to be provided to the prosody control unit.

11. A speech synthesizing method comprising:

providing a non-transitory storage unit, coupled to a processor, that stores speech segments;

providing a segment selection unit;

selecting a plurality of candidate segments for a target segment environment from the storage unit that stores speech segments; and

selecting with the segment selection unit a segment suited to the target segment environment from among a plurality of candidate segments,

wherein the step of selecting the segment comprises performing control to exclude, from the candidate segment which is a candidate of the selection, a segment that has a prosody change amount less than a selection criterion determined based on a prosody change amount of the candidate segments.

12. The speech synthesizing method according to claim 11, wherein the step of selecting the segment comprises:

calculating a prosody change amount of each candidate segment, based on prosody information of the target segment environment and the candidate segments;

calculating a selection criterion based on the prosody change amount;

narrowing down selection candidates, based on the prosody change amount and the selection criterion; and

searching for an optimum segment from among the narrowed-down candidate segments;

wherein the step of narrowing down the candidate selection comprises excluding, from the selection candidates, a segment that has a prosody change amount less than the selection criterion.

13. A non-transitory computer-readable recording medium storing a program that causes a computer which constitutes a speech synthesizing apparatus, to execute:

a processing of selecting a plurality of candidate segments for a target segment environment from a storage unit that stores speech segments; and

a processing of selecting a segment suited to a target segment environment from among a plurality of candidate segments,

wherein the processing of selecting the segment comprises;

performing control excluding, from the candidate segment which is a candidate of the selection, a segment that has a prosody change amount less than a selection criterion that is determined based on a prosody change amount of candidate segments.

14. The recording medium according to claim 13, wherein the processing of selecting the segment comprises:

a processing of calculating a prosody change amount of each candidate segment, based on prosody information of the target segment environment and the candidate segments;

a processing of calculating a selection criterion based on the prosody change amount;

a processing of narrowing down selection candidates, based on the prosody change amount and the selection criterion; and

a processing of searching for an optimum segment from among the narrowed-down candidate segments;

wherein the processing of narrowing down the selection candidates comprises;

a processing of excluding, from the candidates, a segment that has a prosody change amount less than the selection criterion.