Prosody template matching for text-to-speech systems

• US 6,845,358 B2
• Filed: 01/05/2001
• Issued: 01/18/2005
• Est. Priority Date: 01/05/2001
• Status: Expired due to Term

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Applebaum, Ted H., Kibre, Nicholas
• Primary Examiner(s): To, Doris H.
• Assistant Examiner(s): Nolan, Daniel A
• Application Number:
  US09/755,699
• Publication Number:
  US 20020128841A1
• Time in Patent Office: 1,474 Days
• Field of Search: 704/260, 704/258, 704/257, 704/254, 704/246, 704/10
• US Class Current: 704/260
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation

Abstract

A prosody matching template in the form of a tree structure stores indices which point to lookup table and template information prescribing pitch and duration values that are used to add inflection to the output of a text-to-speech synthesizer. The lookup module employs a search algorithm that explores each branch of the tree, assigning penalty scores based on whether the syllable represented by a node of the tree does or does not match the corresponding syllable of the target word. The path with the lowest penalty score is selected as the index into the prosody template table. The system will add nodes by cloning existing nodes in cases where it is not possible to find a one-to-one match between the number of syllables in the target word and the number of nodes in the tree.

Claims

1. A text-to-speech synthesizer system, comprising:

a text input module receptive of target synthesis text;

a prosody module connected to the text input module for associating prosody information with the target synthesis text, the prosody module employing an n-way tree structure of plural traversal paths to identify the prosody information for the target synthesis text;

the prosody module having a prosody pattern lookup module that traverses said tree structure, the lookup module having a stored matrix of penalty values associated with said plural traversal oaths and being operative to identify the prosody information for the target synthesis text corresponding to the traversal path of lowest penalty value; and

a sound generation module connected to the prosody module for converting the target synthesis text to audible speech using the prosody information.

2. The text-to-speech synthesizer system of claim 1 wherein the prosody module employs a tree structure that is based on stress patterns, such that each node of the tree structure corresponds to a stress level that may be associated with a syllabic portion of a text string.

3. The text-to-speech synthesizer system of claim 2 wherein the text input module is operative to segment the target synthesis text into syllabic portions and to determine a stress level for each syllabic portion, thereby forming a stress pattern for the target synthesis text.

4. The text-to-speech synthesizer system of claim 3 wherein the prosody module is operative to traverse the tree structure in order to identify a matching stress pattern that corresponds to the stress pattern for the target synthesis text and to retrieve the prosody information for the target synthesis text using the matching stress pattern.

5. The text-to-speech synthesizer system of claim 1 wherein the prosody information is further defined as pitch modification information and duration modification information.

22. The system of claim 1 wherein the target synthesis text is arranged as target words with an associated context and wherein said prosody pattern lookup module includes a system for determining the associated context of a target word and for modifying at least one penalty value associated with said target word.

6. A method for generating synthesized speech, comprising the steps of:

receiving an input text string;

employing an n-way tree structure to identify prosody information for the input text string, where the tree structure is based on stress patterns such that each node of the tree structure provides a stress level that may be associated with a syllabic portion of a text string;

the prosody module having a prosody pattern lookup module that traverses said tree structure, the lookup module having a stored matrix of penalty values associated with said plural traversal paths and being operative to identify the prosody information for the target synthesis text corresponding to the traversal Path of lowest penalty value; and

converting the input text string into audible speech using the prosody information.

7. The method of claim 6 further comprising the steps of:

segmenting the input text string into syllabic portions;

determining a stress level for each syllabic portion of the input text string, thereby forming a stress pattern for the input text string;

traversing the tree structure in order to identify a matching stress pattern that matches the stress pattern for the input text string; and

using the matching stress pattern to retrieve the prosody information for the input text string.

8. The method of claim 7 wherein the step of traversing the tree structure further comprises the steps of:

comparing a stress level for a syllabic portion of the input text string with a stress level for the corresponding syllabic portion in the tree structure;

determining a matching score indicative of the correlation between the stress level for the syllabic portion of the input text string and the stress level for the corresponding syllabic portion in the tree structure; and

using the matching score to identify a matching stress pattern that correlates to the stress pattern for the input text string.

9. The method of claim 8 wherein the step of determining a matching score further comprises modifying the matching score based on at least one of the context of the syllabic portion within a transcription derived from the input text string and the context of a word within the input text string, where the word incorporates the syllabic portion used to determine the matching score.

10. The method of claim 8 further comprises the steps of:

accumulating a matching score for each path that is traversed in the tree structure;

storing a stress pattern having the lowest matching score;

updating the stress pattern having the lowest matching score when the matching score for a given path is less than or equal to the lowest matching score; and

ceasing to traverse a path in the tree structure when the matching score for the given path exceeds the lowest matching score.

11. The method of claim 7 wherein the step of traversing the tree structure further comprises the step of constructing a stress pattern that correlates to the stress pattern of the input text string, when a matching stress pattern is not identified in the tree structure.

12. The method of claim 11 wherein the step of constructing a stress pattern further comprises the steps of:

identifying one or more target nodes having stress patterns that correlate to the stress pattern of the input text string;

cloning a stress level from an adjacent syllabic portion in the target node, when the number of syllabic portions in the target node is less than the number of syllabic portions in the input text string; and

concatenating the stress level onto the stress pattern of the target node, thereby constructing a stress pattern that correlates to the stress pattern of the input text string.

13. The method of claim 12 wherein the step of constructing a stress pattern further comprises the steps of:

determining a matching score indicative of the correlation between the stress patterns for each of the target nodes and the stress pattern of the input text string; and

using the matching score to identify a target node that most closely correlates to the stress pattern of the input text string.

14. The method of claim 13 further comprising the steps of:

retrieving the prosody information for the identified target node;

cloning a portion of the prosody information that corresponds to the cloned adjacent syllabic portion of the target node; and

concatenating the portion of the prosody information onto the remainder of the prosody information, thereby constructing the prosody information that corresponds to the identified target node.

15. The method of claim 14 further comprising the step of converting the input text string to audible speech using the prosody information that corresponds to the identified target node.

16. The method of claim 6 wherein the prosody information is further defined as pitch modification information and duration modification information.

17. A method for generating prosody information for use in a text-to-speech synthesizer system, comprising the steps of:

receiving an input text string;

determining a pattern of prosodic features associated with the input text string;

identifying a first prosody template from a plurality of prosody templates by traversing an n-way tree structure in order to identify a matching pattern of prosodic features, where the tree structure is based on stress patterns and has plural nodes such that each node provides a stress level that may be associated with a syllabic portion of a text string, and where each prosody template represents a pattern of prosodic features that may be associated with a text string and the first prosody template having a pattern of prosodic features that correlate to the input text string;

cloning a portion of the first prosody template by cloning one of said nodes, when the pattern for the first prosody template is shorter than the pattern for the input text string; and

concatenating the replicated portion of the first prosody template onto the pattern of the first prosody template, thereby constructing a generated prosody template that more closely correlates to the input text string.

18. The method of claim 17 further comprising the steps of using the generated prosody template to retrieve prosody information for the input text string, and converting the input text string into audible speech using the prosody information.

19. The method of claim 17 wherein each prosody template is further defined as a pattern of stress levels for each syllabic portion of a text string.

20. The method of claim 17 wherein the step of determining a pattern of prosodic features further comprises the steps of:

segmenting the input text string into syllabic portions; and

determining a stress level for each syllabic portion of the input text string, thereby forming a stress pattern for the input text string.

21. The method of claim 17 wherein the step of cloning a portion of the first prosody template further comprises the steps of cloning a stress level from an adjacent syllabic portion of the matching pattern, when the number of syllabic portions in the first prosody template is less than the number of syllabic portions of the stress pattern for the input text string, and concatenating the stress level onto the matching pattern of the first prosody template.