Domain adaptation for TTS systems

• US 7,328,157 B1
• Filed: 01/24/2003
• Issued: 02/05/2008
• Est. Priority Date: 01/24/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Peng, Hu, Chu, Min
• Primary Examiner(s): Vo; Huyen X.
• Application Number:
  US10/350,850
• Time in Patent Office: 1,838 Days
• Field of Search: 704/245, 704/258, 704/259, 704/260, 704265-266, 704268-269, 704/261, 704/262, 704/267
• US Class Current: 704/260
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination

Abstract

Embodiments of the present invention pertain to adaptation of a corpus-driven general-purpose TTS system to at least one specific domain. The domain adaptation is realized by adding a limited amount of domain-specific speech that provides a maximum impact on improved perceived naturalness of speech. An approach for generating optimized script for adaptation is proposed, the core of which is a dynamic programming based algorithm that segments domain-specific corpus into a minimum number of segments that appear in the unit inventory. Increases in perceived naturalness of speech after adaptation are estimated from the generated script without recording speech from it.

Claims

1. A method adapting a text-to-speech system, the method comprising:

supplying a domain-specific text corpus that corresponds to a target domain;

supplying a plurality of scripts that correspond to an inventory of speech units utilized by the text-to-speech system to synthesize speech;

generating a list of candidate domain-specific strings using text from the domain-specific text corpus, wherein each candidate domain-specific string occurs at least a predetermined number of times within the domain-specific text corpus, wherein the predetermined number is more than once;

generating a domain-specific script using said domain-specific string so as to include at least one domain-specific string included in the list of candidate domain-specific strings; and

adapting the text-to-speech system based on the domain-specific script so as to improve the perceived naturalness of synthesized speech.

3. The method of claim 1, further comprising identifying from the list of candidate domain-specific strings a first qualified domain-specific string that will maximize improvement of the naturalness of synthetic speech produced by the text-to-speech system on the target domain, wherein generating the domain-specific script comprises generating the domain-specific script so as to include the first qualified domain-specific string.

4. The method of claim 3, wherein identifying the first qualified domain-specific string comprises:

identifying the candidate domain-specific string that, if added to the plurality of scripts that correspond to the inventory of speech units, will maximize the average length of segments utilized by the text-to-speech system to synthesize speech.

5. The method of claim 3, wherein identifying the first qualified domain-specific string is done without recording speech for candidate domain-specific strings in the list.

6. The method of claim 3, wherein identifying the first qualified domain-specific string comprises:

measuring an average segment length before and after each candidate domain-specific string is added to the plurality of scripts that correspond to the inventory of speech units; and

identifying the candidate domain-specific string that produces the greatest increase in average segment length.

7. The method of claim 3, wherein identifying the first qualified domain-specific string comprises:

measuring an average concatenative cost before and after each candidate domain-specific string is added to the plurality of scripts that correspond to the inventory of speech units; and

identifying the candidate domain-specific string that produces the greatest decrease in average concatenative cost.

8. The method of claim 3, wherein identifying the first qualified domain-specific string comprises:

measuring a mean opinion score before and after each candidate domain-specific string is added to the plurality of scripts that correspond to the inventory of speech units; and

identifying the candidate domain-specific string that produces the greatest increase in mean opinion score.

9. The method of claim 3, further comprising removing from the list of candidate domain-specific strings the first qualified domain-specific string.

10. The method of claim 9, further comprising:

repeating said identifying, generating and removing steps for additional qualified domain-specific strings until the list of candidate domain-specific strings is empty, or until the number of qualified domain-specific strings for which speech is added to the unit inventory reaches a predetermined limit.

11. The method of claim 3, further comprising removing from the list of candidate domain-specific strings those candidate domain-specific strings that are sub-strings of other candidate domain-specific strings.

12. The method of claim 3, further comprising removing from the list of candidate domain-specific strings those candidate domain-specific strings that are shorter than a predetermined length.

13. The method of claim 3, wherein generating the domain-specific script comprises generating a domain dependent sentence that includes the first qualified domain-specific string.

14. The method of claim 13, wherein generating the domain dependent sentence comprises manually writing the domain dependent sentence.

15. The method of claim 13, wherein generating the domain dependent sentence comprises selecting a sentence from the domain-specific text corpus.

16. The method of claim 15, wherein selecting a sentence from the domain-specific text corpus comprises selecting a sentence that, when added to the inventory of speech units utilized by the text-to-speech system to synthesize speech, will maximize the average length of segments.

17. The method of claim 15, wherein selecting a sentence from the domain-specific text corpus comprises selecting a sentence that, when added to the inventory of speech units utilized by the text-to-speech system to synthesize speech, will maximize the mean opinion score.

18. The method of claim 15, wherein selecting a sentence from the domain-specific text corpus comprises selecting a sentence that, when added to the inventory of speech units utilized by the text-to-speech system to synthesize speech, will minimize the average concatenative cost.

2. The method of 1, wherein each candidate domain-specific string does not occur within the plurality of scripts that correspond to the inventory of speech units.

19. A method for generating a domain-specific script for domain adaptation of a text-to-speech system, the method comprising:

supplying a domain-specific text corpus that corresponds to a target domain;

supplying a plurality of scripts that correspond to an inventory of speech units utilized by the text-to-speech system to synthesize speech;

generating a list of candidate domain-specific strings using text from the domain-specific text corpus, wherein each candidate domain-specific string occurs a predetermined number of times within the domain-specific text corpus, where in the predetermined number of times is more than once;

selecting from the list, based on an objective criteria, a limited number of candidate domain-specific strings that, if added to the plurality of scripts that correspond to the inventory of speech units, will the naturalness of synthetic speech produced by the text-to-speech system on the target domain;

generating a domain-specific script so as to include the limited number of candidate domain-specific strings; and

adapting the text-to-speech system based on the domain-specific script so as to improve the perceived naturalness of synthesized speech.

20. The method of claim 19, wherein selecting from the list a limited number of candidate domain-specific strings comprises:

selecting from the list a limited number of candidate domain-specific strings that, if added to the plurality of scripts that correspond to the inventory of speech units, will raise an average length of all segments included in the plurality of scripts.

21. The method of claim 19, wherein selecting from the list a limited number of candidate domain-specific strings comprises:

selecting from the list a limited number of candidate domain-specific strings that, if added to the plurality of scripts that correspond to the inventory of speech units, will raise a mean opinion score associated with the plurality of scripts.

22. The method of claim 19, wherein selecting from the list a limited number of candidate domain-specific strings comprises:

selecting from the list a limited number of candidate domain-specific strings that, if added to the plurality of scripts that correspond to the inventory of speech units, will lower an average concatenative cost associated with the plurality of scripts.