System and method for generating customized text-to-speech voices

• US 9,721,558 B2
• Filed: 12/10/2015
• Issued: 08/01/2017
• Est. Priority Date: 05/13/2004
• Status: Expired due to Term

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Feng, Junlan, Schroeter, Juergen, Syrdal, Ann K., Schulz, David, Bangalore, Srinivas, Gilbert, Mazin
• Primary Examiner(s): Neway, Samuel G
• Application Number:
  US14/965,251
• Publication Number:
  US 20160093287A1
• Time in Patent Office: 600 Days
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 13/06 Elementary speech units used in speech synthesisers; Concatenation rules
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams

Abstract

A system and method are disclosed for generating customized text-to-speech voices for a particular application. The method comprises generating a custom text-to-speech voice by selecting a voice for generating a custom text-to-speech voice associated with a domain, collecting text data associated with the domain from a pre-existing text data source and using the collected text data, generating an in-domain inventory of synthesis speech units by selecting speech units appropriate to the domain via a search of a pre-existing inventory of synthesis speech units, or by recording the minimal inventory for a selected level of synthesis quality. The text-to-speech custom voice for the domain is generated utilizing the in-domain inventory of synthesis speech units. Active learning techniques may also be employed to identify problem phrases wherein only a few minutes of recorded data is necessary to deliver a high quality TTS custom voice.

Claims

1. A method comprising:

receiving, at a first time, a selection of an animated character to guide a user on a website;

collecting text data from a pre-existing text data source, to yield collected text data, wherein the collected text data is associated with a domain of the website, wherein the pre-existing text data source exists at the first time, and wherein no in-domain inventory of speech units exists at the first time;

selecting synthesis speech units specific to the domain from a pre-existing inventory of synthesis speech units existing at the first time, wherein the selecting occurs using the collected text data to yield selected synthesis speech units, wherein the synthesis speech units comprise one or more of phonemes, diphones, triphones and syllables;

caching the selected synthesis speech units specific to the domain to generate an in-domain inventory of synthesis speech units; and

generating, via a processor and at a second time which is later than the first time, a custom text-to-speech voice for use by the animated character when performing a specific task in the domain utilizing the in-domain inventory of synthesis speech units.

2. The method of claim 1, further comprising determining whether the custom text-to-speech voice conforms to a selected level of synthesis quality.

3. The method of claim 2, further comprising:

when the custom text-to-speech voice does not conform to the selected level of synthesis quality, collecting additional text data associated with the domain.

4. The method of claim 3, further comprising iteratively collecting the additional text data until the custom text-to-speech voice conforms to the selected level of synthesis quality.

5. The method of claim 1, wherein the pre-existing text data source is one of a domain-related website, e-mail, and transcriptions of conversations.

6. The method of claim 1, wherein the pre-existing text data source is a sector-related website distinct from the website.

7. The method of claim 6, further comprising categorizing websites by sector to identify websites as pre-existing text data sources prior to collecting the text data.

8. The method of claim 1, wherein collecting the text data associated with the domain from the pre-existing text data source further comprises mining specific words and phrases from the pre-existing text data source.

9. The method of claim 8, wherein mining the specific words and phrases from the pre-existing text data source further comprises mining the specific words and phrases using an n-gram selection.

10. The method of claim 8, wherein mining the specific words and phrases from the pre-existing text data source further comprises mining the specific words and phrases using a maximal mutual information approach.

11. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

receiving, at a first time, a selection of an animated character to guide a user on a website;

collecting text data from a pre-existing text data source, to yield collected text data, wherein the collected text data is associated with a domain of the website, wherein the pre-existing text data source exists at the first time, and wherein no in-domain inventory of speech units exists at the first time;

selecting synthesis speech units specific to the domain from a pre-existing inventory of synthesis speech units existing at the first time, wherein the selecting occurs using the collected text data to yield selected synthesis speech units, wherein the synthesis speech units comprise one or more of phonemes, diphones, triphones and syllables;

caching the selected synthesis speech units specific to the domain to generate an in-domain inventory of synthesis speech units; and

generating, via a processor and at a second time which is later than the first time, a custom text-to-speech voice for use by the animated character when performing a specific task in the domain utilizing the in-domain inventory of synthesis speech units.

12. The system of claim 11, further comprising determining whether the custom text-to-speech voice conforms to a selected level of synthesis quality.

13. The system of claim 12, the computer-readable storage medium having additional instructions stored which, when executed by the processor, cause the processor to perform operations comprising:

when the custom text-to-speech voice does not conform to the selected level of synthesis quality, collecting additional text data associated with the domain.

14. The system of claim 13, the computer-readable storage medium having additional instructions stored which, when executed by the processor, cause the processor to perform operations comprising iteratively collecting the additional text data until the custom text-to-speech voice conforms to the selected level of synthesis quality.

15. The system of claim 11, wherein the pre-existing text data source is one of a domain-related website, e-mail, and transcriptions of conversations.

16. The system of claim 11, wherein the pre-existing text data source is a sector-related website distinct from the website.

17. The system of claim 16, the computer-readable storage medium having additional instructions stored which, when executed by the processor, cause the processor to perform operations comprising categorizing websites by sector to identify websites as pre-existing text data sources prior to collecting the text data.

18. The system of claim 11, wherein collecting the text data associated with the domain from the pre-existing text data source further comprises mining specific words and phrases from the pre-existing text data source.

19. The system of claim 18, wherein mining the specific words and phrases from the pre-existing text data source further comprises mining the specific words and phrases using an n-gram selection.

20. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

receiving, at a first time, a selection of an animated character to guide a user on a website;

collecting text data from a pre-existing text data source, to yield collected text data, wherein the collected text data is associated with a domain of the website, wherein the pre-existing text data source exists at the first time, and wherein no in-domain inventory of speech units exists at the first time;

selecting synthesis speech units specific to the domain from a pre-existing inventory of synthesis speech units existing at the first time, wherein the selecting occurs using the collected text data to yield selected synthesis speech units, wherein the synthesis speech units comprise one or more of phonemes, diphones, triphones and syllables;

caching the selected synthesis speech units specific to the domain to generate an in-domain inventory of synthesis speech units; and

generating, via a processor and at a second time which is later than the first time, a custom text-to-speech voice for use by the animated character when performing a specific task in the domain utilizing the in-domain inventory of synthesis speech units.