System and method for unified normalization in text-to-speech and automatic speech recognition

• US 10,199,034 B2
• Filed: 08/18/2014
• Issued: 02/05/2019
• Est. Priority Date: 08/18/2014
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Conkie, Alistair D., Golipour, Ladan
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US14/461,930
• Publication Number:
  US 20160049144A1
• Time in Patent Office: 1,632 Days
• Field of Search: 704231, 704243, 704244, 704249, 704258, 704260, 704266
• CPC Class Codes: G10L 13/06 Elementary speech units used in speech synthesisers; Concatenation rules
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

A system, method and computer-readable storage devices are for using a single set of normalization protocols and a single language lexica (or dictionary) for both TTS and ASR. The system receives input (which is either text to be converted to speech or ASR training text), then normalizes the input. The system produces, using the normalized input and a dictionary configured for both automatic speech recognition and text-to-speech processing, output which is either phonemes corresponding to the input or text corresponding to the input for training the ASR system. When the output is phonemes corresponding to the input, the system generates speech by performing prosody generation and unit selection synthesis using the phonemes. When the output is text corresponding to the input, the system trains both an acoustic model and a language model for use in future speech recognition.

Claims

1. A method comprising:

receiving first text for training an automatic speech recognition process;

receiving second text to be converted to speech via a text-to-speech process;

normalizing, via a processor, the first text using a single set of normalization protocols that apply to both the automatic speech recognition process and the text-to-speech process, to yield first normalized text;

normalizing, via the processor, the second text using the single set of normalization protocols that apply to both the automatic speech recognition process and the text-to-speech process, to yield second normalized text, wherein normalized text comprises the first normalized text and the second normalized text;

generating, via the processor and using the normalized text and a dictionary configured for both the automatic speech recognition process and the text-to-speech process, output comprising one of output text corresponding to the first text and phonemes corresponding to the second text, wherein the dictionary comprises a rate of use of the automatic speech recognition process, a rate of use of the text-to-speech process, accent rules, language preference rules and location-based rules configured for both the automatic speech recognition process and the text-to-speech process;

when the output comprises the phonemes corresponding to the second text, generating speech by performing prosody generation and unit selection synthesis using the phonemes and the dictionary;

when the output comprises the output text corresponding to the first text, training both an acoustic model and a language model using the output text and the dictionary; and

performing speech recognition using the acoustic model and the language model.

2. The method of claim 1, wherein the training of the acoustic model and the language model uses the phonemes.

3. The method of claim 1, further comprising outputting the speech to a user as part of a dialog system.

4. The method of claim 1, wherein the dictionary comprises syllable boundaries.

5. The method of claim 1, wherein the dictionary comprises contextual rules.

6. The method of claim 5, wherein the output is generated based on the contextual rules.

7. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

receiving first text for training an automatic speech recognition process;

receiving second text to be converted to speech via a text-to-speech process;

normalizing the first text using a single set of normalization protocols that apply to both the automatic speech recognition process and the text-to-speech process, to yield first normalized text;

normalizing the second text using the single set of normalization protocols that apply to both the automatic speech recognition process and the text-to-speech process, to yield second normalized text, wherein normalized text comprises the first normalized text and the second normalized text;

generating, using the normalized text and a dictionary configured for both the automatic speech recognition process and the text-to-speech process, output comprising one of output text corresponding to the first text and phonemes corresponding to the second text, wherein the dictionary comprises a rate of use of the automatic speech recognition process, a rate of use of the text-to-speech process, accent rules, language preference rules and location-based rules configured for both the automatic speech recognition process and the text-to-speech process;

when the output comprises the phonemes corresponding to the second text, generating speech by performing prosody generation and unit selection synthesis using the phonemes and the dictionary;

when the output comprises the output text corresponding to the first text, training both an acoustic model and a language model using the output text and the dictionary; and

performing speech recognition using the acoustic model and the language model.

8. The system of claim 7, wherein the training of the acoustic model and the language model uses the phonemes.

9. The system of claim 7, the computer-readable storage medium having additional instructions stored which, when executed by the processor, cause the processor to perform operations comprising outputting the speech to a user as part of a dialog system.

10. The system of claim 7, wherein the dictionary comprises syllable boundaries.

11. The system of claim 7, wherein the dictionary comprises contextual rules.

12. The system of claim 11, wherein the output is generated based on the contextual rules.

13. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

receiving first text for training an automatic speech recognition process;

receiving second text to be converted to speech via a text-to-speech process;

normalizing the first text using a single set of normalization protocols that apply to both the automatic speech recognition process and the text-to-speech process, to yield first normalized text;

normalizing the second text using the single set of normalization protocols that apply to both the automatic speech recognition process and the text-to-speech process, to yield second normalized text, wherein normalized text comprises the first normalized text and the second normalized text;

generating, using the normalized text and a dictionary configured for both the automatic speech recognition process and the text-to-speech process, output comprising one of output text corresponding to the first text and phonemes corresponding to the second text, wherein the dictionary comprises a rate of use of the automatic speech recognition process, a rate of use of the text-to-speech process, accent rules, language preference rules and location-based rules configured for both the automatic speech recognition process and the text-to-speech process;

when the output comprises the phonemes corresponding to the second text, generating speech by performing prosody generation and unit selection synthesis using the phonemes and the dictionary;

when the output comprises the output text corresponding to the first text, training both an acoustic model and a language model using the output text and the dictionary; and

performing speech recognition using the acoustic model and the language model.

14. The computer-readable storage device of claim 13, wherein the training of the acoustic model and the language model uses the phonemes.

15. The computer-readable storage device of claim 13, having additional instructions stored which, when executed by the computing device, cause the computing device to perform operations comprising outputting the speech to a user as part of a dialog system.

16. The computer-readable storage device of claim 13, wherein the dictionary comprises syllable boundaries.

17. The computer-readable storage device of claim 13, wherein the dictionary comprises contextual rules.