Method, device and system for generating segment durations in a text-to-speech system

• US 5,950,162 A
• Filed: 10/30/1996
• Issued: 09/07/1999
• Est. Priority Date: 10/30/1996
• Status: Expired due to Term

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Massey, Noel, Corrigan, Gerald, Karaali, Orhan
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): SAX, ROBERT L
• Application Number:
  US08/739,975
• Time in Patent Office: 1,042 Days
• Field of Search: 704/260, 704/232, 704/200, 704/268, 704/259
• US Class Current: 704/260
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination

Abstract

The present invention teaches a method (400), device and system (300) utilizing at least one of: mapping a sequence of phones to a sequence of articulatory features and utilizing prominence and boundary information, in addition to a predetermined set of rules for type, phonetic context, syntactic and prosodic context for phones to provide provide a system that generates segment durations efficiently with a small training set.

Claims

1. A method for generating segment durations in a text-to-speech system, wherein, for input text that generates a linguistic description of speech to be uttered including at least one segment description, comprising the steps of:

A) generating a linguistic description-based information vector for each segment description in the linguistic description of the input text, wherein the information vector includes a description of a sequence of segments surrounding a described segment and descriptive information for a context associated with the described segment;

B) providing the information vector as input to a pretrained neural network having feedforward neural network elements, wherein the training data for the pretrained neural network has been generated by recording natural speech, partitioning the speech data into segments associated with identified phones, and marking at least one of syntactical, intonational, and stress information; and

C) generating a representation of a duration associated with each phone in a sequence of phones in the described segment by the pretrained neural network.

2. The method of claim 1 wherein:

A) the speech is described as a sequence of phone identifications;

B) the segments for which duration is being generated are segments of speech expressing predetermined phones in the sequence of phone identifications; and

C) segment descriptions include the phone identifications.

3. The method of claim 2 wherein the descriptive information includes at least one of:

A) articulatory features associated with each phone in the sequence of phones;

B) locations of syllable, word and other syntactic and intonational boundaries;

C) syllable strength information;

D) descriptive information of a word type; and

E) rule firing information.

4. The method of claim 1 wherein the representation of the duration is a logarithm of the duration.

5. The method of claim 1 wherein the pretrained neural network has been trained using back-propagation of errors.

6. The method of claim 1 wherein the steps of the method are stored in a memory unit of a computer.

7. The method of claim 1 wherein the steps of the method are embodied in a tangible medium of/for a Digital Signal Processor, DSP.

8. The method of claim 1 wherein the steps of the method are embodied in a tangible medium of/for an Application Specific Integrated Circuit, ASIC.

9. The method of claim 1 wherein the steps of the method are embodied in a tangible medium of a gate array.

20. The method of claim 1, further comprising the step of:

adjusting the representation of the duration associated with each phone in the sequence of phones in the described segment to provide another representation of the duration associated with each phone in the sequence of phones in the described segment.

10. A device for generating segment durations in a text-to-speech system, for input text that generates a linguistic description of speech to be uttered including at least one segment description, comprising:

A) a linguistic description-based information preprocessor, operably coupled to receive the linguistic description of speech to be uttered, for generating a linguistic description-based information vector for each segment description in the linguistic description of the speech to be uttered, wherein the information vector includes a description of a sequence of segments surrounding a described segment and descriptive information for a context associated with the described segment; and

B) a pretrained neural network having feedforward neural network elements, wherein the training data for the pretrained neural network has been generated by recording natural speech, partitioning the speech data into segments associated with identified phones, and marking at least one of syntactical, intonational, and stress information, and wherein the pretrained neural network is operably coupled to the linguistic information preprocessor, for generating a representation of a duration associated with each phone in a sequence of phones in the described segment by the pretrained neural network.

11. The device of claim 10 wherein:

A) the speech is described as a sequence of phone identifications;

B) the segments for which the duration is being generated are segments of speech expressing predetermined phones in the sequence of phone identifications; and

C) segment descriptions include the phone identifications.

12. The device of claim 11 wherein the descriptive information includes at least one of:

A) articulatory features associated with each phone in the sequence of phones;

B) locations of syllable, word and other syntactic and intonational boundaries;

C) syllable strength information;

D) descriptive information of a word type; and

E) rule firing information.

13. The device of claim 10 wherein the representation of the duration is a logarithm of the duration.

14. The device of claim 10 wherein the pretrained neural network has been trained using back-propagation of errors.

21. The device of claim 13, wherein the representation of the duration associated with each phone in the sequence of phones in the described segment is adjusted to provide another representation of the duration associated with each phone in the sequence of phones in the described segment.

22. The text-to-speech synthesizer of claim 21, wherein the representation of the duration associated with each phone in the sequence of phones in the described segment is adjusted to provide another representation of the duration associated with each phone in the sequence of phones in the described segment.

15. A text-to-speech synthesizer having a device for generating segment durations in a text-to-speech system, for input text that generates a linguistic description of speech to be uttered including at least one segment description, the device comprising:

A) a linguistic description-based information preprocessor, operably coupled to receive the linguistic description of speech to be uttered, for generating a linguistic description-based information vector for each segment description in the linguistic description of the input text, wherein the information vector includes a description of a sequence of segments surrounding a described segment and descriptive information for a context associated with the described segment; and

B) a pretrained neural network having feedforward neural network elements, wherein the training data for the pretrained neural network has been generated by recording natural speech, partitioning the speech data into segments associated with identified phones, and marking at least one of syntactical, intonational, and stress information, and wherein the pretrained neural network is operably coupled to the linguistic information preprocessor, for generating a representation of a duration associated with each phone in a sequence of phones in the described segment by the pretrained neural network.

16. The text-to-speech synthesizer of claim 15 wherein:

A) the speech is described as a sequence of phone identifications;

B) the segments for which duration is being generated are segments of speech expressing predetermined phones in the sequence of phone identifications; and

C) segment descriptions include the phone identifications.

17. The text-to-speech synthesizer of claim 16 wherein the information vector for each segment description includes at least one of:

A) articulatory features associated with each phone in the sequence of phones;

B) locations of syllable, word and other syntactic and intonational boundaries;

C) syllable strength information;

D) descriptive information of a word type; and

E) rule firing information.

18. The text-to-speech synthesizer of claim 15 wherein the representation of the duration is a logarithm of the duration.

19. The text-to-speech synthesizer of claim 15 wherein the pretrained neural network has been trained using back-propagation of errors.