Speech signal distribution system providing supplemental parameter associated data

• US 5,943,648 A
• Filed: 04/25/1996
• Issued: 08/24/1999
• Est. Priority Date: 04/25/1996
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Lernout & Hauspie Speech Products NV (Intel Corporation)
• Inventors: Tel, Michael P.
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Opsasnick, Michael N.
• Application Number:
  US08/638,061
• Time in Patent Office: 1,216 Days
• Field of Search: 395/2.79, 395/2.85, 395/2.44
• US Class Current: 704/270.1
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination

Abstract

A speech signal distribution system includes a transmitting subsystem and one or more receiving subsystems. The transmitting subsystem has a text to speech converter for converting text into a data stream of formant parameters. A supplemental parameter generator inserts into the data stream supplemental data, including linguistic boundary data indicating which parameters in the stream of formant parameters are associated with predefined linguistic boundaries in the text. In one preferred embodiment, the boundary data indicates which formant parameters in the data stream are associated with sentence boundaries. In addition, the supplemental parameter generator optionally inserts the text, lip position data corresponding to phonemes in the text, and voice setting data into the data stream. The resulting data stream is compressed and transmitted to the receiving subsystems. The receiving subsystem receives the transmitted compressed data stream, decompresses the data stream to regenerate the full data stream, and splits off the supplemental data. The formant data is buffered until boundary data is received indicating that a full sentence, or other linguistic unit, has been received. Then the formant data is processed by an audio signal generator that converts the formant parameters into an audio speech signal in accordance with a vocal tract model. Voice settings in the supplemental data are passed to the audio signal generator, which modifies audio signal generation accordingly. Lip position data in the supplemental data may be processed by an animation program to generate animated pictures of a person speaking.

Claims

1. A speech signal distribution system comprising:

a text to speech parameter converter for converting text containing sentences into a data stream, said data stream including a stream of speech signal parameters representing spoken text and lacking phrase-level and sentence-level prosodic content, being suitable for driving an audio signal generator that converts said stream of parameters into an audio speech signal in accordance with a vocal tract model;

a supplemental parameter generator in communication with the text to speech parameter converter, such generator inserting into said data stream additional data, representative of linguistic boundaries, that indicate which parameters in said stream of parameters are associated with predefined boundaries of at least one of phrases and sentences in said text; and

a transmitter for transmitting said, data stream.

2. The speech signal distribution system of claim 1, further including:

a receiving subsystem that receives said transmitted data stream, said receiving subsystem including;

said audio signal generator that converts said stream of parameters into an audio speech signal in accordance with said vocal tract model; and

a sentence level data stream buffer for storing said received data stream in a buffer until said received data stream includes boundary data indicating a sentence boundary, and for then enabling said stored data stream up to said sentence boundary to be processed by said audio signal generator.

3. The speech signal distribution system of claim 1,

said text including a sequence of words;

said supplemental parameter generator further inserting into said data stream text data representing at least a subset of the words in said text, wherein said text data is inserted at positions in said data stream coinciding with the corresponding parameters in said stream of parameters.

4. The speech signal distribution system of claim 3, further including

a receiving subsystem that receives said transmitted data stream, said receiving subsystem including:

said audio signal generator that converts said stream of parameters into an audio speech signal in accordance with said vocal tract model; and

a video signal generator for generating a video image that includes images corresponding to at least a subset of said text data in said received data stream.

5. The speech signal distribution system of claim 1,

said supplemental parameter generator further inserting into said data stream voice setting data representing parameters for controlling audio speech generation from said stream of parameters by said audio signal generator.

6. The speech signal distribution system of claim 5 further including

a receiving subsystem that receives said transmitted data stream, said receiving subsystem including:

said audio signal generator that converts said stream of parameters into an audio speech signal in accordance with said vocal tract model and in accordance with said voice setting data in said received data stream.

20. The speech distribution system of claim 1,

said supplemental parameter generator further inserting into said data stream supplemental linguistic processing data representing indications of at least one of surprise, emphasis and mood, said supplemental data representing parameters for controlling audio speech generation from said stream of parameters by said audio signal generator.

21. The speech distribution system of claim 20, further including

a receiving subsystem that receives said transmitted data stream, said receiving subsystem including:

said audio signal generator that converts said stream of parameters into an audio speech signal in accordance with said vocal tract model and in accordance with said supplemental linguistic processing data representing indications of at least one of surprise, emphasis and mood in said received data stream.

22. The speech distribution system of claim 20,

said supplemental parameter generator further inserting into said data stream supplemental linguistic processing data representing indications of at least one of surprise, emphasis and mood, said supplemental data representing parameters for controlling video image generation from said stream of parameters by a video image generator.

23. The speech distribution system of claim 22, further including

a receiving subsystem that receives said transmitted data stream, said receiving subsystem including:

said video image generator that converts said stream of parameters into a video image signal in accordance with said supplemental linguistic processing data representing indications of at least one of surprise, emphasis and mood in said received data stream.

7. A speech signal distribution system, comprising:

a text to speech parameter converter for converting text containing sentences into a data stream, said data stream including a stream of parameters suitable for driving an audio signal generator that converts said stream of parameters into an audio speech signal in accordance with a vocal tract model;

said text including a sequence of words;

a supplemental parameter generator for inserting into said data stream text data representing at least a subset of the words in said text, wherein said text data is inserted at positions in said data stream coinciding with the corresponding parameters in said stream of parameters; and

a transmitter for transmitting said data stream.

8. The speech signal distribution system of claim 7, further including

a receiving subsystem that receives said transmitted data stream, said receiving subsystem including:

said audio signal generator that converts said stream of parameters into an audio speech signal in accordance with said vocal tract model; and

a video signal generator for generating a video image that includes images corresponding to at least a subset of said text data in said received data stream.

9. A speech signal distribution method comprising the steps of:

a. converting text containing sentences into a data stream, said data stream including a stream of speech signal parameters representing spoken text and lacking phrase-level and sentence-level prosodic content, being suitable for driving an audio signal generator that converts said stream of parameters into an audio speech signal in accordance with a vocal tract model;

b. insertng into said data stream, established by step (a), additional data, representative of linguistic boundaries, that indicate which parameters in said stream of parameters are associated with predefined boundaries of at least one of phrases and sentences in said text; and

c. transmitting said data stream.

10. The speech signal distribution method of claim 9, further including at a receiving subsystem:

receiving said transmitted data stream;

converting said stream of parameters into an audio speech signal in accordance with said vocal tract model; and

storing said received data stream in a buffer until said received data stream includes boundary data indicating a predefined linguistic boundary, and for then enabling said stored data stream up to said predefined linguistic boundary to be converted into an audio signal.

11. The speech signal distribution method of claim 9,

said text including a sequence of words;

said inserting step including inserting into said data stream text data representing at least a subset of the words in said text, wherein said text data is inserted at positions in said data stream coinciding with the corresponding parameters in said stream of parameters.

12. The speech signal distribution method of claim 11, further including at a receiving subsystem:

receiving said transmitted data stream;

converting said stream of parameters into an audio speech signal in accordance with said vocal tract model; and

generating a video image that includes images corresponding to at least a subset of said text data in said received data stream.

13. The speech signal distribution method of claim 9,

said inserting step including inserting into said data stream voice setting data representing parameters for controlling audio speech generation from said stream of parameters.

14. The speech signal distribution method of claim 13, further including at a receiving subsystem:

receiving said transmitted data stream;

converting said stream of parameters into an audio speech signal in accordance with said vocal tract model; and

controlling the conversion of said audio speech signal in accordance with said voice setting data in said received data stream.

15. A speech signal distribution method, comprising the steps of:

converting text containing sentences into a data stream, said data stream including a stream of parameters suitable for driving an audio signal generator that converts said stream of parameters into an audio speech signal in accordance with a vocal tract model;

said text including a sequence of words;

inserting into said data stream text data representing at least a subset of the words in said text, wherein said text data is inserted at positions in said data stream coinciding with the corresponding parameters in said stream of parameters; and

transmitting said data stream.

16. The speech signal distribution method of claim 15, further including at a receiving subsystem:

receiving said transmitted data stream;

converting said stream of parameters into an audio speech signal in accordance with said vocal tract model; and

generating a video image that includes images corresponding to at least a subset of said text data in said received data stream.

17. A speech signal distribution system comprising:

a receiving subsystem that receives a data stream transmitted by a remotely located subsystem, said received data stream including (i) a stream of speech signal parameters representing spoken text and lacking phrase-level and sentence-level prosodic content, and (ii) additional data, representative of linguistic boundaries, that indicate which parameters in said stream of speech signal parameters are associated with predefined boundaries of at least one of phrases and sentences in said text;

said receiving subsystem including;

an audio signal generator that converts said stream of speech signal parameters into an audio speech signal in accordance with a vocal tract model; and

a data stream buffer for storing said received data stream in a buffer until said received data stream includes boundary data indicating a linguistic boundary of at least one of phrases and sentences, and for then enabling said stored data stream up to said linguistic boundary to be processed by said audio signal generator.

18. The speech generation system of claim 17, said received data stream further including text data representing at least a subset of the words in said text, wherein said text data is inserted at positions in said data stream coinciding with the corresponding parameters in said stream of speech signal parameters;

said receiving subsystem further including a video signal generator for generating a video image that includes images corresponding to at least a subset of said text data in said received data stream.

19. The speech generation system of claim 17, said received data stream further including voice setting data representing parameters for controlling audio speech generation from said stream of speech signal parameters;

said audio signal generator converting said stream of parameters into an audio speech signal in accordance with said vocal tract model and in accordance with said voice setting data in said received data stream.