Coarticulation method for audio-visual text-to-speech synthesis

• US 7,392,190 B1
• Filed: 08/24/2006
• Issued: 06/24/2008
• Est. Priority Date: 11/07/1997
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Cosatto, Eric, Graf, Hans Peter, Schroeter, Juergen
• Primary Examiner(s): ABEBE, DANIEL DEMELASH
• Application Number:
  US11/466,806
• Time in Patent Office: 670 Days
• Field of Search: 704/260, 704/251, 704/258, 345/473
• US Class Current: 704/260
• CPC Class Codes: G06T 13/205 driven by audio data
• CPC Class Codes: G10L 2021/105 Synthesis of the lips movements from speech, e.g. for talking heads
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence

Abstract

A method for generating animated sequences of talking heads in text-to-speech applications wherein a processor samples a plurality of frames comprising image samples. The processor reads first data comprising one or more parameters associated with noise-producing orifice images of sequences of at least three concatenated phonemes which correspond to an input stimulus. The processor reads, based on the first data, second data comprising images of a noise-producing entity. The processor generates an animated sequence of the noise-producing entity.

Claims

1. A computing device for generating a noise-producing entity, comprising:

a module configured to read first data comprising one or more parameters associated with noise-producing orifice images of sequences of at least three concatenated phonemes which correspond to an input stimulus;

a module configured to read, based on the first data, corresponding second data comprising images of a noise-producing entity; and

a module configured to generate, using the second data, an animated sequence of the noise-producing entity tracking the input stimulus.

2. The computing device of claim 1, further comprising:

a module configured to read acoustic data associated with the second data;

a module configured to convert the acoustic data into sound; and

a module configured to output the sound synchronously with the animated sequence of the noise-producing entity.

3. The computing device of claim 1, wherein the first data comprises one or more equations characterizing noise-producing orifice shapes.

4. The computing device of claim 3, wherein the first data comprises one or more equations characterizing noise-producing orifice shapes.

5. The computing device of claim 2, wherein the module configured to convert further uses a data-to-sound converter.

6. The computing device of claim 2, wherein the first data comprises segments of sampled images of a noise-producing subject.

7. The computing device of claim 2, wherein the second data comprises parameters associated with a noise-producing orifice degree of opening.

8. The computing device of claim 6, wherein the first data comprises animation data, and the second data comprises coarticulation data.

9. The computing device of claim 6, wherein the module configured to generate further overlays the segments onto a common interface to create frames comprising the animation sequence.

10. A method of generating an animated entity, the method comprising:

reading first data associated with noise-producing orifice images of sequences of at least three concatenated phonemes;

reading second data comprising images associated with an animated entity; and

generating the animated entity based on the first data and the second data.

11. The method of claim 10, wherein the first data corresponds to an input stimulus and the animated entity is generated to track the input stimulus.

12. The method of claim 10, wherein the animated entity is a noise producing entity.

13. The method of claim 10, further comprising:

reading acoustic data associated with the sound data;

converting the acoustic data into second; and

outputting the sound synchronously with the generation of the animated entity.

14. A computer-readable medium storing instructions for controlling a computing device to generate an animated entity, the instructions comprising:

reading first data associated with noise-producing orifice images of sequences of at least three concatenated phonemes;

reading second data comprising images associated with animated entity; and

generating the animated entity based on the first data and the second data.

15. The computer-readable medium of claims 14, wherein the first data corresponds to an input stimulus and the animated entity is generated to track the input stimulus.

16. The computer-readable medium of claim 14, wherein the animated entity is a noise producing entity.

17. The computer-readable medium of claim 14, the instructions further comprising:

reading acoustic data associated with the second data;

converting the acoustic data into sound; and

outputting the sound synchronously with the generation of the animated entity.