Device and method for prosody generation at visual synthesis
First Claim
1. A device for prosody generation and visual synthesis, comprising:
- capturing means for capturing sounds and face movement patterns of a physiognomy of a first face during a speech, wherein the face movement patterns include a position and displacement of selected points on the first face;
storing means for storing the captured sounds and face movement patterns;
reproducing means for reproducing the stored sounds and face movements patterns of the first face on a second face; and
amplifying means for amplifying the face movement patterns reproduced on the second face, based on stresses of the speech of the first face.
7 Assignments
0 Petitions
Accused Products
Abstract
A device for prosody generation at visual synthesis. A number of half-syllables are stored together with registered movement patterns in a face. When synthesizing speech, a number of half-syllables are put together into words and sentences. The words and sentences are given a stress and pattern of intonation corresponding to the intended language. In the face, a number of points and their movement patterns are further registered. In connection with the generation of words and sentences, the movement patterns of the different points are amplified depending on a given stress and sentence intonation. The given movement patterns are after that applied to a model, which is applied to a real face at which a life-like animation is obtained, at for instance a translation of a person'"'"'s speech in a first language to a second language.
-
Citations
15 Claims
-
1. A device for prosody generation and visual synthesis, comprising:
-
capturing means for capturing sounds and face movement patterns of a physiognomy of a first face during a speech, wherein the face movement patterns include a position and displacement of selected points on the first face;
storing means for storing the captured sounds and face movement patterns;
reproducing means for reproducing the stored sounds and face movements patterns of the first face on a second face; and
amplifying means for amplifying the face movement patterns reproduced on the second face, based on stresses of the speech of the first face. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A method for prosody generation and visual synthesis using selected polygons on a second face, comprising:
-
capturing sounds and face movement patterns corresponding to polyphones of a first face;
recording speaking stresses of polyphones;
amplifying the captured face movement patterns based on the recorded stresses of the polyphones;
selecting points in the selected polygons on the second face;
reproducing captured sounds and amplified face movement patterns of the first face onto the second face, wherein the points in the selected polygons are allocated a weighting which is influenced by the speaking stresses of the polyphones; and
animating the second face by a movement of selected polygons according to the captured face movement patterns of the first face and reproducing the captured sounds so that a three-dimensional picture is created on the second face. - View Dependent Claims (11, 12, 13, 14, 15)
generating sounds with polyphones of neutral pronunciation; and
registering simultaneously the sounds of polyphones of neutral pronunciation with the corresponding face movement patterns.
-
-
13. A method according to claim 12, comprising recording the face movement patterns of a group of persons.
-
14. A method according to claim 13, wherein the recording has a group of persons including men, women, and children.
-
15. A method according to claim 10 or 11, further comprising producing sounds for polyphones from a text.
Specification