Method for generating an animated three-dimensional video head

• US 7,050,655 B2
• Filed: 08/13/2001
• Issued: 05/23/2006
• Est. Priority Date: 11/06/1998
• Status: Expired due to Term

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Nevengineering Incorporated
• Inventors: Neven, Hartmut, Lutter, Gregory Patrick Lane, Stewartson, James, Westwood, David, Ho, Randall, Nocera, Luciano Pasquale Agostino, Buddemeier, Ulrich F.
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): Kassa, Yosef
• Application Number:
  US09/929,295
• Publication Number:
  US 20030031381A1
• Time in Patent Office: 1,744 Days
• Field of Search: 382/115, 382/117, 382/118, 382/154, 382/285, 382/294, 382/296, 382/282, 382/284, 382/287, 382/385, 345/419, 345/424, 345/473, 348/42, 348/47, 348/50
• US Class Current: 382/285
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

The invention relates to a technique for generating an animated three-dimensional video head based on sensed locations of facial features and texture mapping of corresponding two dimensional video image frames onto a shaped head mesh generated using the sensed locations.

Claims

1. A method for generating a three-dimensional animated video head, comprising steps for:

capturing a sequence of two-dimensional video image frames of the face of an actor;

for each two-dimensional video image frame;

sensing the locations of the actor'"'"'s facial features at predetermined node locations in the two-dimensional video image frame, wherein each node location is associated with a particular facial feature;

driving control points on a three-dimensional head mesh based on the sensed node locations in the two-dimensional video image frame to generate a shaped three-dimensional head mesh;

warping the two-dimensional video image frame used to generate the sensed node locations for projection onto the shaped head mesh; and

texture mapping the warped two-dimensional video image frame onto the shaped head mesh to generate a three-dimensional frame head associated with the respective two-dimensional video image frame; and

animating the three-dimensional video head by displaying a sequence of the three-dimensional frame heads associated with the sequence of two-dimensional video image frames.

2. Method for generating a three-dimensional animated video head as defined in claim 1, wherein the step of sensing the locations of the actor'"'"'s facial features at predetermined node locations in the two-dimensional video image frame is performed using a transformed facial image frame generated based on wavelet transformations.

3. Method for generating a three-dimensional animated video head as defined in claim 1, wherein the step of sensing the locations of the actor'"'"'s facial features at predetermined node locations in the two-dimensional video image frame is performed using a transformed facial image frame generated based on Gabor wavelet transformations.

4. Method for generating a three-dimensional animated video head as defined in claim 1, wherein each two-dimensional image frame, in the sequence of two-dimensional video image frames, has only one view of the actor'"'"'s face.