Animating graphical objects using input video

• US 7,830,384 B1
• Filed: 04/19/2006
• Issued: 11/09/2010
• Est. Priority Date: 04/27/2005
• Status: Active Grant

• Current Assignee: Image Metrics Ltd. (Image Metrics, Inc.)
• Original Assignee: Image Metrics Ltd. (Image Metrics, Inc.)
• Inventors: Edwards, Gareth, Walker, Kevin
• Primary Examiner(s): HARRISON, CHANTE E
• Application Number:
  US11/406,890
• Time in Patent Office: 1,665 Days
• Field of Search: 345/473, 345/475
• US Class Current: 345/473
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings

Abstract

A method and system for driving a graphical object based on a performance of an actor in a video stream. A plurality of key frames are selected in a calibration video stream, allowing animation control values to be defined, in accordance with artistic intent, for each of a set of control variables, corresponding to each of the key frames. Features of the actor in the calibration video stream are parameterized so as to obtain a vector of values of parameters for each frame of the calibration video stream, and, then, an array of distance vectors is derived, characterizing a distance between each pair of vectors of values of parameters among the video stream frames. The space of distance vectors is mapped into the set of control variables according to a mapping operation which is then applied to distance vectors derived from an actual performance video stream to obtain a time sequence of animation control values. Finally, an animation is created on the basis of applying the sequence of animation control values to the graphical object.

Claims

1. A method for driving a graphical object in computer-generated material in film, television and computer games based on a performance of an actor in an actual performance video stream, the method comprising:

a. selecting a plurality of key calibration frames in a calibration video stream to serve as a basis set of frames specific to the actor and the graphical object;

b. defining animation control values, in accordance with artistic intent, for each of a set of control variables, corresponding to each of the key calibration frames;

c. parameterizing features of the actor in the calibration video stream so as to obtain a vector of values of parameters for each frame of the calibration video stream;

d. deriving, for each key calibration frame, a vector containing a distance to all of the key calibration frames;

e. for each frame in the video stream, deriving a vector containing, as elements, distances to all of the basis set of key calibration frames;

f. mapping the space of distance vectors into the set of control variables according to a mapping operation;

g. applying the mapping operation to distance vectors derived from the actual performance video stream to obtain a time sequence of animation control values; and

h. creating the graphical object embodying a particular visual depiction of the actor based on applying the sequence of animation control values to the graphical object.

2. A method in accordance with claim 1, wherein the mapping operation is a matrix of regression coefficients.

3. A method in accordance with claim 1, wherein the actual performance video stream is identical to the calibration video stream.