Method for optimizing off-line facial feature tracking

• US 20030031344A1
• Filed: 08/13/2001
• Published: 02/13/2003
• Est. Priority Date: 08/13/2001
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Nevengineering Incorporated
• Inventors: Maurer, Thomas, Poehlker, Bjoern, Neven, Hartmut
• Granted Patent: US 6,834,115 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

The present invention relates to a technique for optimizing off-line facial feature tracking. Facial features in a sequence of image frames are automatically tracked while a visual indication is presented of the plurality of tracking node locations on the respective image frames. The sequence of image frames may be manually paused at a particular image frame in the sequence of image frames if the visual indication of the tracking node locations indicates that at least one location of a tracking node for a respective facial feature is not adequately tracking the respective facial feature. The location of the tracking node may be reinitialized by manually placing the tracking node location at a position on the particular image frame in the monitor window that corresponds to the respective facial feature. Automatic tracking of the facial feature may be continued based on the reinitialized tracking node location.

Claims

1. Method for optimizing off-line facial feature tracking, comprising the steps for:

providing a monitor window that has a visual indication of a plurality of tracking node locations with respect to facial features in a sequence of image frames, the monitor window having a control for pausing at an image frame in the sequence of image frames;

automatically tracking the facial features in the sequence of image frames while presenting the visual indication of the plurality of tracking node locations on the respective image frames;

manually pausing the sequence of image frames at a particular image frame in the sequence of image frames if the visual indication of the tracking node locations indicates that at least one location of a tracking node for a respective facial feature is not adequately tracking the respective facial feature;

reinitializing the at least one location of the tracking node by manually placing the tracking node location at a position on the particular image frame in the monitor window that corresponds to the respective facial feature; and

continuing automatic tracking of the facial feature based on the reinitialized at least one tracking node location.

2. Method for optimizing off-line facial feature tracking as defined in claim 1, wherein the tracking of facial features in the sequence of facial image frames of the speaking actor is performed using bunch graph matching.

3. Method for optimizing off-line facial feature tracking as defined in claim 1, wherein the tracking of facial features in the sequence of facial image frames of the speaking actor is performed using transformed facial image frames generated based on wavelet transformations.

4. Method for optimizing off-line facial feature tracking as defined in claim 1, wherein the tracking of facial features in the sequence of facial image frames of the speaking actor is performed using transformed facial image frames generated based on Gabor wavelet transformations.

5. Apparatus for optimizing off-line facial feature tracking, comprising:

a monitor window means for providing a visual indication of a plurality of tracking node locations with respect to facial features in a sequence of image frames, the monitor window having a control for pausing at an image frame in the sequence of image frames;

means for automatic tracking of the facial features in the sequence of image frames while presenting the visual indication of the plurality of tracking node locations on the respective image frames;

means for manually pausing the sequence of image frames at a particular image frame in the sequence of image frames if the visual indication of the tracking node locations indicates that at least one location of a tracking node for a respective facial feature is not adequately tracking the respective facial feature;

means for reinitializing the at least one location of the tracking node by manually placing the tracking node location at a position on the particular image frame in the monitor window that corresponds to the respective facial feature; and

means for continuing automatic tracking of the facial feature based on the reinitialized at least one tracking node location.