Method of image processing using three facial feature points in three-dimensional head motion tracking

• US 6,580,810 B1
• Filed: 06/10/1999
• Issued: 06/17/2003
• Est. Priority Date: 02/26/1999
• Status: Expired due to Term

• Current Assignee: Cyberlink Corporation
• Original Assignee: Cyberlink Corporation
• Inventors: Ouhyoung, Ming, Wu, Fu-Che, Yang, Tzong-Jer
• Primary Examiner(s): Au, Amelia M.
• Assistant Examiner(s): MILLER, MARTIN E
• Application Number:
  US09/329,671
• Time in Patent Office: 1,468 Days
• Field of Search: 382/103, 382/106, 382/107, 382/117, 382/118, 382/285, 382/295, 382/296, 382/154, 348/14.1
• US Class Current: 382/103
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame
• CPC Class Codes: G06T 9/004 Predictors, e.g. intraframe, interframe coding
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A method of image processing in three-dimensional (3-D) head motion tracking is disclosed. The method includes: providing a user'"'"'s source image to a first processing device; capturing the user'"'"'s first image and providing it to a second processing device; selecting three facial feature points of the first image from the second processing device to form a 3-D feature triangle; capturing user'"'"'s consecutive video frames and providing them to the second processing device when the user proceeds with head motions; tracking the three facial feature points corresponding to the consecutive video frames to form a series of actual 2-D feature triangle; rotating and translating the 3-D feature triangle freely to form a plurality of geometric transformations, selecting one of the geometric transformations with acceptable error between the two consecutive 2-D feature triangles, repeating the step until the last frame of the consecutive video frames and geometric transformations corresponding to various consecutive video frames are formed; and providing the geometric transformations to the first processing device to generate a head motion corresponding to the user'"'"'s source image.

Claims

1. A method of image processing three-dimensional (3-D) head motion with three facial feature points comprising the following steps:

(a) providing a user'"'"'s source image to a first processing device;

(b) capturing said user'"'"'s first image and providing it to a second processing device;

(c) selecting three facial feature points of said first image from said second processing device to form a 3-D feature triangle;

(d) capturing user'"'"'s consecutive video frames and providing them to the second processing device when said user proceeds with head motions;

(e) tracking said three facial feature points corresponding to said consecutive video frames to form a series of actual 2-D feature triangle;

(f) rotating and translating said 3-D feature triangle freely to form a plurality of geometric transformations;

(g) selecting one of the geometric transformations with acceptable error between the two consecutive 2-D feature triangles;

(h) repeating the step (f) and (g) until the last frame of said consecutive video frames and geometric transformations corresponding to various consecutive video frames are formed; and

(i) providing said geometric transformations to said first processing device to generate a head motion corresponding to said user'"'"'s source image.

2. The method of claim 1, wherein said source image is an existing photograph.

3. The method of claim 1, wherein in said Step (b) a camera captures user'"'"'s first image and provides it to a second processing device.

4. The method of claim 1, wherein in said Step (c) a 3-D feature triangle is formed by selecting with a positioning device three facial feature points of the first image from said second processing device.

5. The method of claim 1, wherein in said Step (c) a 3-D feature triangle is formed by selecting with a mouse three facial feature points of the first image from said second processing device.

6. The method of claim 1, wherein said geometric transformations include the coordinates and surface normal of the three points of the 3-D feature triangle.

7. The method of claim 1, wherein said three feature points are selected points of the face considered largely as a rigid body.

8. The method of claim 1, wherein said three feature points are selected points of the lateral canthus of the eyes and the nose on the face.

9. The method of claim 1, wherein said three feature points are selected points of the ear-tips and the lower chin on the face.

10. The method of claim 1, wherein in said step (f), said 3-D feature triangle is first rotated and then translated.

11. The method of claim 1, wherein a pivot of said head motion is a point in center of an upper neck of said user'"'"'s source image.

12. The method of claim 1, wherein in said step (i), said geometric transformations are transmitted to said first processing device through a transmission medium for generating head motion corresponding to said user'"'"'s source image.

13. The method of claim 12, wherein in said step (i), said transmission medium includes network transmission medium.

14. A method of image processing three-dimensional (3-D) head motion with three facial feature points comprising the following steps:

(a) providing a user'"'"'s source image to a first processing device;

(b) capturing said user'"'"'s first image and providing it to a second processing device;

(c) selecting three facial feature points of said first image from said second processing device;

(d) pre-building a 3-D feature triangle in accordance with the three facial feature points;

(e) capturing user'"'"'s consecutive video frames and providing them to the second processing device when said user proceeds with head motions;

(f) tracking said three facial feature points corresponding to said consecutive video frames to form a series of actual 2-D feature triangle;

(g) rotating and translating said pre-built 3-D feature triangle freely with steepest-descent iterative method to form a plurality of geometric transformations;

(h) selecting one of the geometric transformations with acceptable error between said two consecutive actual 2-D feature triangles to form an estimated 3-D feature triangle;

(i) replacing said pre-built 3-D feature triangle with said estimated 3-D feature triangle and repeating sequentially step (g) and (h) until the last frame of said consecutive video frames; and

(j) providing said geometric transformations to said first processing device to generate a head motion corresponding to said user'"'"'s source image.