Method and system using a data-driven model for monocular face tracking

• US 9,400,921 B2
• Filed: 05/09/2001
• Issued: 07/26/2016
• Est. Priority Date: 05/09/2001
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Bouguet, Jean-Yves, Grzeszczuk, Radek, Gokturk, Salih
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US09/852,398
• Publication Number:
  US 20030012408A1
• Time in Patent Office: 5,557 Days
• Field of Search: 382/103, 382/118, 382/154, 382/203, 382/243, 382/255, 345/419, 345/420, 345/427, 345/473, 345/474, 348/169, 348/170, 356/12, 356/14
• US Class Current: 1/1
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A method and system using a data-driven model for monocular face tracking are disclosed, which provide a versatile system for tracking three-dimensional (3D) images, e.g., a face, using a single camera. For one method, stereo data based on input image sequences is obtained. A 3D model is built using the obtained stereo data. A monocular image sequence is tracked using the built 3D model. Principal Component Analysis (PCA) can be applied to the stereo data to learn, e.g., possible facial deformations, and to build a data-driven 3D model (“3D face model”). The 3D face model can be used to approximate a generic shape (e.g., facial pose) as a linear combination of shape basis vectors based on the PCA analysis.

Claims

1. A method for image processing comprising:

obtaining stereo data based on input image sequences from of varying facial expressions;

building a three-dimensional (3D) model using the obtained stereo data to obtain principal shape vectors; and

tracking a second input image sequence using the 3D model to approximate a linear combination of the principal shape vectors of a facial expression in the second input image sequence, wherein the second input image sequence is a monocular image sequence.

2. The method of claim 1, wherein the building of the 3D model includes processing the obtained stereo data using a Principal Component Analysis (PCA).

3. The method of claim 2, wherein the processed stereo data using PCA allows the 3D model to approximate a generic shape as the linear combination of the shape basis vectors.

4. The method of claim 1, wherein the tracking of the monocular image sequence includes tracking of a monocular image sequence of facial deformations using the built 3D model.

5. A computing system comprising:

an input unit to stereo data based on input image sequences from of varying facial expressions; and

a processing unit to build a three-dimensional (3D) model using the obtained stereo data to approximate a generic shape as a linear combination of shape basis vectors and track a second input image sequence using the 3D model to approximate a linear combination of the principal shape vectors of a facial expression in the second input image sequence, wherein the second input image sequence is a monocular image sequence.

6. The computing system of claim 5, wherein the processing unit is to process the obtained stereo data using a Principal Component Analysis (PCA).

7. The computing system of claim 6, wherein the processed stereo data using PCA allows the 3D model to approximate a generic shape as the linear combination of the shape basis vectors.

8. The computing system of claim 5, wherein the processing unit is to track a monocular image sequence of facial deformations using the built 3D model.

9. A non-transitory machine-readable medium providing instructions, which if executed by a processor, causes the processor to perform an operation comprising:

obtaining stereo data based on input image sequences from of varying facial expressions;

building a three-dimensional (3D) model using the obtained stereo data to approximate a generic shape as a linear combination of shape basis vectors; and

tracking a second input image sequence using the 3D model to approximate a linear combination of the principal shape vectors of a facial expression in the second input image sequence, wherein the second input image sequence is a monocular image sequence.

10. The machine-readable medium of claim 9, further providing instructions, which if executed by the processor, causes the processor to perform an operation comprising:

processing the obtained stereo data using a Principal Component Analysis (PCA).

11. The machine-readable medium of claim 10, further providing instructions, which if executed by the processor, causes the processor to perform an operation comprising:

approximate a generic shape as the linear combination of the shape basis vectors based on the processed stereo data using PCA.

12. The machine-readable medium of claim 9, further providing instructions, which if executed by the processor, causes the processor to perform an operation comprising:

tracking of a monocular image sequence of facial deformations using the built 3D model.