Method and system using a data-driven model for monocular face tracking

• US 20030012408A1
• Filed: 05/09/2001
• Published: 01/16/2003
• Est. Priority Date: 05/09/2001
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Grzeszczuk, Radek, Bouguet, Jean-Yves, Gokturk, Salih
• Granted Patent: US 9,400,921 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A method and system using a data-driven model for monocular face tracking are disclosed, which provide a versatile system for tracking three-dimensional (3D) images, e.g., a face, using a single camera. For one method, stereo data based on input image sequences is obtained. A 3D model is built using the obtained stereo data. A monocular image sequence is tracked using the built 3D model. Principal Component Analysis (PCA) can be applied to the stereo data to learn, e.g., possible facial deformations, and to build a data-driven 3D model (“3D face model”). The 3D face model can be used to approximate a generic shape (e.g., facial pose) as a linear combination of shape basis vectors based on the PCA analysis.

Claims

1. A method for image processing comprising:

obtaining stereo data based on input image sequences;

building a three-dimensional (3D) model using the obtained stereo data; and

tracking a monocular image sequence using the built 3D model.

2. The method of claim 1, wherein the obtaining of the stereo data includes obtaining stereo data based on input image sequences of varying facial expressions.

3. The method of claim 1, wherein the building of the 3D model includes processing the obtained stereo data using a Principal Component Analysis (PCA).

4. The method of claim 3, wherein the processed stereo data using PCA allows the 3D model to approximate a generic shape as a linear combination of shape basis vectors.

5. The method of claim 1, wherein the tracking of the monocular image sequence includes tracking of a monocular image sequence of facial deformations using the built 3D model.

6. A computing system comprising:

an input unit to obtain stereo data based on input image sequences; and

a processing unit to build a three-dimensional (3D) model using the obtained stereo data and to track a monocular image sequence using the built 3D model.

7. The computing system of claim 6, wherein the input unit is to obtain the stereo data based on input image sequences of varying facial expressions.

8. The computing system of claim 6, wherein the processor is to process the obtained stereo data using a Principal Component Analysis (PCA).

9. The computing system of claim 6, wherein the processor is to approximate a generic shape as a linear combination of shape base vectors based on the PCA processed stereo data.

10. The computing system of claim 6, wherein the processor is to track a monocular image sequence of facial deformations using the built 3D model.

11. A machine-readable medium providing instructions, which if executed by a processor, causes the processor to perform an operation comprising:

obtaining stereo data based on input image sequences;

building a three-dimensional (3D) model using the obtained stereo data; and

tracking a monocular image sequence using the built 3D model.

12. The machine-readable medium of claim 11, further providing instructions, which if executed by the processor, causes the processor to perform an operation comprising:

obtaining stereo data based on input image sequences of varying facial expressions.

13. The machine-readable medium of claim 11, further providing instructions, which if executed by the processor, causes the processor to perform an operation comprising:

processing the obtained stereo data using a Principal Component Analysis (PCA).

14. The machine-readable medium of claim 11, further providing instructions, which if executed by the processor, causes the processor to perform an operation comprising:

approximating a generic shape as a linear combination of shape basis vectors based on the processed stereo data using PCA.

15. The machine-readable medium of claim 11, further providing instructions, which if executed by the processor, causes the processor to perform an operation comprising:

tracking of a monocular image sequence of facial deformations using the built 3D model.