Head motion estimation from four feature points

• US 20030063777A1
• Filed: 09/28/2001
• Published: 04/03/2003
• Est. Priority Date: 09/28/2001
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Trajkovic, Miroslav, Gutta, Srinivas, Philomin, Vasanth
• Granted Patent: US 7,027,618 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

Linear method for performing head motion estimation from facial feature data, the method comprising the steps of: obtaining first facial image and detecting a head in the first image; detecting position of four points P of said first facial image where P={p₁, p₂, p₃, p₄}, and p_k=(x_k, y_k); obtaining second facial image and detecting a head in the second image; detecting position of four points P′ of the second facial image where P′={p₁′, p₂′, p₃′, p₄′} and p_k′=(x_k′, y_k′); and, determining the motion of the head represented by a rotation matrix R and translation vector T using the points P and P′.

The head motion estimation is governed according to an equation:

P _i ′=RP _i +T,

where $R=\left[\begin{array}{c}{r}_1^T\\ r_2^T\\ r_3^T\end{array}\right]={\left[r_{\mathrm{ij}}\right]}_{3\times 3}$

and T=[T₁ T₂ T₃]^T represents camera rotation and translation respectively.

Claims

1. A linear method for performing head motion estimation from facial feature data, the method comprising the steps of:

obtaining first facial image and detecting a head in said first image;

detecting position of four points P of said first facial image where P={p₁, p₂, p₃, p₄}, and p_k=(x_k, y_k);

obtaining a second facial image and detecting a head in said second image;

detecting position of four points P′

of said second facial image where P′

={p₁′

, p₂′

, p₃′

, p₄′

} and p_k′

=(x_k′

, y_k′

)); and

,

determining the motion of the head represented by a rotation matrix R and translation vector T using said points P and P′

.

2. The linear method of claim 1, wherein said four points P of said first facial image and four points P′

of said second facial image include locations of outer corners of each eye and mouth of each respective first and second facial image.

3. The linear method of claim 1, wherein said head motion estimation is governed according to:

4. The linear method of claim 3, wherein said head motion estimation is governed according to said rotation matrix R, said method further comprising the steps of:

determining rotation matrix R that maps points P_k to F_k for characterizing a head pose, said points F₁, F₂, F₃, F₄ representing three-dimensional (3-D) coordinates of the respective four points of a reference, frontal view of said facial image, and P_k is the three-dimensional (3-D) coordinates of an arbitrary point where P_i=[X_i Y_i Z_i]^T, said mapping governed according to the relation;

R(P₂−

P₁)∝

[1 0 0]^T R(P₆−

P₅)∝

[0 1 0]^T

wherein P₅ and P₆ are midpoints of respective line segments connecting points P₁P₂ and P₃P₄ and, line segment connecting points P₁P₂ is orthogonal to a line segment connecting points P₅P₆, and ∝

indicates a proportionality factor.

5. The linear method of claim 4, wherein components r1, r2 and r3 are computed as:

r₂^T(P₂−

P₁)=0 r₃^T(P₂−

P₁)=0 r₁^T(P₆−

P₅)=0 r₃^T(P₆−

P₅)=0

6. The linear method of claim 5, wherein components r1, r2 and r3 are computed as:

r₃=(P₆−

P₅)×

(P₂−

P₁), r₂=r₃×

(P₂−

P₁) r₁=r₂×

r₃

7. The linear method of claim 4, wherein

8. The linear method of claim 7, further comprising the step of:

decomposing said rotation matrix R using Singular Value Decomposition (SVD) to obtain a form R=USV^T.

9. The linear method of claim 7, further comprising the step of computing a new rotation matrix according to R=UV^T.

10. A linear method for performing head motion estimation from facial feature data, the method comprising the steps of:

obtaining image position of four points P_k of a facial image;

determining a rotation matrix R that maps points P_k to F_k for characterizing a head pose, said points F₁, F₂, F₃, F₄ representing three-dimensional (3-D) coordinates of the respective four points of a reference, frontal view of said facial image, and P_k is the three-dimensional (3-D) coordinates of an arbitrary point where P_i=[X_i Y_i Z_i]^T, said mapping governed according to the relation;

R(P₂−

P₁)∝

[1 0 0]^T R(P₆−

P₅)∝

[0 1 0]^T

wherein P₅ and P₆ are midpoints of respective line segments connecting points P₁P₂ and P₃P₄ and, line segment connecting points P₁P₂ is orthogonal to a line segment connecting points P₅P₆, and ∝

indicates a proportionality factor.

11. The linear method of claim 10, wherein components r1, r2 and r3 are computed as:

r₂^T(P₂−

P₁)=0 r₃^T(P₂−

P₁)=0 r₁^T(P₆−

P₅)=0 r₃^T(P₆−

P₅)=0

12. The linear method of claim 11, wherein components r1, r2 and r3 are computed as:

r₃=(P₆−

P₅)×

(P₂−

P₁), r₂=r₃×

(P₂−

P₁) r₁=r₂×

r₃

13. The linear method of claim 12, wherein a motion of head points is represented according to P_i′

=RP_i+T where

14. The linear method of claim 13, wherein

15. The linear method of claim 14, further comprising the step of:

decomposing said rotation matrix R using Singular Value Decomposition (SVD) to obtain a form R=USV^T.

16. The linear method of claim 15, further comprising the step of computing a new rotation matrix according to R=UV^T.

17. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for performing head motion estimation from facial feature data, the method steps comprising:

obtaining first facial image and detecting a head in said first image;

detecting position of four points P of said first facial image where P={p₁, p₂, p₃, p₄}, and p_k=(x_k, y_k);

obtaining a second facial image and detecting a head in said second image;

detecting position of four points P′

of said second facial image where P′

={p₁′

, p₂′

, p₃′

, p₄′

} and p_k′

=(x_k′

, y_k′

)); and

,

determining the motion of the head represented by a rotation matrix R and translation vector T using said points P and P′

.

18. The program storage device readable by machine as claimed in claim 17, wherein said four points P of said first facial image and four points P′

of said second facial image include locations of outer corners of each eye and mouth of each respective first and second facial image.

19. The program storage device readable by machine as claimed in claim 17, wherein said head motion estimation is governed according to:

20. The program storage device readable by machine as claimed in claim 19, wherein said head pose estimation is governed according to said rotation matrix R, said method further comprising the steps of:

determining rotation matrix R that maps points P_k to F_k for characterizing a head pose, said points F₁, F₂, F₃, F₄ representing three-dimensional (3-D) coordinates of the respective four points of a reference, frontal view of said facial image, and P_k is the three-dimensional (3-D) coordinates of an arbitrary point where P_i=[X_i Y_i Z_i]^T, said mapping governed according to the relation;

R(P₂−

P₁)∝

[1 0 0]^T R(P₆−

P₅)∝

[0 1 0]^T

wherein P₅ and P₆ are midpoints of respective line segments connecting points P₁P₂ and P₃P₄ and, line segment connecting points P₁P₂ is orthogonal to a line segment connecting points P₅P₆, and vindicates a proportionality factor.