Apparatus and method for estimating a facial pose and a face recognition system using the method

• US 20070041644A1
• Filed: 06/20/2006
• Published: 02/22/2007
• Est. Priority Date: 08/17/2005
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Jung-bae, Kee, Seok-cheol
• Application Number:
  US11/455,705
• Publication Number:
  US 20070041644A1
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships

Abstract

An apparatus for estimating a facial pose. The apparatus includes a pre-processing module that provides feature points of a subject'"'"'s face of a received image, and a pose-estimation module that computes sizes of a left half plane and a right half plane of the face from the provided feature points, and a lateral rotation angle of the face from the computed sizes.

Claims

1. A facial-pose-estimation device comprising:

a pre-processing module that provides feature points of a face of a received image; and

a pose-estimation-module that computes sizes of a left half plane and a right half plane of the face from the provided feature points, and a lateral rotation angle of the face from the computed sizes.

2. The device of claim 1, wherein the pre-processing module includes a first pre-processing module that provides local feature points.

3. The device of claim 2, wherein the first pre-processing module includes:

a face-model database that stores training images referred to in a face recognition process;

a Gabor filter module that obtains first response variables by applying a set of Gabor wavelet filters to feature points of at least two stored training images, and a second response variable by applying the set of Gabor wavelet filters to the feature points of the face of the received image; and

a similarity-computing module that compensates for displacement of the feature points of the face of the received image to maximize a similarity between the first response variables and the second response variable, and provides the feature points.

4. The device of claim 3, wherein the feature points of the face of the received image are equal to the average feature points of the at least two stored training images.

5. The device of claim 2, wherein the pre-processing module further comprises a second pre-processing module that approximates the provided local feature points to global feature points.

6. The device of claim 5, wherein the second pre-processing module approximates the provided local feature points to the global feature points by extracting parameters of a face shape model via principal components analysis (PCA).

7. The device of claim 1, wherein the post estimation module includes:

a face-size-computing module that extracts representative feature points delineating boundaries of the face from the provided feature points and shapes the face into a 2-D figure using the extracted representative feature points, to thereby compute the sizes of a left half plane and a right half plane of the face; and

a rotation-computing module that computes a lateral rotation angle of the face from the calculated size ratio.

8. The device of claim 7, wherein the face-size-computing module shapes a cheek region into a square and a chin region into a quarter of cylinder.

9. The device of claim 7, wherein the calculated size ratio is represented by a function of the lateral rotation angle.

10. A method of estimating a facial pose, comprising:

(a) providing feature points of a face of a received image;

(b) computing sizes of a left half plane and a right half plane of the face from the provided feature points; and

(c) computing a lateral rotation angle of the face from the calculated size ratio.

11. The method of claim 10, wherein operation (a) includes providing local feature points.

12. The method of claim 11, wherein the providing local feature points includes:

(a) applying a set of Gabor wavelet filters to feature points of at least two stored training images referred to in a face recognition process, and obtaining first response variables;

(b) applying the set of Gabor wavelet filters to the feature points of the face of the received image;

(c) compensating for displacement of the feature points of the face of the received image to maximize a similarity between the first response variables and the second response variable; and

(d) providing the compensated feature points.

13. The method of claim 12, wherein the feature points of the face are equal to the average of the feature points of the at least two stored training images.

14. The method of claim 11 further comprising approximating the local feature points to global feature points.

15. The method of claim 14, wherein the approximating local feature points to global feature points includes approximating the provided local feature points to global feature points by extracting parameters of the face shape model via principal components analysis (PCA).

16. The method of claim 10, wherein operation (b) includes:

extracting representative feature points delineating boundaries of the face;

shaping the image of the face of the received image into a 2-D figure; and

computing sizes of a left half plane and a right half plane of the face of the 2-D figure.

17. The method of claim 16, wherein the shaping the image into the 2-D figure includes shaping a cheek region into a square figure and a chin region into a quarter of cylinder.

18. The method of claim 10, wherein the calculated size ratio is represented by the function of the lateral rotation angle used in operation (c).

19. A face recognition system comprising:

a face-image database that stores face images;

an image-providing module that provides an image including a face image of a subject that is being searched for;

a facial-pose-estimation module that computes a lateral rotation angle of a left half plane and a right half plane of the subject'"'"'s face from a size ratio thereof; and

an image-comparison module that rotates the face image of the subject by the computed rotation angle in the opposite direction, and searches for an image similar to the face image of the subject.

20. The system of claim 19 further comprising a display module that displays an image provided by the image providing-module or a face image found in the face-image database by the image-comparison module.

21. A face recognition system comprising:

a face-image database that stores face images;

an image-input module that receives the face images;

a facial-pose-estimation module that computes the lateral rotation angle of the left half plane and the right half plane of the subject'"'"'s face from the calculated size ratio; and

an image-comparison module that rotates the face image of the subject by the computed rotation angle in the opposite direction, and searches for an image similar to the face image of the subject.

22. The system of claim 21 further comprising an operation-execution module that executes an operation when a face image found in the face image database matches the face image provided by the image-comparison module.

23. A computer program product providing a program executing a method of estimating a facial pose, the method comprising:

providing feature points of a face of a received image;

computing sizes of a left half plane and a right half plane of the face from the provided feature points; and

computing a lateral rotation angle of the face from the calculated size ratio.

24. The computer program product of claim 23, wherein the providing includes providing local feature points, by:

applying a set of Gabor wavelet filters to feature points of at least two stored training images referred to in a face recognition process, and obtaining first response variables;

applying the set of Gabor wavelet filters to the feature points of the face of the received image; and

compensating for displacement of the feature points of the face of the received image to maximize a similarity between the first response variables and the second response variable.

25. The computer program product of claim 24, wherein the Gabor wavelet filter is convoluted with predetermined feature points of the face images, to thereby compute a predetermined result value definable by:

(WI)(k_j,x₀)=∫

P_j(x−

x₀)I(x)dx,

wherein P_j(x−

x₀) denotes a Gabor wavelet, I(x) denotes an image, and P_j(x−

x₀) is definable by;

$\begin{array}{c}{P}_j<br><br>\left(x\right)=\frac{k_j^2}{{\mathrm{\sigma <br><br>}}^2}<br><br>\exp <br><br>\left(-\frac{k_j^2<br><br>x^2}{2<br><br>{\mathrm{\sigma <br><br>}}^2}\right)<br><br>\left(\exp <br><br>\left(\mathrm{i<br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>k_j<br><br>x\right)-\exp <br><br>\left(-\frac{{\mathrm{\sigma <br><br>}}^2}{2}\right)\right)\\ =\frac{k_j^2}{{\mathrm{\sigma <br><br>}}^2}<br><br>\exp <br><br>\left(-\frac{k_j^2<br><br>x^2}{2<br><br>{\mathrm{\sigma <br><br>}}^2}\right)<br><br>\left[\cos <br><br>\left(k_j<br><br>x\right)-\exp <br><br>\left(-\frac{{\mathrm{\sigma <br><br>}}^2}{2}\right)+i<br><br>\text{\hspace{1em}<br><br>}<br><br>\sin <br><br>\left(k_j<br><br>x\right)\right],\end{array}$

wherein the values of v and μ

are definable by;

$k_j=\left(\begin{array}{c}{k}_{\mathrm{jx}}\\ k_{\mathrm{jy}}\end{array}\right)=\left(\begin{array}{c}{k}_v<br><br>\cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\phi <br><br>}}_{\mathrm{\mu <br><br>}}\\ k_v<br><br>\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\phi <br><br>}}_{\mathrm{\mu <br><br>}}\end{array}\right),\mathrm{and}$$k_v=2^{-\frac{v+2}{2}}<br><br>\mathrm{\pi <br><br>},{\mathrm{\phi <br><br>}}_{\mathrm{\mu <br><br>}}=\mathrm{\mu <br><br>}<br><br>\frac{\mathrm{\pi <br><br>}}{8},j=\mathrm{\mu <br><br>}+8<br><br>v$$v=0,1,\mathrm{\dots <br><br>}<br><br>\text{\hspace{1em}<br><br>},4$$\mathrm{\mu <br><br>}=0,1,\mathrm{\dots <br><br>}<br><br>\text{\hspace{1em}<br><br>},7$

wherein k_v and φ

_μ

respectively denote a frequency and a directional characteristic, v has 5 frequencies and μ

has 8 directional characteristics.