FACE RECOGNITION METHOD

• US 20100135541A1
• Filed: 01/06/2009
• Published: 06/03/2010
• Est. Priority Date: 12/02/2008
• Status: Active Grant

• Current Assignee: National Tsing Hua University
• Original Assignee: National Tsing Hua University
• Inventors: LAI, Shang-Hong, LIAO, Chia-Te
• Granted Patent: US 8,300,900 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06F 18/24147 Distances to closest patterns, e.g. nearest neighbour classification
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

The invention discloses a face recognition method that reconstructs a 3D face model from a single face image, synthesizes a set of face images under different conditions (such as pose, light . . . ) via the 3D face model, feeds the set of face images under different conditions to the face recognition classifier for training, and making intermediate decisions whether to-be identified individual from a series of video frames is a legal system user by the face recognition classifier. Moreover, the method not only recognizes legal system users, but also rejects imposters, a function inspired by the idea of LLE. Finally, better reliability can be achieved by fusing temporal intermediate decisions.

Claims

1. A face recognition method comprising:

obtaining an original face image from a database and reconstructing a 3D face model for said original face image using a 3D morphable face model;

providing synthesized face images from said 3D face model for training a face recognition classifier;

receiving a plurality of video frames of a face, and obtaining a similarity probability for each said video frame from said face recognition classifier; and

fusing said temporal similarity probability for each said video frame by accumulation to obtain a face recognition confidence value, and if said face recognition confidence value is larger than a threshold, said face is recognized to be the same as the face from said original face image.

2. The face recognition method according to claim 1, wherein said original face image is a 2D image.

3. The face recognition method according to claim 1, wherein steps for reconstructing said 3D face model for said original face image include:

constructing said 3D morphable face model, wherein said 3D morphable face model comprises a 3D geometry data set and a 3D texture data set;

expressing said 3D morphable face model by a linear combination of a plurality of eigen head basis vectors; and

obtaining a geometry data set, a texture data set and 3D characteristic parameters by photometric error minimization for reconstructing said 3D face model corresponding to said original face image.

4. The face recognition method according to claim 3, wherein said 3D geometry data set can be expressed as:

S=(x₁,y₁,z₁,x₂,y₂,z₂, . . . ,x_N,y_N,z_N)∈

,

wherein N is the number of facial sampling points.

5. The face recognition method according to claim 3, wherein said 3D texture data can be expressed as:

T=(r₁,g₁,b₁,r₂,g₂,b₂, . . . ,r_N,g_N,b_N)∈

,

wherein N is the number of facial sampling points.

6. The face recognition method according to claim 3, wherein said

geometry data set is:

S(α

)=S+Σ

_i=1^mα

_is_i; and

said texture data set is;

T(β

)=T+Σ

_i=1^mβ

_it_i,

wherein S is the mean shape vector, T is the mean texture vector, s_i is the i_th eigen-shape basis vector, t_i is the i_th eigen-texture basis vector, α

=[α

₁, . . . ,α

_m] are the corresponding coefficients of said eigen-shape basis vectors and β

=[β

₁, . . . ,β

_m] are the corresponding coefficients of said eigen-texture basis vectors.

7. The face recognition method according to claim 3, wherein said 3D characteristic parameters are produced by minimizing a cost function:

∥

I_input−

I_model(α

,β

,P_ortho,f,R,t_2d)∥

,

wherein I_input is said input original face image, P_ortho is an orthographic projection matrix, f is the scaling factor, R is the 3 D rotation matrix, and t_2d is the translation vector.

8. The face recognition method according to claim 1, wherein the step for obtaining said face recognition confidence value by fusing said temporal similarity probability of each said video frame by accumulation is:

$P_i^t=\frac{\left(\mathrm{\omega <br><br>}\times <br><br>P_i^{t-1}+\left(1-\mathrm{\omega <br><br>}\right)\times <br><br>P_i^{t,\mathrm{SVC}}\right)}{\underset{i}{\sum <br><br>}<br><br>\left(\mathrm{\omega <br><br>}\times <br><br>P_i^{t-1}+\left(1-\mathrm{\omega <br><br>}\right)\times <br><br>P_i^{t,\mathrm{SVC}}\right)},i=1,2,\mathrm{\dots <br><br>}<br><br>,n,$

wherein ω

is a forgetting factor, P_i^t,SVC is a posterior probability, denoting the probability that a single video frame belongs to the i_th class at time t, and P_i^t,SVC is merged with previously accumulated probability P_i<sup>t−

1</sup> to contribute to the final similarity probability P_i^t accumulated up to time t.

9. The face recognition method according to claim 1 including:

calculating a rejection probability P_rej^t for each said video frame; and

summing up said similarity probability P_i^t along with said rejection probability {circumflex over (P)}_rej^t up to time t for normalization and obtaining a posterior probability {circumflex over (P)}_i^t of said video frames up to time t belonging to class c.

10. The face recognition method according to claim 9, wherein steps for calculating said rejection probability of each said video frame comprises:

${\mathrm{\varepsilon <br><br>}}^t<br><br>\left(w\right)=<br><br>I^t-\underset{j=1}{\overset{K}{\sum <br><br>}}<br><br>w_j^{c,t}<br><br>I_j^c<br><br>;<br><br>\mathrm{and}$ $P_{\mathrm{rej}}^t=1-\exp <br><br>\left(\frac{-{\mathrm{\varepsilon <br><br>}}^t}{{\mathrm{\varepsilon <br><br>}}_{\mathrm{threshold}}^c}\right),<br><br>{\hat{P}}_{\mathrm{rej}}^t=\mathrm{\omega <br><br>}\times <br><br>{\hat{P}}_{\mathrm{rej}}^{t-1}+\left(1-\mathrm{\omega <br><br>}\right)\times <br><br>P_{\mathrm{rej}}^t,$

wherein w_j^c,t denotes the weights of K nearest neighbors in class c that best approximate said video frame I^t, and Σ

_j=1^Kw_j^c,t=1;

{I_j^c}_j=1^K is the set of K nearest neighbors of the same class; and

ε

_threshold^c is the parameter for controlling rejection rate.

11. The face recognition method according to claim 9, wherein the step for summing up said similarity probability along with said rejection probability up to time t for normalization can be expressed as:

${\hat{P}}_i^t=\frac{{\mathrm{\beta <br><br>}}^t\times <br><br>\left(\mathrm{\omega <br><br>}\times <br><br>P_i^{t-1}+\left(1-\mathrm{\omega <br><br>}\right)\times <br><br>P_i^{t,\mathrm{SVC}}\right)}{\begin{array}{c}\left(\mathrm{\omega <br><br>}\times <br><br>{\hat{P}}_{\mathrm{rej}}^{t-1}+\left(1-\mathrm{\omega <br><br>}\right)\times <br><br>P_{\mathrm{rej}}^t\right)+{\mathrm{\beta <br><br>}}^t\times <br><br>\\ \underset{i}{\sum <br><br>}<br><br>\left(\mathrm{\omega <br><br>}\times <br><br>P_i^{t-1}+\left(1-\mathrm{\omega <br><br>}\right)\times <br><br>P_i^{t,\mathrm{SVC}}\right)\end{array}},$

wherein i=1, 2, . .. , n, β

^t=1−

{circumflex over (P)}_rej^t is the acceptance rate at time t, and {circumflex over (P)}^t=[{circumflex over (P)}₁^t, {circumflex over (P)}₂^t, . . . , {circumflex over (P)}_n^t, {circumflex over (P)}_rej^t]T is the posterior probability distribution.

12. The face recognition method according to claim 9, wherein if {circumflex over (P)}_i^t is greater than said threshold and

$\arg $



max i 

P ^ i t = n + 1 ,

then said face image is determined to be belonging to an imposter;

otherwise, said face image is recognized as a legal system user and his/her identity is farther recognized, i.e.

$c=\arg <br><br><br><br>\underset{i}{\max }<br><br>{\hat{P}}_i^t,$

and c=1,2, . . . ,n.