User recognition system and method thereof

• US 20060204058A1
• Filed: 12/01/2005
• Published: 09/14/2006
• Est. Priority Date: 12/07/2004
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Yoon, Ho-Sub, Kim, Do-Hyung, Cho, Young-Jo, Lee, Jae-Yeon
• Granted Patent: US 7,734,067 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

In realizing a user recognition system and a method thereof, a user feature vector is extracted from an input facial image, and at least one cluster is generated and a user feature template is enrolled. The cluster includes the feature vector as a member of it. When a user facial image is inputted, a user feature vector is extracted from the image, and a similarity between the feature vector and the user feature template is calculated. When the similarity is greater than a predetermined threshold value, a user of the user feature template is recognized as the user of the input image.

Claims

1. A method of recognizing a user from an input image, the method comprising:

a) extracting a user feature vector from the input image;

b) calculating a similarity between the extracted feature vector and a user feature template, the user feature template being user feature information that includes a cluster having a user feature vector as a member of the cluster; and

c) recognizing a enrolled user of the feature template as an input user when the similarity between the extracted user feature vector and the feature template exceeds a predetermined recognition threshold value.

2. The method of claim 1, further comprising:

i) extracting one or more user feature vectors from the input image, generating at least one cluster having at least one the user feature vectors as a member of the cluster, and enrolling a user feature template including the generated at least one cluster.

3. The method of claim 2, wherein the cluster having a feature vector as a member of the cluster is generated at least once, and a similarity between the feature vector and a centroid of each cluster exceeds a predetermined cluster classification threshold value.

4. The method of claim 2, wherein i) comprises:

i-1) setting a first feature vector among the feature vectors extracted from the input image as a member of a first cluster and setting the first feature vector as a first centroid of the first cluster;

i-2) when there exists one or more additional feature vectors in addition to the first feature vector, calculating similarity between each one of the plurality of feature vectors and the centroid of the first cluster;

i-3) after the step i-2), setting every feature vector with the calculated similarity exceeding the predetermined cluster classification threshold value as a member of the first cluster; and

i-4) after the step i-2), generating a second cluster having every feature vector with the calculated similarity not exceeding the predetermined cluster classification threshold value as a member of the second cluster.

5. The method of claim 1, wherein b) comprises calculating a similarity between the user feature template and the user feature vector by comparing a similarity between each cluster of the user feature template and the user feature vector.

6. The method of claim 5, wherein b) further comprises calculating similarities between each of the clusters and the user feature vector when one user feature template includes a plurality of clusters, and calculating a higher similarity as a similarity between the corresponding user feature template and the user feature vector.

7. The method of claim 1, wherein when a given user is recognized in c), the method further comprises:

d) capturing input images of the recognized user;

e) extracting a feature vector from the captured image and re-verifying whether the captured image corresponds to an image of the recognized user; and

f) when the input image is re-verified to be the same as the image of the recognized user in e), updating a user feature template of the recognized user with the captured images.

8. The method of claim 7, wherein e) comprises:

e-1) extracting a feature vector from the captured images;

e-2) calculating a similarity between the extracted feature vector and the user feature template of the recognized user;

e-3) re-verifying whether the captured images correspond to the image of the recognized user when the calculated similarity exceeds a predetermined update threshold value.

9. The method of claim 8, wherein the update threshold value is set to be greater than the recognition threshold value.

10. The method of claim 7, wherein f) comprises:

f-1) calculating a similarity between a feature vector of the captured image and a cluster of the user feature template of the recognized user;

f-2) searching for a winner cluster having a highest similarity, and calculating a similarity between the corresponding feature vector and the winner cluster and the predetermined cluster classification threshold value; and

f-3) including the corresponding feature vector to the winner cluster as a member of the winner cluster when the similarity between the corresponding feature vector and the winner cluster is greater than the predetermined cluster classification threshold value.

11. The method of claim 10, wherein after f-2), when the similarity between the corresponding feature vector and the winner cluster is less than the predetermined cluster classification threshold value, the method further comprises generating a cluster and including the corresponding feature vector to the generated cluster as a member of the generated cluster.

12. The method of claim 10, wherein after f-3), the method further comprises:

f-4) setting a centroid of the winner cluster having the additional member; and

f-5) removing a member corresponding to the reset centroid from the winner cluster when a total number of members of the winner cluster exceeds a predetermined maximum number of cluster members.

13. The method of claim 11, wherein when a cluster is added and a total number of clusters exceeds a predetermined maximum number of cluster members, the method further comprises:

selecting a cluster to be removed from among clusters of the user feature template based on at least more than one cluster having a highest mean similarity with other clusters and the least referred cluster in the user recognition process; and

removing the selected cluster from the user feature template.

14. The method of claim 3 or 4 or 12, wherein when the cluster includes a plurality of members of feature vectors, the centroid of the cluster corresponds to a feature vector having the greatest sum of similarities of the centroid of the cluster and other members of the cluster.

15. The method of claim 14, wherein the centroid is retrieved from

$C_k=\arg$

 

⁢

max i = 0 , m ⁢

∑

j = 0 m ⁢

S ⁡

( F i , F j ) ,  

⁢

i ≠

j ⁢

 

⁢

and ⁢

 

⁢

F i , F j ∈

C k

wherein C_k is a cluster, c_k is a centroid of the cluster C_k, and F_i and F_j;

are feature vectors.

16. The method of claim 5 or 6 or 8, wherein the similarity between the user feature template and the user feature vector is calculated by

S_k=max_i=0,m S(F_q,c_i), c_iε

C_i and C_iε

T_k

wherein S_k is the similarity, C_i is a cluster, c_i;

is a centroid of the cluster C_i, and F_q is a feature vector.

17. The method of claim 13, wherein the cluster having the highest mean similarity with another cluster among the clusters of the user feature template is calculated by

$C_p=\arg$

 

⁢

max i = 0 , m ⁢

∑

j = 0 m ⁢

S ⁡

( C i , C j ) , m - 1 ⁢

i ≠

j ⁢

 

⁢

and ⁢

 

⁢

C i , C j ∈

T p

wherein T_p is a template of a user P, C_i and C_j are clusters of the template T_p, and C_p is a centroid cluster of the template T_p.

18. The method of claim 13, wherein the cluster to be removed is selected by

C_d=arg max_i=0,n (α

(1−

C_ri)+(1−

α

)C_si)

wherein C_ri is a contribution degree of the i-th cluster, C_si is a mean similarity between i-th cluster and other clusters, and a is a weight value.

19. A system recognizing a user from an input image, the system comprising:

a database storing a user feature template as user feature information;

a feature vector calculator extracting a user feature vector from the input image;

a user feature information enroller generating a plurality of clusters having the feature vector as a member of the cluster, and storing a user feature template including the plurality of clusters in the database; and

a user recognizer calculating similarities between the feature vector and the stored user feature template, and recognizing a user of the user feature template as a user of the input image when the similarity exceeds a predetermined recognition threshold value.

20. The system of claim 19, wherein the user recognizer compares a similarity between the feature vector and the stored user feature template and controls the user feature information enroller to generate a user feature template and store the cluster in the database when the similarity does not exceed the recognition threshold value.

21. The system of claim 19 or 20, wherein the cluster having a feature vector as a member of the cluster is generated at least once, and a similarity between the feature vector and a centroid of each cluster exceeds a predetermined cluster classification threshold value.

22. The system of claim 19 or 20, wherein the user recognizer

calculates a similarity between the user feature template and the user feature vector by comparing a similarity between each cluster of the user feature template and the user feature vector,

calculates a highest similarity between the user feature template and the user feature vector by calculating similarities between each of the clusters and the user feature vector when a user feature template includes a plurality of clusters, and

when the highest similarity exceeds the recognition threshold value, recognizes an enrolled user of the user feature template as a user of the input image.

23. The system of claim 19, further comprising:

a tracking unit capturing an input image of a user when the user is recognized;

a re-verification unit calculating a feature vector from the captured image, and calculating a similarity between the feature vector and a user feature template of the recognized user, and

re-verifying whether the captured image corresponds to an input image of the recognized user when the similarity exceeds a predetermined update threshold value; and

a user feature information update unit updating a user feature template stored in the database with the captured image when the captured image corresponds to the input image of the recognized user.

24. The system of claim 23, wherein the update threshold value is set to be greater than the recognition threshold value.

25. The system of claim 24, wherein the user feature information update unit

calculates similarities between feature vectors of the captured images and clusters of a user feature template of the recognized user and searches for a winner cluster having the highest similarity, and compares a similarity between the corresponding feature vector and the winner cluster and a predetermined cluster classification threshold value,

adds the corresponding feature vector as a member of the winner cluster when a similarity between the corresponding feature vector and the winner cluster is greater than the predetermined cluster classification threshold value, and

generates a cluster and adds the corresponding feature vector as a member of the generated cluster when the similarity between the corresponding feature vector and the winner cluster is not greater than the predetermined cluster classification threshold value.

26. The system of claim 25, wherein the user feature information update unit sets a centroid of the cluster including the additional member, and removes a member corresponding to the reset centroid from the winner cluster when total members of the winner cluster exceed a predetermined maximum number of cluster members.

27. The system of claim 25, wherein the user feature information update unit

selects a cluster to be removed from among clusters of the user feature template based on at least more than one cluster having the highest mean similarity with other clusters and the least referred cluster in the user recognition process, and removes the selected cluster from the user feature template when a cluster is added and a total number of clusters exceed the predetermined maximum number of cluster members.

28. The system of claim 26, wherein, when the cluster has a plurality of members, a centroid of the cluster corresponds to a feature vector having the greatest sum of similarities between the feature vector and the plurality of members of the cluster, and the centroid is derived from the following equation:

$C_k=\arg <br><br>\text{\hspace{1em}<br><br>}<br><br>{\max }_{i=0,m}<br><br>\underset{j=0}{\overset{m}{\sum <br><br>}}<br><br>S<br><br>\left(F_i,F_j\right),\text{\hspace{1em}<br><br>}<br><br>i\ne <br><br>j<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{and}<br><br>\text{\hspace{1em}<br><br>}<br><br>F_i,F_j\in <br><br>C_k$

wherein C_k is a cluster, c_k is a centroid of the cluster C_k, and F_i and F_j are feature vectors.

29. The system of claim 19 or 23, wherein a similarity between the user feature template and the user feature vector is calculated from the following equation:

S_k=max_i=0,m S(F_q,c_i), c_iε

C_i and C_iε

T_k

wherein S_k is a;

similarity, C_i is a cluster, c_i is a centroid of the cluster C_i, and F_q is a feature vector)

30. The system of claim 27, wherein a cluster having the highest mean similarity among other clusters of the user feature template is calculated from the is following equation:

$C_p=\arg <br><br>\text{\hspace{1em}<br><br>}<br><br>{\max }_{i=0,m}<br><br>\frac{\underset{j=0}{\overset{m}{\sum <br><br>}}<br><br>S<br><br>\left(C_i,C_j\right),}{m-1}<br><br>i\ne <br><br>j<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{and}<br><br>\text{\hspace{1em}<br><br>}<br><br>C_i,C_j\in <br><br>T_p$

wherein T_p is a template of user P, C_i, C_j are clusters of the template T_p, and the cluster C_p is a centroid cluster of the template T_p.

31. The system of claim 27, wherein a cluster to be removed is selected by the following equation:

C_d=arg max_i=0,n (α

(1−

C_ri)+(1−

α

)C_si)

wherein C_ri is a contribution degree of the i-th cluster, C_si is a mean similarity between the i-th cluster and other clusters, and α

is a weight value.