Multi-view face recognition method and system

• US 8,615,113 B2
• Filed: 10/20/2008
• Issued: 12/24/2013
• Est. Priority Date: 10/19/2007
• Status: Expired due to Fees

• Current Assignee: Beijing Samsung Telecom R&D Center (Samsung Electronics Co. Ltd.), Samsung Electronics Co. Ltd.
• Original Assignee: Beijing Samsung Telecom R&D Center (Samsung Electronics Co. Ltd.), Samsung Electronics Co. Ltd.
• Inventors: Lee, Jongha, Huang, Xiangsheng, Lee, Seong-deok, Kim, Jungbae
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): WOLDEMARIAM, AKLILU K
• Application Number:
  US12/289,071
• Publication Number:
  US 20090180671A1
• Time in Patent Office: 1,891 Days
• Field of Search: 382/100, 382/115, 382/118, 382/159, 382/190, 382/224
• US Class Current: 382/118
• CPC Class Codes: G06F 18/2132 based on discrimination criteria, e.g. discriminant analysis
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 40/169 Holistic features and representations, i.e. based on the facial image taken as a whole

Abstract

A multi-view face recognition method and system are provided. In the multi-view face recognition method, two images to be recognized are input, a linear projection matrix is calculated based on grouped images in a training set, two feature vectors corresponding to the two input images are extracted based on the linear projection matrix, a distance between the two extracted feature vectors is calculated, and it is determined based on the distance between the two feature vectors whether the two input images belong to a same person.

Claims

1. A multi-view face recognition method comprising:

inputting two images to be recognized;

calculating a linear projection matrix based on grouped images in a training set, wherein the calculating of the linear projection matrix comprises;

grouping images for each person in the training set based on a similarity between the images for each person in the training set;

calculating a general within-class scatter matrix for the grouped training set by calculating a within-class scatter matrix for each group of a same person in the training set, adding the within-class scatter matrixes of the groups of the same person, dividing a sum of the within-class scatter matrixes by the number of groups of the same person to obtain a within-class scatter matrix of each person, and dividing a sum resulting from the addition of the within-class scatter matrixes by the number of persons in the training set to obtain the general within-class scatter matrix;

calculating a general between-class scatter matrix for the grouped training set; and

calculating the linear projection matrix based on the general within-class scatter matrix and the general between-class scatter matrix;

extracting two feature vectors corresponding to the two input images based on the linear projection matrix;

calculating a distance between the two extracted feature vectors; and

determining whether the two input images belong to a same person, based on the distance between the two feature vectors.

2. The multi-view face recognition method of claim 1, further comprising:

performing face detection on the input images to determine whether the input images are images having a face region;

locating eyes if it is determined that the input images are images having a face region; and

dividing face regions from the input images based on the located eyes to obtain face images.

3. The multi-view face recognition method of claim 2, wherein if it is determined that the input images are images not having a face region, information is output.

4. The multi-view face recognition method of claim 1, wherein the calculating of the within-class scatter matrix of each group of the same person in the training set comprises:

calculating a covariance matrix between each image in a group of the same person and another image in the group; and

adding the covariance matrixes calculated for all the images in the group, and dividing a sum of the covariance matrixes by the number of calculated covariance matrixes, to obtain the within-class scatter matrix of the group.

5. The multi-view face recognition method of claim 1, wherein the calculating of the within-class scatter matrix of each group of a same person in the training set comprises:

calculating a mean image of all the images in a group;

calculating a covariance matrix between each image of the group and a mean image of the group; and

adding calculated covariance matrixes of all the images of the group and dividing a sum of the calculated covariance matrixes by the number of calculated covariance matrixes, to obtain the within-class scatter matrix of the group.

6. The multi-view face recognition method of claim 1, wherein the calculating of the general within-class scatter matrix comprises:

calculating a covariance matrix between each image in each group of each person in the training set and another image in the group; and

adding the calculated covariance matrixes of the groups of each person in the training set, and dividing a sum of the calculated covariance matrixes by the number of calculated covariance matrixes, to obtain the general scatter matrix within class.

7. The multi-view face recognition method of claim 1, wherein the calculating of the general between-class scatter matrix comprises:

calculating a mean image of all the images in each group of each person in the training set and calculating a covariance matrix between each image of each group of each person in the training set and the mean image of the group; and

adding the calculated covariance matrixes and dividing a sum of the calculated covariance matrixes by the number of calculated covariance matrixes, to obtain the general scatter matrix within class.

8. The multi-view face recognition method of claim 1, wherein the calculating of the general between-class scatter matrix comprises:

calculating a between-class scatter matrix between each person and another person in the training set;

adding the between-class scatter matrixes between each person and others in the training set and dividing a sum of the between-class scatter matrixes by the number of calculated between-class scatter matrixes between each person and others, to obtain a between-class scatter matrix of each person; and

adding between-class scatter matrixes of all of the persons in the training set and dividing a sum of the between-class scatter matrixes of all of the persons by the number of persons in the training set, to obtain the general between-class scatter matrix.

9. The multi-view face recognition method of claim 1, wherein the calculating of the general between-class scatter matrix comprises:

calculating a covariance matrix between each image of each person in the training set and each image of another person in the training set; and

adding the calculated covariance matrixes and dividing a sum of the calculated covariance matrixes by the number of calculated covariance matrixes, to obtain the general between-class scatter matrix.

10. The multi-view face recognition method of claim 1, wherein the calculating of the general between-class scatter matrix comprises:

calculating a mean image of all the image in each group of each person, wherein the mean image is referred to as a mean image of each group;

calculating a covariance matrix between the mean image of each group of each person and a mean image of the training set;

adding the calculated covariance matrixes for the groups of each person and dividing a sum of the calculated covariance matrixes by the number of calculated covariance matrixes for the groups of each person, to obtain a between-class scatter matrix for each person; and

adding between-class scatter matrixes for all of the persons in the training set and dividing a sum of the between-class scatter matrixes for all of the persons by the number of persons in the training set, to obtain the general between-class scatter matrix.

11. The multi-view face recognition method of claim 1, wherein the calculating of the general between-class scatter matrix comprises:

calculating a mean image of all the images in each group of each person in the training set, wherein the mean image is referred to as a mean image of each group;

calculating a mean image of all the images of the other persons in the training set, wherein the mean image is referred to as a mean image of the other persons;

calculating a covariance matrix between the mean image of each group and the mean image of the other persons;

adding the calculated covariance matrixes for all of the groups of each person and dividing a sum of the calculated covariance matrixes for all of the groups of each person by the number of calculated covariance matrixes for all the groups of each person, to obtain a between-class scatter matrix of each person; and

adding the between-class scatter matrixes of all of the persons in the training set and dividing a sum of the between-class scatter matrixes of all of the persons by the number of persons in the training set, to obtain the general between-class scatter matrix.

12. The multi-view face recognition method of claim 1, wherein the calculating of the general between-class scatter matrix comprises:

calculating a mean image of all the images in the training set, wherein the mean image is referred to as a mean image of the training set;

calculating a mean image of all the images in each group of each person in the training set, wherein the mean image is referred to as a mean image of each group;

calculating a covariance matrix between the mean image of the training set and the mean image of each group for each person; and

adding the calculated covariance matrixes and dividing a sum of the calculated covariance matrixes by the number of calculated covariance matrixes, to obtain the general between-class scatter matrix.

13. The multi-view face recognition method of claim 1, wherein the calculating of the general between-class scatter matrix comprises:

calculating a mean image of all the images in each group for each person in the training set, wherein the mean image is referred to as a mean image of each group;

calculating a mean image of all the images for the other persons in the training set, wherein the mean image is referred to as a mean image of the other persons;

calculating a covariance matrix between the mean image of each group and the mean image of the other persons; and

adding the calculated covariance matrixes and dividing a sum of the calculated covariance matrixes by the number of calculated covariance matrixes, to obtain the general between-class scatter matrix.

14. The multi-view face recognition method of claim 1, wherein the grouping of the images for each person in the training set based on similarity between the images for each person in the training set comprises:

selecting a number of images from the images of each person in the training set to serve as initial clustering centers;

calculating a similarity between each image of the remaining images of each person and each of the initial cluster centers;

allocating the remaining images to respective groups which are most similar to the remaining images, to form new groups;

calculating clustering centers of the new groups; and

repeating the similarity calculating operation, the remaining image allocating operation, and the clustering center calculation operation until a standard measurement function is convergent.

15. A multi-view face recognition system comprising:

an inputting unit inputting two images to be recognized;

a face recognition engine learning unit calculating a linear projection matrix based on grouped images in a training set, wherein the face recognition engine learning unit comprises;

a grouping unit grouping images of each person in the training set based on similarity between images of each person in the training set;

a within-class scatter matrix calculating unit calculating a general within-class scatter matrix for the grouped training set by calculating a within-class scatter matrix for each group of a same person in the training set, adding the within-class scatter matrixes of the groups of the same person, dividing a sum of the within-class scatter matrixes by the number of groups of the same person to obtain a within-class scatter matrix of each person, and dividing a sum resulting from the addition of the within-class scatter matrixes by the number of persons in the training set to obtain the general within-class scatter matrix;

a between-class scatter matrix calculating unit calculating a general between-class scatter matrix for the grouped training set; and

a linear projection matrix calculating unit calculating the linear projection matrix based on the general within-class scatter matrix and the general between-class scatter matrix;

a feature extracting unit extracting two feature vectors corresponding to the two input images based on the linear projection matrix provided by the face recognition engine learning unit;

a feature vector distance calculating unit calculating a distance between the two feature vectors extracted by the feature extracting unit; and

a determining unit determining whether the two input images belong to a same person, based on the distance between the two feature vectors calculated by the feature vector distance calculating unit.

16. The multi-view face recognition system of claim 15, further comprising:

a face detecting unit performing face detection on the input images to determine whether the input images are images having a face region;

an eye locating unit locating eyes if the face detecting unit determines that the input images are images having a face region;

a face region dividing unit dividing face regions from the input images based on the eyes located by the eye locating unit to obtain face images.

17. The multi-view face recognition system of claim 16, further comprising an outputting unit outputting information if the face detecting unit determines that the input images are images not having a face region.