Apparatus for and method of constructing multi-view face database, and apparatus for and method of generating multi-view face descriptor

• US 7,643,684 B2
• Filed: 07/15/2004
• Issued: 01/05/2010
• Est. Priority Date: 07/15/2003
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Lee, Won-sook, Sohn, Kyung-ah
• Primary Examiner(s): Seth; Manav
• Application Number:
  US10/891,263
• Publication Number:
  US 20050013507A1
• Time in Patent Office: 2,000 Days
• Field of Search: 382/100, 382/103, 382/115, 382/118, 382/181, 382/190, 382/195, 382/197, 382/199, 382/201, 382203-206, 382/209, 382216-221, 382/294, 382/305, 348135-143, 348152-161, 348169-172
• US Class Current: 382/190
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation

Abstract

An apparatus for generating a multi-view face descriptor includes a multi-view face database storing sets of training face images and sets of test face images, each set including a mosaic view of a single identity; a basis matrix generator generating a basis matrix of the training face images shown from respective viewpoints searched from the multi-view face database; a feature extractor extracting feature vectors of a test face image shown from each viewpoint using the basis matrix of the training face images searched from the multi-view face database; a to-be-registered view selector searching the test face images using the feature vectors extracted by the feature extractor and selecting a plurality of viewpoints as registered viewpoints according to the search results; and a face descriptor generator generating a multi-view face descriptor by synthesizing face descriptors having the plurality of selected viewpoints.

Claims

1. An apparatus for generating a multi-view face descriptor comprising:

a multi-view face database which stores sets of training face images and sets of test face images, each set including a mosaic view of a single identity;

a basis matrix generator which generates a basis matrix of the training face images shown from respective viewpoints searched from the multi-view face database;

a feature extractor which extracts feature vectors of a test face image shown from each viewpoint using the basis matrix of the training face images searched from the multi-view face database;

a to-be-registered view selector which searches the test face images using the feature vectors extracted by the feature extractor and selects a plurality of the viewpoints as registered viewpoints according to the search results; and

a face descriptor generator which generates a multi-view face descriptor by synthesizing face descriptors having the plurality of selected viewpoints.

2. The apparatus of claim 1, wherein the basis matrix generator performs training of viewpoints adjacent to a predetermined viewpoint as well as the predetermined viewpoint.

3. The apparatus of claim 1, wherein the basis matrix generator designates different sub-regions on the face images, according to up and down rotation angles and left and right rotation angles of the face images.

4. The apparatus of claim 3, wherein the basis matrix generator designates different feature vectors in the sub-regions on the face images, according to up and down rotation angles and left and right rotation angles of the face images.

5. The apparatus of claim 1, wherein the feature extractor designates different sub-regions on the face images, according to up and down rotation angles and left and right rotation angles of the face images.

6. The apparatus of claim 5, wherein the feature extractor designates different feature vectors in the sub-regions on the face images, according to up and down rotation angles and left and right rotation angles of the face images.

7. The apparatus of claim 1, wherein the feature extractor generates an entire Fourier feature vector and sub-region Fourier feature vectors of the test face image and sets different sub-regions according to the viewpoints.

8. The apparatus of claim 1, wherein the to-be-registered view selector statistically analyzes face descriptors with a plurality of viewpoints, whose quasi view size is larger than a predetermined value, and viewpoints, which frequently appear in real life, and selects the registered viewpoints by combining at least one standard among the face descriptors with a plurality of viewpoints according to the analysis result and face descriptors with a plurality of viewpoints, with which quasi views corresponding to the given viewpoints can cover the mosaic view.

9. A method of generating a multi-view face descriptor comprising:

providing a multi-view face database storing sets of training face images and sets of test face images, each set including a mosaic view of a single identity;

generating a basis matrix of the training face images shown from respective viewpoints searched from the multi-view face database;

extracting feature vectors of a test face image shown from each viewpoint using the basis matrix of the training face images searched from the multi-view face database; and

searching, using at least one processing device, the test face images using the feature vectors extracted by the feature extractor, selecting a plurality of viewpoints as registered viewpoints according to the search results and generating a multi-view face descriptor by synthesizing face descriptors having the plurality of selected viewpoints.

10. The method of claim 9, wherein, in the generating of the multi-view face descriptor, face descriptors with a plurality of viewpoints, whose quasi view size is larger than a predetermined value, are selected.

11. The method of claim 9, wherein, in the generating of the multi-view face descriptor, viewpoints, which frequently appear in real life, are statistically analyzed, and the face descriptors with a plurality of viewpoints are selected according to the analysis result.

12. The method of claim 9, wherein, in the generating of the multi-view face descriptor, face descriptors with a plurality of viewpoints, with which quasi views corresponding to the given viewpoints can cover the mosaic view, are selected.

13. The method of claim 9, wherein the generating of a basis matrix further comprises training the selected respective viewpoints and adjacent viewpoints together.

14. At least one computer readable recording medium having recorded thereon a computer readable program to control at least one processing device to perform a method of generating a multi-view face descriptor performed by one or more computers, the method comprising:

providing a multi-view face database storing sets of training face images and sets of test face images, each set including a mosaic view of a single identity;

generating a basis matrix of the training face images shown from respective viewpoints searched from the multi-view face database;

extracting feature vectors of a test face image shown from each viewpoint using the basis matrix of the training face images searched from the multi-view face database; and

searching the test face images using the feature vectors extracted by the feature extractor, selecting a plurality of viewpoints as registered viewpoints according to the search results and generating a multi-view face descriptor by synthesizing face descriptors having the plurality of selected viewpoints.

15. An apparatus for generating a multi-view descriptor relative to a selected view using a multi-view database which stores sets of training images and sets of test images, each set including a mosaic view of a single identity, comprising:

a basis matrix generator which generates a basis matrix of the training images shown from respective viewpoints searched from the multi-view database;

a feature extractor which extracts feature vectors of a test image shown from each viewpoint using the basis matrix of the training images;

a view selector which searches the test images using the feature vectors extracted by the feature extractor and selects ones of the viewpoints as registered viewpoints according to the search results; and

a descriptor generator which generates a multi-view descriptor by synthesizing descriptors having the plurality of selected viewpoints.

16. The apparatus of claim 15, wherein:

the feature vectors comprise entire image feature vectors and subregion feature vectors, and

the feature extractor further selects sub-regions of the test images and obtains the subregion feature vectors from the subregions test images, and obtains the entire image feature vectors from the test images.

17. The apparatus of claim 16, wherein each of the subregion feature vectors and the entire image feature vector corresponds to one of the selected viewpoints and enable identification of the corresponding object when the viewpoint is other than the selected viewpoint.

18. The apparatus of claim 15, wherein each feature vector defines at least one quasi view of a corresponding one of the selected viewpoints and which is not included in the selected viewpoints, and the at least one quasi view allows identification of an object in a rotational range relative to the corresponding one selected viewpoint.

19. The apparatus of claim 18, wherein the rotational range of the quasi view relative to the corresponding one selected viewpoint varies as a function of rotational angle of the selected viewpoint relative to a primary viewpoint.

20. The apparatus of claim 16, wherein a number of subregions for one of the selected viewpoints is different from a number of subregions for another one of the selected viewpoints.

21. The apparatus of claim 16, wherein the feature extractor comprises:

a fourier transformer which, for each test image, fourier transforms the test image and at least one of the subregions of the test image,

an entire image projection unit which, for each test image, projects the fourier transformed test image to obtain the entire image feature vector for the test image, and

a subregion image projection unit which, for each subregion of the test image, projects the fourier transformed subregion image to obtain the subregion image feature vector for the subregion of the test image.

22. The apparatus of claim 21, wherein the entire image projection unit and the subregion image projection unit use at least one of a principal component analysis (PCA) method, a linear discriminant analysis (LDA) method, or a Principal Component Linear Discriminant Analysis (PCLDA) method to obtain the entire image and subregion image feature vectors.