Method and apparatus for extracting feature vector used for face recognition and retrieval

• US 7,391,889 B2
• Filed: 10/15/2003
• Issued: 06/24/2008
• Est. Priority Date: 10/15/2002
• Status: Active Grant

• Current Assignee: NEC Corporation, Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: NEC Corporation, Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Kim, Hyun-woo, Kamei, Toshio, Kim, Tae-kyun, Kee, Seok-cheol, Hwang, Won-jun
• Primary Examiner(s): Bella; Matthew C.
• Assistant Examiner(s): Rosario; Dennis
• Application Number:
  US10/685,002
• Publication Number:
  US 20040170305A1
• Time in Patent Office: 1,714 Days
• Field of Search: 382115-118, 382/190, 382/191, 382/195, 382/201, 382/205, 382/206, 382/225
• US Class Current: 382/118
• CPC Class Codes: G06V 40/169 Holistic features and representations, i.e. based on the facial image taken as a whole
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

A method and apparatus for extracting feature vectors for face recognition and retrieval are provided. An entire Fourier feature vector is generated for an entire face area of a normalized face image using first and second normalized vectors, and a central Fourier feature vector is generated for a central face area of the normalized face image using third and fourth normalized vectors. An entire intensity feature vector is generated for the entire face area, and a local intensity feature vector is generated for a predetermined number of face component-wise areas. An entire composite feature vector is generated by coupling the first and second normalized vectors and the entire intensity feature, and a central composite feature vector is generated by coupling the third and fourth normalized vectors and the local intensity feature vectors.

Claims

1. A method of extracting feature vectors for face recognition and retrieval, comprising

(a) generating an entire Fourier feature vector for an entire face area of a normalized face image, and generating a central Fourier feature vector for a central face area of the normalized face image;

and

(b) generating an entire intensity feature vector for the entire face area, generating an entire composite feature vector using a first vector generated from a Fourier transformation of the entire face area and the entire intensity feature vector, generating a local intensity feature vector for a predetermined number of face component-wise areas, and generating a central composite feature vector using a second vector generated from the Fourier transformation of the central face area and the local intensity feature vectors.

2. The method of claim 1, wherein the step (a) comprises:

(a1) obtaining a Fourier spectrum and Fourier amplitude for the entire face area by performing Fourier transformation on the entire face area, and defining first and second feature vectors using the Fourier spectrum and Fourier amplitude obtained for the entire face area;

(a2) obtaining a Fourier spectrum and Fourier amplitude for the central face area by performing Fourier transformation on the central face area, and defining third and fourth feature vectors using the Fourier spectrum and Fourier amplitude obtained for the central face area;

(a3) generating the first and second normalized vectors by projecting the first and second feature vectors onto a first PCLDA sub-space and normalizing resulting feature vectors, and generating the entire Fourier feature vector by coupling the first and second normalized vectors into a single coupled vector and projecting the single coupled vector onto a first linear discriminant space; and

(a4) generating the third and fourth normalized vectors by projecting the third and fourth feature vectors onto a second PCLDA sub-space and normalizing resulting feature vectors, and generating the central Fourier feature vector by coupling the third and fourth normalized vectors into a single coupled vector and projecting the single coupled vector onto a second linear discriminant space.

3. The method of claim 2, wherein in the step (a1), the Fourier spectrum is obtained from the entire face area, and the Fourier amplitude is obtained from a first block-divided face area.

4. The method of claim 3, wherein the first block-divided face area comprises an entire area, which is obtained by clipping the entire face area into a predetermined-sized image, a four-block area, which is obtained by dividing the entire area into four blocks, and a sixteen-block area, which is obtained by dividing the entire area into sixteen blocks.

5. The method of claim 2, wherein in the step (a2), the Fourier spectrum is obtained from the central face area, and the Fourier amplitude is obtained from a second block-divided area.

6. The method of claim 5, wherein the second block-divided face area comprises a central area, which corresponds to the central face area, a four-block area, which is obtained by dividing the central area into four blocks, and a sixteen-block area, which is obtained by dividing the central area into sixteen blocks.

7. The method of claim 1, wherein the step (b) comprises:

(b1) generating first and second normalized vectors for the entire face area of the normalized face image, and generating third and fourth normalized vectors for the central face area of the normalized face image;

(b2) generating an entire intensity feature vector for the entire face area and a local intensity feature vector for a predetermined number of face component-wise areas;

(b3) generating an entire composite feature vector by coupling the first and second normalized vectors and the entire intensity feature vector into a single coupled vector and projecting the coupled vector onto a third linear discriminant space; and

(b4) generating a central composite feature vector by coupling the third and fourth normalized vectors and the local intensity feature vector into a single coupled vector and projecting the coupled vector onto a fourth linear discriminant space.

8. The method of claim 7, wherein the step (b2) comprises:

(b21) converting the normalized face image into a frontal face image by performing pose estimation and compensation on the normalized face image;

(b22) defining a fifth feature vector by performing raster scanning on the entire face area of a resulting pose-compensated face image;

(b23) defining a sixth feature vector by performing raster scanning on a plurality of face component-wise areas of the pose-compensated face image;

(b24) generating the entire intensity feature vector by projecting the fifth feature vector onto a third PCLDA sub-space and normalizing a resulting feature vector into a fifth unit vector; and

(b25) generating the local intensity feature vector by projecting the sixth feature vector onto a fourth PCLDA sub-space and normalizing a resulting feature vector into a sixth unit vector.

9. The method of claim 8, wherein in the step (b21), a pose of the normalized face image is estimated using a PCA-DFFS method.

10. The method of claim 7, wherein the step (b3) comprises:

(b31) coupling the first and second normalized vectors, generated in (b1), with the entire intensity feature vector, generated in (b2), into a single coupled vector;

(b32) projecting the coupled vector onto the third linear discriminant space; and

(b33) quantizing each component of the projected vector and storing a resulting vector as the entire composite feature vector.

11. The method of claim 7, wherein-the step (b4) comprises:

(b41) coupling the third and fourth normalized vectors, generated in (b1), with the local intensity feature vector, generated in (b2), into a single coupled vector;

(b42) projecting the coupled feature vector onto the fourth linear discriminant space; and

(b43) quantizing each component of the projected vector and storing a resulting vector as the central composite feature vector.

13. A computer-readable recording medium on which a program enabling the method of claim 1 is recorded.

12. A method of extracting feature vectors for face recognition and retrieval, comprising

(a) generating an entire Fourier feature based on a cascaded LDA of Fourier characteristics of an entire face area of a normalized face image and generating a central Fourier feature based on a cascaded LDA of Fourier characteristics of a central face area of the normalized face image;

(b) generating an entire intensity vector for the entire face area, generating an entire composite feature based on a cascaded LDA of a first vector generated from a Fourier transformation of the central face area and the entire intensity vector, generating a subregion intensity vector for a predetermined number of face component-wise areas, and generating a subregion composite feature based on a cascaded LDA of a second vector generated from the Fourier transformation of the central face area and the subregion intensity vector; and

(c) outputting either the entire Fourier feature and the central Fourier feature or the composite feature and the subregion composite feature for the face recognition and retrieval.

14. A computer-readable recording medium on which a program enabling the method of claim 12 is recorded.

15. An apparatus for extracting feature vectors for face recognition and retrieval, the apparatus comprising:

a first unit which generates an entire Fourier feature vector for an entire face area of a normalized face image, and generates a central Fourier feature vector for a central face area of the normalized face image; and

a second unit which generates an entire intensity feature vector for the entire face area, generates an entire composite feature vector using a first vector generated from a Fourier transformation of the entire face area and the entire intensity feature vector, generates a local intensity feature vector for a predetermined number of face component-wise areas, and generates a central composite feature vector using a second vector generated from a Fourier transformation of the central face area and the local intensity feature vector.

16. The apparatus of claim 15, wherein the first unit comprises:

a Fourier feature generator which generates first and second normalized vectors for the entire face area of the normalized face image and third and fourth normalized vectors for the central face area;

an entire Fourier feature vector generator which generates an entire Fourier feature vector by coupling the first and second normalized vectors into a single coupled vector and projecting the coupled vector onto a first linear discriminant space; and

a central Fourier feature vector generator which generates a central Fourier feature vector by coupling the third and fourth normalized vectors into a single coupled vector and projecting the coupled vector onto a second linear discriminant space.

17. The apparatus of claim 15, wherein the second unit comprises:

a Fourier feature generator which generates first and second normalized vectors for the entire face area of the normalized face image and third and fourth normalized vectors for the central face area of the normalized face image;

an intensity feature generator which generates an entire intensity feature vector for the entire face area and a local intensity feature vector for a predetermined number of face component-wise areas;

an entire composite feature vector generator which generates an entire composite feature vector by coupling the first and second normalized vectors with the entire intensity feature vector; and

a central composite feature vector generator which generates a central composite feature vector by coupling the third and fourth normalized vectors with the local intensity feature vector.

18. The apparatus of claim 17, wherein the Fourier feature generator comprises:

an entire Fourier feature vector generator which generates the first and second normalized vectors by projecting first and second feature vectors that are defined by a Fourier spectrum and Fourier amplitude, obtained by Fourier-transforming the entire face area, onto a first PCLDA sub-space and normalizing resulting feature vectors; and

a central Fourier feature vector generator which generates the third and fourth normalized vectors by projecting third and fourth feature vectors that are defined by a Fourier spectrum and Fourier amplitude, obtained by Fourier-transforming the central face area, onto a second PCLDA sub-space and normalizing resulting feature vectors.

19. The apparatus of claim 18, wherein the Fourier spectrum used in the entire Fourier feature vector generator is obtained from the entire face area and the Fourier amplitude for the entire face area is obtained from an entire area, which is obtained by clipping the entire face area into a predetermined-sized area, a four-block area, which is obtained by dividing the entire area into four blocks, and a sixteen-block area, which is obtained by dividing the entire area into sixteen blocks.

20. The apparatus of claim 18, wherein the Fourier spectrum used in the central Fourier feature vector generator is obtained from the central face area and the Fourier amplitude for the central face area is obtained from a central area, a four-block area, which is obtained by dividing the central area into four blocks, and a sixteen-block area, which is obtained by dividing the central area into sixteen blocks.

21. The apparatus of claim 17, wherein the intensity feature generator comprises:

a pose estimation and compensation unit which estimates a pose of the normalized face image and converts the normalized face image into a frontal face image based on the result of the estimation;

an entire intensity feature vector generator which generates the entire intensity feature vector by projecting a fifth feature vector that is obtained by performing raster scanning on the entire face area of the normalized face image onto a third PCLDA sub-space and normalizing a resulting feature vector into a fifth unit vector; and

a local intensity feature vector generator which generates the local intensity feature vector by projecting a sixth feature vector that is obtained by performing raster scanning on a plurality of face component-wise areas of a pose-compensated face image onto a fourth PCLDA sub-space and normalizing a resulting feature vector into a sixth unit vector.