Method and apparatus for extracting feature vector used for face recognition and retrieval

• US 20040170305A1
• Filed: 10/15/2003
• Published: 09/02/2004
• Est. Priority Date: 10/15/2002
• Status: Active Grant

• Current Assignee: NEC Corporation, Samsung Electronics Co. Ltd.
• Original Assignee: NEC Corporation, Samsung Electronics Co. Ltd.
• Inventors: Kim, Hyun-Woo, Kamei, Toshio, Kim, Tae-Kyun, Hwang, Won-Jun, Kee, Seok-Cheol
• Granted Patent: US 7,391,889 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06V 40/169 Holistic features and representations, i.e. based on the facial image taken as a whole
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

A method and apparatus for extracting feature vectors for face recognition and retrieval are provided. An entire Fourier feature vector is generated for an entire face area of a normalized face image using first and second normalized vectors, and a central Fourier feature vector is generated for a central face area of the normalized face image using third and fourth normalized vectors. An entire intensity feature vector is generated for the entire face area, and a local intensity feature vector is generated for a predetermined number of face component-wise areas. An entire composite feature vector is generated by coupling the first and second normalized vectors and the entire intensity feature, and a central composite feature vector is generated by coupling the third and fourth normalized vectors and the local intensity feature vectors.

Claims

1. A method of extracting feature vectors for face recognition and retrieval, the method comprising the steps of:

(a) generating an entire Fourier feature vector for an entire face area of a normalized face image using first and second normalized vectors, and generating a central Fourier feature vector for a central face area of the normalized face image using third and fourth normalized vectors; and

(b) generating an entire intensity feature vector for the entire face area and a local intensity feature vector for a predetermined number of face component-wise areas, generating an entire composite feature vector by coupling the first and second normalized vectors and the entire intensity feature, and generating a central composite feature vector by coupling the third and fourth normalized vectors and the local intensity feature vectors.

2. The method of claim 1, wherein the step (a) comprises:

(a1) obtaining a Fourier spectrum and Fourier amplitude for the entire face area by performing Fourier transformation on the entire face area, and defining first and second feature vectors using the Fourier spectrum and Fourier amplitude obtained for the entire face area;

(a2) obtaining a Fourier spectrum and Fourier amplitude for the central face area by performing Fourier transformation on the central face area, and defining third and fourth feature vectors using the Fourier spectrum and Fourier amplitude obtained for the central face area;

(a3) generating the first and second normalized vectors by projecting the first and second feature vectdrs onto a first PCLDA sub-space and normalizing resulting feature vectors, and generating the entire Fourier feature vector by coupling the first and second normalized vectors into a single coupled vector and projecting the single coupled vector onto a first linear discriminant space; and

(a4) generating the third and fourth normalized vectors by projecting the third and fourth feature vectors onto a second PCLDA sub-space and normalizing resulting feature vectors, and generating the central Fourier feature vector by coupling the third and fourth normalized vectors into a single coupled vector and projecting the single coupled vector onto a second linear discriminant space.

3. The method of claim 2, wherein in the step (a1), the Fourier spectrum is obtained from the entire face area, and the Fourier amplitude is obtained from a first block-divided face area.

4. The method of claim 3, wherein the first block-divided face area comprises an entire area, which is obtained by clipping the entire face area into a predetermined-sized image, a four-block area, which is obtained by dividing the entire area into four blocks, and a sixteen-block area, which is obtained by dividing the entire area into sixteen blocks.

5. The method of claim 2, wherein in the step (a2), the Fourier spectrum is obtained from the central face area, and the Fourier amplitude is obtained from a second block-divided area.

6. The method of claim 5, wherein the second block-divided face area comprises a central area, which corresponds to the central face area, a four-block area, which is obtained by dividing the central area into four blocks, and a sixteen-block area, which is obtained by dividing the central area into sixteen blocks.

7. The method of claim 1, wherein the step (b) comprises:

(b1) generating the first and second normalized vectors for the entire face area of the normalized face image, and generating the third and fourth normalized vectors for the central face area of the normalized face image;

(b2) generating an entire intensity feature vector for the entire face area and a local intensity feature vector for a predetermined number of face component-wise areas;

(b3) generating an entire composite feature vector by coupling the first and second normalized vectors and the entire intensity feature vector into a single coupled vector and projecting the coupled vector onto a third linear discriminant space; and

(b4) generating a central composite feature vector by coupling the third and fourth normalized vectors and the local intensity feature vector into a single coupled vector and projecting the coupled vector onto a fourth linear discriminant space.

8. The method of claim 7, wherein the step (b2) comprises:

(b21) converting the normalized face image into a frontal face image by performing pose estimation and compensation on the normalized face image;

(b22) defining a fifth feature vector by performing raster scanning on the entire face area of a resulting pose-compensated face image;

(b23) defining a sixth feature vector by performing raster scanning on a plurality of face component-wise areas of the pose-compensated face image;

(b24) generating the entire intensity feature vector by projecting the fifth feature vector onto a third PCLDA sub-space and normalizing a resulting feature vector into a fifth unit vector; and

(b25) generating the local intensity feature vector by projecting the sixth feature vector onto a fourth PCLDA sub-space and normalizing a resulting feature vector into a sixth unit vector.

9. The method of claim 8, wherein in the step (b21), a pose of the normalized face image is estimated using a PCA-DFFS method.

10. The method of claim 7, wherein the step (b3) comprises:

(b31) coupling the first and second normalized vectors, generated in (b1), with the entire intensity feature vector, generated in (b2), into a single coupled vector;

(b32) projecting the coupled vector onto the third linear discriminant space; and

(b33) quantizing each component of the projected vector and storing a resulting vector as the entire composite feature vector.

11. The method of claim 7, wherein-the step (b4) comprises:

(b41) coupling the third and fourth normalized vectors, generated in (b1), with the local intensity feature vector, generated in (b2), into a single coupled vector;

(b42) projecting the coupled feature vector onto the fourth linear discriminant space; and

(b43) quantizing each component of the projected vector and storing a resulting vector as the central composite feature vector.

13. A computer-readable recording medium on which a program enabling the method of claim 1 is recorded.

12. A method of extracting feature vectors for face recognition and retrieval, the method comprising the steps of:

(a) generating an entire Fourier feature vector for an entire face area of a normalized face image and a central Fourier feature vector for a central face area of the normalized face image;

(b) generating an entire intensity feature vector for the entire face area and a local intensity feature vector for a predetermined number of face component-wise areas;

(c) generating an entire composite feature vector by coupling the entire Fourier feature vector and the entire intensity feature vector into a single coupled vector and projecting the coupled vector onto a first linear discriminant space; and

(d) generating a central composite feature vector by coupling the central Fourier feature vector and the local intensity feature vector into a single coupled vector and projecting the coupled vector onto a second linear discriminant space.

14. A computer-readable recording medium on which a program enabling the method of claim 12 is recorded.

15. An apparatus for extracting feature vectors for face recognition and retrieval, the apparatus comprising:

a first unit which generates an entire Fourier feature vector for an entire face area of a normalized face image using first and second normalized vectors, and generates a central Fourier feature vector for a central face area of the normalized face image using third and fourth normalized vectors; and

a second unit which generates an entire intensity feature vector for the entire face image and a local intensity feature vector for a predetermined number of face component-wise areas, generates an entire composite feature vector by coupling the first and second normalized vectors and the entire intensity feature vector, and generates a central composite feature vector by coupling the third and fourth normalized vectors and the local intensity feature vector.

16. The apparatus of claim 15, wherein the first unit comprises:

a Fourier feature generator which generates first and second normalized vectors for the entire face area of the normalized face image and third and fourth normalized vectors for the central face area;

an entire Fourier feature vector generator which generates an entire Fourier feature vector by coupling the first and second normalized vectors into a single coupled vector and projecting the coupled vector onto a first linear discriminant space; and

a central Fourier feature vector generator which generates a central Fourier feature vector by coupling the third and fourth normalized vectors into a single coupled vector and projecting the coupled vector onto a second linear discriminant space.

17. The apparatus of claim 15, wherein the second unit comprises:

a Fourier feature generator which generates first and second normalized vectors for the entire face area of the normalized face image and third and fourth normalized vectors for the central face area of the normalized face image;

an intensity feature generator which generates an entire intensity feature vector for the entire face area and a local intensity feature vector for a predetermined number of face component-wise areas;

an entire composite feature vector generator which generates an entire composite feature vector by coupling the first and second normalized vectors with the entire intensity feature vector; and

a central composite feature vector generator which generates a central composite feature vector by coupling the third and fourth normalized vectors with the local intensity feature vector.

18. The apparatus of claim 17, wherein the Fourier feature generator comprises:

an entire Fourier feature vector generator which generates the first and second normalized vectors by projecting first and second feature vectors that are defined by a Fourier spectrum and Fourier amplitude, obtained by Fourier-transforming the entire face area, onto a first PCLDA sub-space and normalizing resulting feature vectors; and

a central Fourier feature vector generator which generates the third and fourth normalized vectors by projecting third and fourth feature vectors that are defined by a Fourier spectrum and Fourier amplitude, obtained by Fourier-transforming the central face area, onto a second PCLDA sub-space and normalizing resulting feature vectors.

19. The apparatus of claim 18, wherein the Fourier spectrum used in the entire Fourier feature vector generator is obtained from the entire face area and the Fourier amplitude for the entire face area is obtained from an entire area, which is obtained by clipping the entire face area into a predetermined-sized area, a four-block area, which is obtained by dividing the entire area into four blocks, and a sixteen-block area, which is obtained by dividing the entire area into sixteen blocks.

20. The apparatus of claim 18, wherein the Fourier spectrum used in the central Fourier feature vector generator is obtained from the central face area and the Fourier amplitude for the central face area is obtained from a central area, a four-block area, which is obtained by dividing the central area into four blocks, and a sixteen-block area, which is obtained by dividing the central area into sixteen blocks.

21. The apparatus of claim 17, wherein the intensity feature generator comprises:

a pose estimation and compensation unit which estimates a pose of the normalized face image and converts the normalized face image into a frontal face image based on the result of the estimation;

an entire intensity feature vector generator which generates the entire intensity feature vector by projecting a fifth feature vector that is obtained by performing raster scanning on the entire face area of the normalized face image onto a third PCLDA sub-space and normalizing a resulting feature vector into a fifth unit vector; and

a local intensity feature vector generator which generates the local intensity feature vector by projecting a sixth feature vector that is obtained by performing raster scanning on a plurality of face component-wise areas of a pose-compensated face image onto a fourth PCLDA sub-space and normalizing a resulting feature vector into a sixth unit vector.