Face recognition method using artificial neural network and apparatus thereof

• US 7,295,687 B2
• Filed: 07/31/2003
• Issued: 11/13/2007
• Est. Priority Date: 08/13/2002
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Tae-kyun, Kee, Seok-cheol
• Primary Examiner(s): Desire; Gregory M
• Application Number:
  US10/630,787
• Publication Number:
  US 20040034611A1
• Time in Patent Office: 1,566 Days
• Field of Search: 340/5.52, 340/5.53, 340/5.83, 359/559, 359/561, 382/118, 382/156, 382/159, 382/224, 706/2, 706/52, 706/20
• US Class Current: 382/118
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

A face recognition method using artificial neural network and an apparatus thereof are provided. The apparatus comprises an eigenpaxel selection unit which generates eigenpaxels indicating characteristic patterns of a face and selects a predetermined number of eigenpaxels among the generated eigenpaxels; an eigenfiltering unit which filters an input image with the selected eigenpaxels; a predetermined number of neural networks, each of which corresponds to one of the selected eigenpaxels, receives an image signal which is filtered by the corresponding eigenpaxel, and output a face recognition result; and a determination unit which receives the recognition result from each of the neural networks and outputs a final face recognition result of the input image.

Claims

1. A face recognition apparatus which recognizes a face by using an artificial neural network, comprising:

an eigenpaxel selection unit which generates eigenpaxels representing characteristic patterns of a face and selects a predetermined number of eigenpaxels among the generated eigenpaxels;

an eigenfiltering unit which filters an input facial image with the selected eigenpaxels;

a predetermined number of neural networks, each of which corresponds to one of the selected eigenpaxels, receives an image signal which is filtered by the corresponding eigenpaxel, and output a face recognition result; and

a determination unit which receives the recognition result from each of the neural networks and outputs a final face recognition result of the input image.

2. The apparatus of claim 1, wherein the eigenpaxel selection unit comprises:

an eigenpaxel generation unit which divides an arbitrary face image into predetermined blocks, and by applying a principal component analysis to each of the blocks, generates eigenpaxels indicating predetermined pattern characteristics of the face.

3. The apparatus of claim 2, wherein the eigenpaxel selection unit further comprises:

an eigenpaxel selection unit which selects a predetermined number of vectors among the selected eigenpaxels in order of decreasing eigenvalues of the vectors.

4. The apparatus of claim 1, wherein the eigenfiltering unit elgenfilters the input image by performing convolution operation while moving each eigenpaxel for the input image.

5. The apparatus of claim 4, wherein the eigenfiltering unit divides the input image into predetermined blocks, and performs convolution operations moving the divided blocks over the eigenpaxels such that the blocks and eigenpaxels overlap to a predetermined part, and outputs an eigenfiltered image which is reduced to a predetermined size.

6. The apparatus of claim 1, wherein the neural network is trained by a back propagation algorithm having added momentum, and at this time weighting coefficient (w_i) is updated by the following equation:

Δ

W_ij(t+1)=η

δ

_jo_i+α

Δ

W_ij(t)

where η

denotes a learning coefficient, j denotes the index of a current unit, i denotes the index of a unit preceding the unit j, δ

_j denotes the output error of unit j, O_i denotes the output of unit i, and α

denotes a constant determining the affect of momentum.

7. The apparatus of claim 1, wherein the number of neurons in the output layer of the neural network is the same as the number of learned faces.

8. The apparatus of claim 1, wherein the determination unit comprises:

a reliability value calculation unit which converts the received output value of the output layer neurons of each neural network into a value in a predetermined range, calculates reliability values, and adds reliability values for the output layer neurons of each neural network;

a face selection unit which selects a face detected by a predetermined number of high level neural networks according to the reliability value;

a grade calculation unit which gives a grade to the reliability value of each neural network, and for the selected face, calculates a first grade order of the entire neural network and a second grade order of a predetermined number of high level neural networks;

a grouping unit which classifies the selected faces into predetermined face groups, by comparing the first grade order and second grade order with a predetermined first threshold grade order and a predetermined second threshold grade order, respectively; and

a recognition result output unit which outputs a face belonging to a first face group having a first grade and a second grade bigger than the first threshold grade order and the second threshold grade order, respectively, as the recognition result.

9. The apparatus of claim 8, wherein the reliability value calculation unit converts each output value of the output layer neurons of each neural network, into a ratio of each output value of the neural network to the sum of the output values of a neural network.

10. The apparatus of claim 9, wherein the reliability value calculation unit converts each output value of the output layer neurons of each neural network according to the following equation:

$y_{\mathrm{\alpha <br><br>}}=\frac{\exp \left(10<br><br>w_{\mathrm{\alpha <br><br>}}\right)}{\underset{\mathrm{\beta <br><br>}}{\sum <br><br>}<br><br>\exp \left(10<br><br>w_{\mathrm{\beta <br><br>}}\right)}$

where W_α

denotes the actual output value of each neuron of a neural network and β

denotes the total number of output layer neurons.

11. The apparatus of claim 10, wherein the grade calculation unit gives a grade to the reliability value of each neural network, and calculates a first grade order, in which the grades of entire neural network for the selected faces are arranged in order from a high level neural network to a low level neural network, and a second grade order, in which the grades of the predetermined number of neural networks for the selected faces are arranged in order from a high level neural network to a low level neural network.

12. The apparatus of claim 10, wherein among a plurality of faces belonging to a first face group, the recognition result output unit outputs a face which is input by the reliability value calculation unit and has a bigger sum of reliability values, as the recognition result.

13. The apparatus of claim 9, wherein the reliability value calculation unit calculates the reliability value of a neural network according to the following equation by using the converted output value:

conf=Y_max,1(Y_max,1−

Y_max,2)

where Y_max,1 denotes the largest converted output value and Y_max,2 denotes the second largest converted output value.

14. The apparatus of claim 13, wherein the reliability value of the neural network is set as the reliability value of an output layer neuron having the largest converted output value, and the reliability values of the remaining output layer neurons are set to 0.

15. A face recognition apparatus which recognizes a face by using an artificial neural network, comprising:

an eigenpaxel selection unit which generates eigenpaxels representing characteristic patterns of a face and selects a predetermined number of eigenpaxels among the generated eigenpaxels;

an input image normalization unit which receives a face image and outputs a face image normalized in a predetermined format;

an eigenfiltering unit which filters the normalized input image with the respective selected eigenpaxels;

a sub-sampling unit which sub-samples the images filtered by respective eigenpaxels and outputs images of a size appropriate for neural network input;

a predetermined number of neural networks, each of which corresponds to one of the selected eigenpaxels, receives an image signal which is filtered with a corresponding eigenpaxel and sub-sampled, and outputs a face recognition result; and

a determination unit which receives the recognition result from each of the neural networks and outputs a final face recognition result of the input image.

16. A face recognition method for recognizing a face by using an artificial neural network, comprising:

(a) generating eigenpaxels representing characteristic patterns of a face and selecting a predetermined number of eigenpaxels among the generated eigenpaxels;

(b) training the neural network using the selected eigenpaxels;

(d) obtaining an eigenfiltered image containing patterns of face characteristics corresponding to the eigenfilters by eigenfiltering an input image with the respective selected eigenpaxels;

(e) performing face recognition on each pattern of face characteristics by outputting the eigenfiltered image to a neural network corresponding to each of the selected eigenpaxels; and

(f) analyzing the face recognition result of each neural network and outputting a final face recognition result of the input image.

17. The method of claim 16, wherein the step (a) further comprises:

dividing an arbitrary face image into predetermined blocks, and by applying a principal component analysis to each of the blocks, generating eigenpaxels indicating regular pattern characteristics of the face.

18. The method of claim 17, wherein the step (a) further comprises:

selecting a predetermined number of vectors among the generated eigenpaxels in order of decreasing eigenvalues of the vectors.

19. The method of claim 16, wherein in the step (b), the neural network is trained by a back propagation algorithm having added momentum, and at this time weighting coefficient (w_i) is updated by the following equation:

Δ

W_ij(t+1)=η

δ

_jo_i+α

Δ

W_ij(t)

where η

denotes a learning coefficient, j denotes the index of a current unit, i denotes the index of a unit preceding the unit j, δ

_j denotes the output error of unit j, O_i denotes the output of unit i, and α

denotes a constant determining the affect of momentum.

20. The method of claim 16, wherein the number of neurons in the output layer of the neural network is the same as the number of learned faces.

21. The method of claim 16, wherein in the step (d) the input image is eigenfiltered by performing convolution operation with moving each eigenpaxel for the input image.

22. The method of claim 21, wherein in the step (d) the input image is divided into predetermined blocks, and convolution operations are performed by moving the divided blocks over the eigenpaxels such that the blocks and eigenpaxels overlap to a predetermined part, and then an eigenfiltered image which is reduced to a predetermined size is obtained.

23. The method of claim 16, wherein the step (b) comprises:

(b1) normalizing the input learning image into a predetermined form;

(b2) eigenfiltering the normalized learning image with the eigenpaxels selected in the step (a) and obtaining an eigenfiltered image containing patterns of face characteristics corresponding to the eigenfilters; and

(b3) by sub-sampling each of the eigenfiltered images and inputting a neural network corresponding to each of the selected eigenpaxels, training each of the neural networks.

24. The method of claim 16, wherein the step (f) comprises:

(f1) converting the output value of the output layer neurons into a value in a predetermined range, calculating reliability values, and adding reliability values for the output layer neurons of each neural network;

(f2) selecting a face detected by a predetermined number of high level neural networks according to the reliability value;

(f3) giving a grade to the reliability value of each neural network, and for the selected face, calculating a first grade order of the entire neural network and a second grade order of a predetermined number of high level neural networks

(f4) classifying the selected faces into predetermined face groups, by comparing the first grade order and second grade order with a predetermined first threshold grade order and a predetermined second threshold grade order, respectively; and

(f5) outputting a face belonging to a first face group having a first grade and a second grade bigger than the first threshold grade order and the second threshold grade order, respectively, as the recognition result.

25. The method of claim 24, wherein the step (f1) further comprises:

converting each output value of the output layer neurons of each neural network, into a ratio of each output value of the neural network to the sum of the output values of a neural network.

26. The method of claim 25, wherein in the step (f1) each output value of the output layer neurons of each neural network is converted according to the following equation:

$y_{\mathrm{\alpha <br><br>}}=\frac{\exp \left(10<br><br>w_{\mathrm{\alpha <br><br>}}\right)}{\underset{\mathrm{\beta <br><br>}}{\sum <br><br>}<br><br>\exp \left(10<br><br>w_{\mathrm{\beta <br><br>}}\right)}$

where W_α

denotes the actual output value of each neuron of a neural network and β

denotes the total number of output layer neurons.

27. The method of claim 26, wherein the step (f3) comprises:

giving a grade to the reliability value of each neural network; and

calculating a first grade order, in which the grades of entire neural network for the selected faces are arranged in order from a high level neural network to a low level neural network, and a second grade order, in which the grades of the predetermined number of neural networks for the selected faces are arranged in order from a high level neural network to a low level neural network.

28. The method of claim 26, wherein in the step (g5), among a plurality of faces belonging to a first face group, a face which has a bigger sum of reliability values calculated in the step (g1) is output as the recognition result.

29. The method of claim 25, wherein the step (f1) further comprises:

calculating the reliability value of a neural network according to the following equation by using the converted output value;

conf=Y_max,1(Y_max,1−

Y_max,2)

where Y_max,1 denotes the largest converted output value and Y_max,2 denotes the second largest converted output value.

30. The method of claim 29, wherein the reliability value of each neural network is set as the reliability value of an output layer neuron having the largest converted output value, and the reliability values of the remaining output layer neurons are set to 0.

31. A tangible computer readable medium having embodied thereon a computer program for the face recognition method of claim 24.

32. A tangible computer readable medium having embodied thereon a computer program for the face recognition method of claim 16.

33. A face recognition method for recognizing a face by using an artificial neural network, comprising:

(a) generating eigenpaxels representing characteristic patterns of a face and selecting a predetermined number of eigenpaxels among the generated eigenpaxels;

(b) training the neural network using the selected eigenpaxels;

(c) normalizing an input face image into a predetermined form of the face image;

(d) obtaining an elgenfiltered image containing patterns of face characteristics corresponding to the elgenfilters by eigenfiltering the normalized face image with the respective selected eigenpaxels;

(e) obtaining an image of a size appropriate to a neural network input by sub-sampling each of the elgenfiltered images;

(f) performing face recognition on each pattern of face characteristics by outputting the eigenfiltered and sub-sampled image to a neural network corresponding to each of the selected eigenpaxels; and

(g) analyzing the face recognition result of each neural network and outputting a final face recognition result of the input image.

34. A tangible computer readable medium having embodied thereon a computer program for the face recognition method of claim 33.