Apparatus for and method of feature extraction for image recognition

• US 20060008150A1
• Filed: 07/19/2004
• Published: 01/12/2006
• Est. Priority Date: 07/07/2004
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Wang, Dejun, Kee, Seokcheol, Zhao, Jiali, Ren, Haibing
• Granted Patent: US 7,715,659 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

An apparatus for and method of performing a most informative feature extraction (MIFE) method in which a facial image is separated into sub-regions, and each sub-region makes individual contribution for performing facial recognition. Specifically, each sub-region is subjected to a sub-region based adaptive gamma (SadaGamma) correction or sub-region based histogram equalization (SHE) in order to account for different illuminations and expressions. A set of reference images is also divided into sub-regions and subjected to the SadaGamma correction or SHE. A comparison is made between the each corrected sub-region and each corresponding sub-region of the reference images. Based upon the comparisons made individually for the sub-regions of the facial image, one of the stored reference images having the greatest correspondence is chosen. While usable individually, using the MIFE and/or SadaGamma correction or SHE together achieves a lower error ratio in face recognition under different expressions, illuminations and occlusions.

Claims

1. A system for performing image recognition, the system comprising:

an image input device which inputs a first image;

a database having a set of reference images; and

a comparison unit which receives the first image having been divided into first image sub-regions, compares the first image sub-regions with corresponding reference image sub-regions of the reference images, and determines based on the comparison which of the reference images has a greatest correlation to the first image based on the comparisons between the reference image sub-regions and the first image sub-regions.

2. The system of claim 1, wherein the comparison unit, after each comparison of the first and corresponding reference image sub-regions, stores an identification of which of the reference images has the sub-region with a greatest correspondence with the first image sub-region, and after a predetermined number of the first and reference image sub-regions have been compared, reviews the stored identifications to determine which of the reference images has a greatest number of reference image sub-regions corresponding to the first image sub-regions in order to determine which of the reference images has the greatest correlation to the first image.

3. The system of claim 1, wherein:

for each of i=1 to D, the comparison unit compares an i^th first image sub-region with a corresponding i^th reference image sub-region of each of the reference images, and determines which of the reference images includes the i^th reference image sub-region having a greatest correspondence to the i^th first image sub-region, and

D is a number of first image sub-regions.

4. The system of claim 1, further comprising a correction unit which compares each of the first image sub-regions with a mean for the corresponding first image sub-region to remove an influence of illumination and/or occlusion in each of the first image sub-regions to produce a corrected first image, wherein the comparison unit compares the corrected first image sub-regions with the reference image sub-regions to determine which of the reference images has the greatest correlation.

5. The system of claim 1, wherein the image input device is a camera which inputs the first image to the comparison unit.

6. The system of claim 1, wherein the image input device is a scanner which digitizes the first image and inputs the first image to the comparison unit.

7. The system of claim 1, wherein the image input device is a memory reader which receives a memory storing the first image and inputs the first image read from the memory to the comparison unit.

8. The system of claim 7, wherein the memory is included in an identification card.

9. The system of claim 8, wherein the identification card comprises a travel document card.

10. The system of claim 1, wherein the comparison unit comprises a processor which compares the first image sub-regions with corresponding reference image sub-regions of the reference images, and determines based on the comparison which of the reference images has the greatest correlation to the first image based on the comparisons between the reference image sub-regions and the first image sub-regions.

11. The system of claim 1, wherein the comparison unit comprises processors, where each of the processors compares at least one of the first image sub-regions with corresponding reference image sub-regions of the reference images, and determines based on the comparison which of the reference images has the greatest correlation to the first image based on the comparisons between the reference image sub-regions and the first image ub-regions.

12. The system of claim 4, wherein the comparison unit and the correction unit comprises a processor.

13. The system of claim 4, wherein the comparison unit comprises a first processor and the correction unit comprise a second processor other than the first processor.

14. The system of claim 4, wherein the comparison unit further stores the first image having the first image sub-regions in the database.

15. The system of claim 1, wherein the database comprises a storage medium disposed in a computer having the comparison unit.

16. The system of claim 1, wherein the database comprises a storage medium disposed in a computer other than a computer having the comparison unit.

17. The system of claim 1, wherein the first image is of a non-rigid surface.

18. The system of claim 17, wherein the non-rigid surface comprises a surface on a body element.

19. The system of claim 18, wherein the body element is a face.

22. The system of claim 2, wherein the predetermined number of the first and reference image sub-regions comprises a total number of the first image sub-regions.

23. The system of claim 2, wherein the predetermined number of the first and reference image sub-regions is less than a total number of the first image sub-regions.

24. The system of claim 2, wherein the predetermined number of the first and reference image sub-regions is a number of sub-regions at which, based on comparisons of the first and reference image sub-regions made at the predetermined number, one of the reference images is determined to have the greatest correlation if it is a statistical impossibility for another of the reference images to have the greatest correlation to the first image.

25. The system of claim 1, wherein the comparison unit outputs identification information for the reference image having the greatest correlation.

26. The system of claim 25, wherein the identification information comprises a name of a person in the reference image.

27. The system of claim 1, wherein the comparison unit compares the first and reference images sub-regions by, for a j^th image sub-region of the first image and the reference images, calculating a label I for the j^th sub-region as

28. The system of claim 1, further comprising a feature identification unit which identifies at least one predetermined feature of the first image so as to normalize the first image as compared to the reference images, wherein the comparison unit receives the normalized first image having been divided into first image sub-regions.

29. The system of claim 4, further comprising a correction unit which compares each of the reference image sub-regions with the mean for the corresponding reference image sub-region to remove an influence of illumination and/or occlusion in each of the reference image sub-regions to produce corrected reference images, wherein the comparison unit compares the corrected first image sub-regions with the corrected reference image sub-regions to determine which of the reference images has the greatest correlation.

30. The system of claim 4, wherein the database stores corrected reference images in which each of the reference image sub-regions has been compared with the mean for the corresponding reference image sub-region to remove an influence of illumination and/or occlusion in each of the reference image sub-regions, wherein the comparison unit compares the corrected first image sub-regions with the corrected reference image sub-regions to determine which of the reference images has the greatest correlation.

31. The system of claim 4, wherein:

the mean comprises an average value for a representative number of each reference image sub-region calculated as follows;

$\frac{1}{N}<br><br>\underset{x_i<br><br>\mathrm{\epsilon <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{trainingset}}{\sum <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>x_i,$

N is a number of images of the training set, and

the training set comprises ones of the reference images.

36. The system of claim 29, wherein the processor further performs a histogram equalization for each of the first image sub-regions to provide the corrected first image sub-regions.

20. The system of claim 20, wherein each of the first image sub-regions has a height h and a width w, the first image has a height H and a width W, and a number of first image sub-regions is int(H/h)*int(W/w).

21. The system of claim 20, wherein a size of each of the first image sub-regions is determined experimentally.

32. A system for performing image recognition, the system comprising:

an image input device which inputs a first image;

a database having a set of reference images; and

a correction unit which receives the first image having been divided into first image sub-regions, compares each of the first image sub-regions with a mean for the corresponding first image sub-region to remove an influence of illumination and/or occlusion in each of the first image sub-regions to produce corrected first image sub-regions, and produces a corrected first image based on the corrected first image sub-regions.

33. The system of claim 32, wherein the correction unit further performs an adaptive gamma correction for each of the first image sub-regions in order to provide the corrected first image sub-regions.

34. The system of claim 33, wherein:

the correction unit

selects a Gamma parameter for each of the first image sub-regions by minimizing a distance between a pair wise k^th first image sub-region and k^th sub-region of a mean image as follows,

I_xy^k′

=G(I_xyk;

γ

^k*),

computes γ

^k* as follows,

${\mathrm{\gamma <br><br>}}^{k*}=\arg <br><br>\text{\hspace{1em}<br><br>}<br><br>\underset{\mathrm{\gamma <br><br>}}{\min }<br><br>\underset{x,y}{\sum <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{dis}<br><br>\left(G<br><br>\left(I_{\mathrm{xy}}^k;<br><br>{\mathrm{\gamma <br><br>}}^k\right)-I_{0<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{xy}}^k\right),\mathrm{and}$

performs Gamma correction as follows;

$G<br><br>\left(I_{\mathrm{xy}};<br><br>\mathrm{\gamma <br><br>}\right)=c·<br><br>I_{\mathrm{xy}}^{\frac{1}{\mathrm{\gamma <br><br>}}},$

I^k is the k^th first image sub-region of the first image,

I^k_o is the k^th sub-region of the mean image,

I is the first image,

I_o is the mean image, and

c is a constant.

35. The system of claim 34, wherein:

the mean image comprises an average value for a representative number of each reference image sub-region calculated as follows;

$\frac{1}{N}<br><br>\underset{x_i<br><br>\mathrm{\epsilon <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{trainingset}}{\sum <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>x_i,$

and

N is a number of images of the training set of reference images.

37. The system of claim 32, wherein the image input device is a camera which inputs the first image to the comparison unit.

38. The system of claim 32, wherein the image input device is a scanner which digitizes the first image and inputs the first image to the comparison unit.

39. The system of claim 32, wherein the image input device is a memory reader which receives a memory which stores the first image and inputs the first image read from the memory to the comparison unit.

40. The system of claim 32, further comprising a writing unit which records the corrected first image on a storage medium as part of a database of corrected images.

41. The system of claim 32, further comprising a correlation system comprising:

a database having a set of reference images; and

a comparison unit which receives the corrected first image from the correction unit, compares the first image with the reference images, and determines based on the comparison which of the reference images has a greatest correlation to the first image based on the comparisons.

42. The system of claim 41, wherein in order to determine the greatest correlation, the comparison unit performs one of a Principal Components Analysis (PCA), a Linear Discriminant Analysis (LDA), and a correlation method.

43. A method of determining a correspondence between an obtained image divided into obtained image sub-regions and a set of reference images divided into corresponding reference image sub-regions, comprising:

determining a greatest correlation between one of the obtained image sub-regions and corresponding one reference image sub-regions;

determining another greatest correlation between another one of the obtained image sub-regions and corresponding another reference image sub-regions; and

selecting one of the reference images based upon the one and another greatest correlations.

44. The method of claim 43, further comprising determining the remaining greatest correlations between the remaining ones of the obtained image sub-regions and the corresponding remaining image sub-regions, wherein the selecting comprises determining which one of the reference images is most frequently determined to have the greatest correlation for the obtained image sub-regions.

45. A method of removing an influence of illumination and/or occlusions of an obtained image, comprising:

for each sub-region of an obtained image, determining a sub-region factor which minimizes a difference between the sub-region and a mean for the sub-region; and

applying the sub-region factor to the corresponding sub-region for each of the sub-regions such that the entire obtained image is corrected.

46. The method of claim 45, wherein the factor comprises an adaptive gamma correction which minimizes a difference between the corresponding sub-region and the mean associated with the sub-region.

47. The method of claim 45, further comprising storing the corrected obtained image.

48. The method of claim 45, further comprising using the corrected obtained image to determine a correlation between the corrected obtained image and another image.

49. The method of claim 48, wherein the using the corrected obtained image further comprises:

determining a correlation between one of the sub-regions of the corrected obtained image and a corresponding one sub-region of the another image;

determining another correlation between another one of the corrected obtained sub-regions and a corresponding another sub-region of the another image; and

determining an overall correspondence between the corrected obtained image and the another image based upon the one and another correlations.

50. The method of claim 45, further wherein the using the corrected obtained image to determine the correlation comprises using a Principal Components Analysis.

51. The method of claim 45, further wherein the using the corrected obtained image to determine the correlation comprises using a Linear Discriminant Analysis.

52. At least one computer readable medium encoded with processing instructions for implementing a method of 43 performed by at least one computer.

53. At least one computer readable medium encoded with processing instructions for implementing a method of 45 performed by at least one computer.