Illumination Detection Using Classifier Chains

• US 20080219517A1
• Filed: 02/27/2008
• Published: 09/11/2008
• Est. Priority Date: 03/05/2007
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Gangea, Mihnea, Corcoran, Peter, Blonk, Leendert
• Granted Patent: US 8,503,800 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images

Abstract

A face illumination normalization method includes acquiring a digital image including a face that appears to be illuminated unevenly. One or more uneven illumination classifier programs are applied to the face data to determine the presence of the face within the digital image and/or the uneven illumination condition of the face. The uneven illumination condition may be corrected to thereby generate a corrected face image appearing to have more uniform illumination, for example, to enhance face recognition.

Claims

1. A face illumination normalization method, comprising:

(a) acquiring a digital image including data corresponding to a face that appears to be illuminated unevenly;

(b) applying one or more uneven illumination classifier programs to the face data;

(c) identifying the face data as corresponding to said face within the digital image;

(d) determining an uneven illumination condition for the face also as a result of the applying of the one or more uneven illumination classifier programs;

(e) correcting the uneven illumination condition of the face based on the determining to thereby generate a corrected face image appearing to have more uniform illumination; and

(f) electronically storing, transmitting, applying a face recognition program to, editing, or displaying the corrected face image, or combinations thereof.

2. The method of claim 1, further comprising applying a face recognition program to the corrected face image.

3. The method of claim 1, wherein the detecting of the face and the determining of the uneven illumination condition of the face are performed simultaneously.

4. The method of claim 1, further comprising applying a set of feature detector programs to reject non-face data from being identified as face data.

5. The method of claim 1, further comprising applying a front illumination classifier program to the face data.

6. The method of claim 5, further comprising determining an illumination condition based on acceptance of the face data by one of the classifier programs.

7. The method of claim 6, wherein the digital image is one of multiple images in a series that include said face, and wherein said correcting is applied to a different image in the series than said digital image within which the illuminating condition is determined.

8. The method of claim 1, wherein said uneven illumination classifier programs comprise a top illumination classifier, a bottom illumination classifier, and one or both of right and left illumination classifiers.

9. The method of claim 8, further comprising applying a front illumination classifier program to the face data.

10. The method of claim 1, wherein the applying comprises applying at least two full classifier sets after determining that no single illumination condition applies and that the face data is not rejected as a face.

11. A face detection method, comprising:

(a) acquiring a digital image

(b) extracting a sub-window from said image

(c) applying two or more shortened face detection classifier cascades, trained to be selectively sensitive to a characteristic of a face region,

(d) based on the applying, determining a probability that a face with a certain form of the characteristic is present within the sub-window;

(e) based on the determining, applying an extended face detection classifier cascade trained for sensitivity to said form of said characteristic;

(f) providing a final determination that a face exists within the image sub-window; and

(g) repeating steps (b)-(e) one or more times for one or more further sub-windows from the image or one or more further characteristics, or both.

12. The method of claim 11, wherein the characteristic or characteristics comprise a directional illumination of the face region, an in-plane rotation of the face region, a 3D pose variation of the face region. a degree of smile, a degree of eye-blinking, a degree of eye-winking, a degree of mouth opening, facial blurring, eye-defect, facial shadowing, facial occlusion, facial color, or facial shape, or combinations thereof.

13. The method of claim 11, wherein the characteristic comprises a directional illumination, and the method further comprises determining an uneven illumination condition by applying one or more uneven illumination classifier cascades.

14. The method of claim 13, further comprising applying a front illumination classifier cascade.

15. The method of claim 14, further comprising determining an illumination condition of a face within a sub-window based on acceptance by one of the classifier cascades.

16. The method of claim 15, wherein the digital image is one of multiple images in a series that include the face, and the method further comprises correcting an uneven illumination condition of the face within a different image in the series than said digital image within which the illuminating condition is determined.

17. The method of claim 13, wherein said uneven illumination classifier cascades comprise a top illumination classifier, a bottom illumination classifier, and one or both of right and left illumination classifiers.

18. An improved face detection method, comprising:

(a) acquiring a digital image

(b) extracting a sub-window from said image

(c) applying two or more shortened face detection classifier cascades, trained to be selectively sensitive to directional facial illumination;

(d) based on the applying, determining a probability that a face having a certain form of directional facial illumination is present within the sub-window;

(e) based on the determining, applying an extended face detection classifier cascade trained for sensitivity to said form of directional face illumination;

(f) providing a final determination that a face exists within the image sub-window

(g) repeating steps (b)-(e) one or more times for one or more further sub-windows from the image or one or more further directional facial illuminations, or both

19. The method of claim 18, wherein the digital image is one of multiple images in a series that include the face, and the method further comprises correcting an uneven illumination condition of the face within a different image in the series than said digital image within which the illuminating condition is determined.

20. The method of claim 18, wherein said uneven illumination classifier cascades comprise a top illumination classifier, a bottom illumination classifier, and one or both of right and left illumination classifiers.

21. The method of claim 20, further comprising applying a front illumination classifier cascade.

22. The method of claim 21, further comprising determining an illumination condition of a face within a sub-window based on acceptance by one of the classifier cascades.

23. A digital image acquisition device including an optoelectonic system for acquiring a digital image, and a digital memory having stored therein processor-readable code for programming the processor to perform a face illumination normalization method, wherein the method comprises:

(a) acquiring a digital image including data corresponding to a face that appears to be illuminated unevenly;

(b) applying one or more uneven illumination classifier programs to the face data;

(c) identifying the face data as corresponding to said face within the digital image;

(d) determining an uneven illumination condition for the face also as a result of the applying of the one or more uneven illumination classifier programs;

(e) correcting the uneven illumination condition of the face based on the determining to thereby generate a corrected face image appearing to have more uniform illumination; and

(f) electronically storing, transmitting, applying a face recognition program to, editing, or displaying the corrected face image, or combinations thereof.

24. The device of claim 23, wherein the method further comprises applying a face recognition program to the corrected face image.

25. The device of claim 23, wherein the detecting of the face and the determining of the uneven illumination condition of the face are performed simultaneously.

26. The device of claim 23, wherein the method further comprises applying a set of feature detector programs to reject non-face data from being identified as face data.

27. The device of claim 23, wherein the method further comprises applying a front illumination classifier program to the face data.

28. The device of claim 27, wherein the method further comprises determining an illumination condition based on acceptance of the face data by one of the classifier programs.

29. The device of claim 28, wherein the digital image is one of multiple images in a series that include said face, and wherein said correcting is applied to a different image in the series than said digital image within which the illuminating condition is determined.

30. The device of claim 23, wherein said uneven illumination classifier programs comprise a top illumination classifier, a bottom illumination classifier, and one or both of right and left illumination classifiers.

31. The device of claim 30, wherein the method further comprises applying a front illumination classifier program to the face data.

32. The device of claim 23, wherein the applying comprises applying at least two full classifier sets after determining that no single illumination condition applies and that the face data is not rejected as a face.

33. A digital image acquisition device including an optoelectonic system for acquiring a digital image, and a digital memory having stored therein processor-readable code for programming the processor to perform a face illumination normalization method, wherein the method comprises:

(a) acquiring a digital image

(b) extracting a sub-window from said image

(c) applying two or more shortened face detection classifier cascades, trained to be selectively sensitive to a characteristic of a face region,

(d) based on the applying, determining a probability that a face with a certain form of the characteristic is present within the sub-window;

(e) based on the determining, applying an extended face detection classifier cascade trained for sensitivity to said form of said characteristic;

(f) providing a final determination that a face exists within the image sub-window; and

(g) repeating steps (b)-(e) one or more times for one or more further sub-windows from the image or one or more further characteristics, or both.

34. The device of claim 33, wherein the characteristic or characteristics comprise a directional illumination of the face region, an in-plane rotation of the face region, a 3D pose variation of the face region. a degree of smile, a degree of eye-blinking, a degree of eye-winking, a degree of mouth opening, facial blurring, eye-defect, facial shadowing, facial occlusion, facial color, or facial shape, or combinations thereof.

35. The device of claim 33, wherein the characteristic comprises a directional illumination, and the method further comprises determining an uneven illumination condition by applying one or more uneven illumination classifier cascades.

36. The device of claim 35, wherein the method further comprises applying a front illumination classifier cascade.

37. The device of claim 36, wherein the method further comprises determining an illumination condition of a face within a sub-window based on acceptance by one of the classifier cascades.

38. The device of claim 37, wherein the digital image is one of multiple images in a series that include the face, and the method further comprises correcting an uneven illumination condition of the face within a different image in the series than said digital image within which the illuminating condition is determined.

39. The device of claim 33, wherein said uneven illumination classifier cascades comprise a top illumination classifier, a bottom illumination classifier, and one or both of right and left illumination classifiers.

40. A digital image acquisition device including an optoelectonic system for acquiring a digital image, and a digital memory having stored therein processor-readable code for programming the processor to perform a face illumination normalization method, wherein the method comprises:

(a) acquiring a digital image

(b) extracting a sub-window from said image

(c) applying two or more shortened face detection classifier cascades, trained to be selectively sensitive to directional facial illumination;

(d) based on the applying, determining a probability that a face having a certain form of directional facial illumination is present within the sub-window;

(e) based on the determining, applying an extended face detection classifier cascade trained for sensitivity to said form of directional face illumination;

(f) providing a final determination that a face exists within the image sub-window

(g) repeating steps (b)-(e) one or more times for one or more further sub-windows from the image or one or more further directional facial illuminations, or both

41. The device of claim 40, wherein the digital image is one of multiple images in a series that include the face, and the method further comprises correcting an uneven illumination condition of the face within a different image in the series than said digital image within which the illuminating condition is determined.

42. The device of claim 40, wherein said uneven illumination classifier cascades comprise a top illumination classifier, a bottom illumination classifier, and one or both of right and left illumination classifiers.

43. The device of claim 42, wherein the method further comprises applying a front illumination classifier cascade.

44. The device of claim 43, wherein the method further comprises determining an illumination condition of a face within a sub-window based on acceptance by one of the classifier cascades.