Method and apparatus for red-eye detection

• US 20040233299A1
• Filed: 05/19/2003
• Published: 11/25/2004
• Est. Priority Date: 05/19/2003
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Ioffe, Sergey, Chinen, Troy T.
• Granted Patent: US 7,343,028 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

A method and an apparatus automatically detect red-eye regions in a photographic image. The method according to one embodiment accesses digital image data representing a color photographic image; detects red-eye candidate regions in the photographic image by processing the digital image data, the step of detecting red-eye candidate regions including calculating feature values for multiple positions of the photographic image, and identifying red-eye candidate regions based on the calculated feature values resulting from the calculating step; and performs face detection to determine whether an identified red-eye candidate region is associated with a face.

Claims

1. A method of automatically detecting red-eye regions in a photographic image, said method comprising:

accessing digital image data representing a color photographic image;

detecting red-eye candidate regions in said photographic image by processing said digital image data, said step of detecting red-eye candidate regions including

calculating feature values for multiple positions of said photographic image, and

identifying red-eye candidate regions based on said calculated feature values resulting from said calculating step; and

performing face detection to determine whether an identified red-eye candidate region is associated with a face.

2. The method of automatically deleting red-eye regions in a photographic image as recited in claim 1, wherein said step of performing face detection includes

calculating feature values for said photographic image, and

determining whether a detected red-eye candidate region is associated with a face based on said feature values.

3. The method of automatically detecting red-eye regions in a photographic image as recited in claim 1, further comprising:

detecting an outline for each identified red-eye candidate region associated with a face.

4. The method of automatically detecting red-eye regions in a photographic image as recited in claim 1, further comprising:

generating a red-eye detection mask identifying red-eye candidate pixels in said photographic image, including converting image data of said color photographic image into a color channel with increased contrast between object pixels and local background pixels.

5. The method of automatically detecting red-eye regions in a photographic image as recited in claim 3, further comprising:

generating a red-eye detection mask identifying red-eye candidate pixels in said photographic image, including converting image data of said color photographic image into a color channel with increased contrast between object pixels and local background pixels; and

combining results of said step of generating a red-eye detection mask and said step of detecting an outline to identify pixels in said digital image data to be subjected to red-eye reduction.

6. The method of automatically detecting red-eye regions in a photographic image as recited in claim 1, wherein said sub-step of calculating feature values for multiple positions of said photographic image includes converting image data of said color photographic image into a color channel with increased contrast between object pixels and local background pixels.

7. The method of automatically detecting red-eye regions in a photographic image as recited in claim 1, wherein said sub-step of calculating feature values for multiple positions of said photographic image calculates feature values from Y, Cb, and Cr image data.

8. The method of automatically detecting red-eye regions in a photographic image as recited in claim 1, wherein said sub-step of calculating feature values for multiple positions of said photographic image is performed by scanning photographic image data using a window of fixed size and repeating said scanning for scaled versions of said photographic image, such that said step of detecting red-eye candidate regions identifies red-eye candidate regions of various sizes.

9. The method of automatically detecting red-eye regions in a photographic image as recited in claim 1, wherein said sub-step of calculating feature values calculates a plurality of weak classifiers.

10. The method of automatically detecting red-eye regions in a photographic image as recited in claim 1, wherein said step of performing face detection computes discrete wavelet transform coefficients to distinguish faces from non-faces.

11. The method of automatically detecting red-eye regions in a photographic image as recited in claim 3, wherein said step of detecting an outline for an identified red-eye candidate region detects a top red-eye edge for multiple rotations of the corresponding image region.

12. The method of automatically detecting red-eye regions in a photographic image as recited in claim 1, wherein

said step of detecting red-eye candidate regions generates a set of red-eye candidate regions, and

said step of performing face detection eliminates candidates from said set of red-eye candidate regions found to not be associated with a face.

13. A apparatus for automatically detecting red-eye regions in a photographic image, said apparatus comprising:

an image data unit for providing digital image data representing a color photographic image;

a red-eye candidate detecting unit for detecting red-eye candidate regions in said photographic image by processing said digital image data, said red-eye candidate detecting unit detecting red-eye candidate regions by

calculating feature values for multiple positions of said photographic image, and

identifying red-eye candidate regions based on said calculated feature values; and

a face detection unit for performing face detection to determine whether a detected red-eye candidate region is associated with a face.

14. The apparatus according to claim 13, wherein said face detection unit performs face detection by

calculating feature values for said photographic image, and

determining whether a detected red-eye candidate region is associated with a face based on said feature values.

15. The apparatus according to claim 13, further comprising:

a red-eye outline detection unit for detecting an outline for each identified red-eye candidate region associated with a face.

16. The apparatus according to claim 13, further comprising:

a complementary red-eye pixel detecting unit for generating a red-eye detection mask identifying red-eye candidate pixels in said photographic image, said complementary red-eye pixel detecting unit converting image data of said color photographic image into a color channel with increased contrast between object pixels and local background pixels.

17. The apparatus according to claim 15, further comprising:

a complementary red-eye detecting unit for generating a red-eye detection mask identifying red-eye candidate pixels in said photographic image, said complementary red-eye pixel detecting unit converting image data of said color photographic image into a color channel with increased contrast between object pixels and local background pixels,

wherein the output of said complementary red-eye detecting unit is combined with the out of said red-eye outline detection unit to identify pixels in said digital image data to be subjected to red-eye reduction.

18. The apparatus according to claim 13, wherein said red-eye candidate detection unit converts image data of said color photographic image into a color channel with increased contrast between object pixels and local background pixels.

19. The apparatus according to claim 13, wherein said red-eye candidate detection unit calculates feature values from Y, Cb, and Cr image data.

20. The apparatus according to claim 13, wherein said red-eye detection unit scans photographic image data using a window of fixed size and repeats said scanning for scaled versions of said photographic image, thereby detecting red-eye candidate regions of various sizes.

21. The apparatus according to claim 13, wherein said red-eye detection unit calculates said feature values as a plurality of weak classifiers.

22. The apparatus according to claim 13, wherein said face detection unit computes discrete wavelet transform coefficients to distinguish faces from non-faces.

23. The apparatus according to 15, wherein said red-eye outline detection unit detects a top red-eye edge for multiple rotations of the corresponding image region.

24. The apparatus according to claim 13, wherein

said red-eye candidate detection unit generates a set of red-eye candidate regions, and

said face detection unit performs face detection to eliminate candidates from said set of red-eye candidate regions found to not be associated with a face.

25. The apparatus according to claim 13, wherein said red-eye candidate detection unit iteratively compares a response value, calculated as a combination of feature values, for an image region to intermediate threshold values to achieve early rejection of image regions not associated with a red-eye.

26. The apparatus according to claim 25, wherein said intermediate threshold values are obtained through a training session, using sample images of red-eye and non-red-eye image regions, of various size and orientation.

27. The apparatus according to claim 13, wherein said face detection unit iteratively compares a face detection response value for an image region to intermediate threshold value to achieve early rejection of image regions not associated with a face.

28. The apparatus according to claim 27, wherein said intermediate threshold values are obtained through a training session, using sample images of face and non-face regions, of various size and orientation.

29. A method of detecting red-eye pixels in a photographic image, said method comprising:

accessing digital image data representing a color photographic image;

normalizing each of a plurality of pixels of said digital image data based on pixels in the neighborhood of the pixel being normalized;

applying a linear discriminant function to the normalized pixels; and

determining whether each of a plurality of normalized pixels is a red-eye pixel or not based on the result of applying the linear discriminant function.

30. The method according to claim 29, wherein said step of normalizing normalizes each of the plurality of pixels based on a mean value of neighborhood pixels.

31. The method according to claim 30, wherein said step of normalizing subtracts the mean value of neighborhood pixels from the pixel being normalized.

32. The method according to claim 30, wherein said step of normalizing divides a value of the pixel being normalized by the mean value of neighborhood pixels.

33. The method according to claim 29, wherein said step linear discriminant function is a Fisher linear discriminant function.

34. An apparatus for detecting red-eye pixels in a photographic image, said method comprising:

an image data input for providing digital image data representing a color photographic image; and

a red-eye pixel detecting unit for detecting red-eye pixels by

normalizing each of a plurality of pixels of said digital image data based on pixels in the neighborhood of the pixel being normalized;

applying a linear discriminant function to the normalized pixels; and

determining whether each of a plurality of normalized pixels is a red-eye pixel or not based on the result of applying the linear discriminant function.

35. The apparatus according to claim 34, wherein said red-eye pixel detecting unit normalizes each of the plurality of pixels based on a mean value of neighborhood pixels.

36. The apparatus according to claim 35, wherein said red-eye pixel detecting unit normalizes pixels by subtracting the mean value of neighborhood pixels from the pixel being normalized.

37. The apparatus according to claim 35, wherein said red-eye pixel detecting unit normalizes pixels by dividing a value of the pixel being normalized by the mean value of neighborhood pixels.

38. The apparatus according to claim 34, wherein said step linear discriminant function is a Fisher linear discriminant function.