Automated detection and correction of eye color defects due to flash illumination

• US 5,748,764 A
• Filed: 04/03/1995
• Issued: 05/05/1998
• Est. Priority Date: 07/19/1993
• Status: Expired due to Fees

• Current Assignee: Eastman Kodak Company
• Original Assignee: Eastman Kodak Company
• Inventors: Benati, Paul J., Cosgrove, Patrick A., Gray, Robert T.
• Primary Examiner(s): Couso, Jose L.
• Assistant Examiner(s): Bella, Matthew C.
• Application Number:
  US08/415,864
• Time in Patent Office: 1,128 Days
• Field of Search: 382/169, 382/149, 382/162, 382/117, 382/299, 382/167, 358/453, 358/457, 358/461
• US Class Current: 382/117
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

A method of determining and correcting for eye color defects in an image due to flash illumination includes determining whether an eye color defect group candidate corresponds to a defective eye base based on group shape, coloration and brightness.

Claims

1. A method of detecting for eye color defects of a subject in a digital image due to flash illumination, comprising the steps of:

a) defining a spatial region within the digital image in which one or more eye color defects may exist, which region includes at least a portion of the subject'"'"'s head;

b) sampling pixels within such spatial region for their color content and comparing each such sample pixel with a plurality of threshold values which are representative of eye color defects to identify possible eye color defect pixels;

c) segmenting the identified possible eye color defective pixels into one or more spatially contiguous groups;

d) calculating a first score for each pixel of each segmented group and for each group based upon a plurality of features including group size, group shape, coloration, and brightness to identify eye color defect group candidates;

e) selecting a seed pixel based on its score from each identified eye color defect group candidate and determining all the neighboring pixels which are within a predetermined score range of their neighboring pixels and those pixels in a group which represent a significant pixel score transition so that the determined transitions identify the outer boundary of an eye color defect group candidate; and

f) calculating a second score for each pixel for each eye color defect group candidate based on a plurality of features including group size, group shape, coloration, and brightness to determine an actual eye color defect group.

2. The method of claim 1 further including correcting of the defective color in each eye color defect group.

3. The method of claim 2 wherein the threshold value represents minimum and maximum allowed values consistent with the expected color of a eye color defect group.

4. The method of claim 1 wherein the group size feature is area, the group shape feature is eccentricity, the coloration and brightness features are hue, saturation, lightness, and red to maximum (green, blue) ratio.

5. The method of claim 1 wherein the scores per pixel in steps d) and f) are equal to the product of the scores of individual features, and the scores of individual features are weighted functions of the feature values of the pixels.

6. The method of claim 5 wherein the group score is an average of all the pixels.

7. A method of detecting for eye color defects of a subject in a digital image due to flash illumination, comprising the steps of:

a) defining a spatial region within the digital image in which one or more eye color defects may exist, which region includes at least a portion of the subject'"'"'s head;

b) sampling pixels within such spatial region for their color content and comparing each such sampled pixel with a plurality of threshold values which are representative of eye color defects to identify possible eye color defect pixels;

c) segmenting the identified possible eye color defect pixels into one or more spatially contiguous groups;

d) calculating a first score for each pixel of each segmented group and for each group based upon a plurality of features including group size, group shape, coloration, and brightness to identify eye color defect group candidates;

e) selecting a seed pixel based on its score from each identified eye color defect group candidate and determining all the neighboring pixels which are within a predetermined score range of their neighboring pixels and those pixels in a group which represent a significant pixel score transition so that the determined transitions identify the outer boundary of an eye color defect group candidate;

f) calculating a second score for each pixel for each eye color defect group candidate based on a plurality of features including group size, group shape, coloration, and brightness;

g) selecting the best group score of an eye color defect group candidate and comparing it relative to a predetermined threshold group score to determine whether a first eye color defect group is present and identifying it as corresponding to the most likely eye; and

h) determining whether a second actual eye color defect group is present based on whether the area of one of the color defect group candidate'"'"'s eye is within a predetermined threshold ratio of the area of the first actual eye color defect group corresponding to the most likely eye, and whether this group candidate is within a predetermined threshold angular subtense of the most likely eye along the predetermined horizontal axis of the subject'"'"'s head.

8. The method of claim 7 further including correcting of the defective color in each eye color defect group.

9. The method of claim 8 wherein the threshold value represents minimum and maximum allowed values consistent with the expected color of a eye color defect group.

10. The method of claim 7 wherein the group size feature is area, the group shape feature is eccentricity, the coloration and brightness features are hue, saturation, lightness, and red to maximum (green, blue) ratio.

11. The method of claim 7 wherein the scores per pixel in steps d) and f) are equal to the product of the scores of individual features, and the scores of individual features are weighted functions of the feature values of the pixels.

12. The method of claim 11 wherein the group score is an average of all the pixels.

13. A method of detecting and correcting for eye color defects of a subject in a digital image due to flash illumination, comprising the steps of:

a) defining a spatial region within the digital image in which one or more eye color defects may exist, which region includes at least a portion of the subject'"'"'s head;

b) sampling pixels within such spatial region for their color content and comparing each such sampled pixel with a plurality of threshold values which are representative of eye color defects to identify possible eye color defect pixels;

c) segmenting the identified possible eye color defect pixels into one or more spatially contiguous groups;

d) calculating a first score for each pixel of each segmented group and for each group based upon a plurality of features including group size, group shape, coloration, and brightness to identify eye color defect group candidates;

e) selecting a seed pixel based on its score from each identified eye color defect group candidate and determining all the neighboring pixels which are within a predetermined score range of their neighboring pixels and those pixels in a group which represent a significant pixel score transition so that the determined transitions identify the outer boundary of an eye color defect group candidate;

f) calculating a second score for each pixel for each eye color defect group candidate based on a plurality of features including group size, group shape, coloration, and brightness;

g) selecting the best group score of an eye color defect group candidate and comparing it relative to a predetermined threshold group score to determine whether a first eye color defect group is present and identifying it as corresponding to the most likely eye; and

h) determining whether a second actual eye color defect group is present based on whether the area of one of the color defect group candidate'"'"'s eye is within a predetermined threshold ratio of the area of the first actual eye color defect group corresponding to the most likely eye, and whether this group candidate is within a predetermined threshold angular subtense of the most likely eye along the predetermined horizontal axis of the subject'"'"'s head;

i) correcting the defective color in each eye color defect group by;

(i) determining the correct resolution and whether each pixel at the corrected resolution is an eye color defect pixel;

(ii) categorizing the eye color defect pixels at the corrected resolution into either body, border, or glint categories; and

(iii) correcting the eye color defect pixels in the body, border, or glint categories.

14. The method of claim 13 wherein the threshold value represents minimum and maximum allowed values consistent with the expected color of a eye color defect group.

15. The method of claim 13 wherein the group size feature is area, the group shape feature is eccentricity, the coloration and brightness features are hue, saturation, lightness, and red to maximum (green, blue) ratio.

16. The method of claim 13 wherein the scores per pixel in steps d) and f) are equal to the product of the scores of individual features, and the scores of individual features are weighted functions of the feature values of the pixels.

17. The method of claim 16 wherein the group score is an average of all the pixels.

18. The method of claim 13 wherein step i) (i) includes determining at a lower resolution a correction factor for each low resolution pixel which is a function of the number of eye color defect pixels at the original resolution disposed at the same image location.

19. The method of claim 13 wherein step i) (i) includes categorizing at a higher resolution each original eye color defect pixel into eye color defect pixels and non-eye color defect pixels.

20. The method of claim 13 including desaturating and reducing the illuminance for each body, border, and glint pixel.