Red-eye detection based on red region detection with eye confirmation

• US 6,895,112 B2
• Filed: 02/13/2001
• Issued: 05/17/2005
• Est. Priority Date: 02/13/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Platt, John C., Yan, Jie, Zhang, Hong-Jiang, Chen, Xiangrong, Chen, Tong-Xian
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US09/783,505
• Publication Number:
  US 20020136450A1
• Time in Patent Office: 1,554 Days
• Field of Search: 382224-227, 382115-118, 382162-167, 382274-275, 3482241-225, 348576-580, 358518-523
• US Class Current: 382/167
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

Red-eye detection based on red region detection with eye confirmation initially identifies pixels that correspond to the color of red-eye within an image. A determination is then made as to whether these identified pixels and surrounding areas are part of an eye or not part of an eye. Those identified pixels that are determined to be part of an eye are the detected red-eye regions.

Claims

1. A system comprising:

a region detection module to detect regions of an image that include pixels of a particular one or more colors, wherein the region detection module comprises;

a pixel identifier that is trained to colors associated with red-eye, and wherein the pixel identifier is to identify pixels within the region having colors that are close to the colors associated with red-eye;

a pixel grouper coupled to receive the identified pixels from the pixel identifier and group together adjacent pixels;

a filter to receive an indication of the groups of pixels from the pixel grouper and to identify, based on a set of rules, which of the groups are to be output to the eye confirmation module as detected regions; and

an eye confirmation module to receive the detected regions from the region detection module and identify, for each of the detected regions, whether the detected region is part of an eye.

2. A system as recited in claim 1, wherein one of the rules is:

if greater than a threshold amount of pixels in the image are the particular one or more colors then none of the pixel groups are detected regions.

3. A system as recited in claim 1, wherein one of the rules is:

if the group is a single pixel then the group is not a detected region.

4. A system as recited in claim 1, wherein one of the rules is:

if the group is more rectangular than circular then the group is not a detected region.

5. A system as recited in claim 1, wherein one of the rules is:

if the group has an aspect ratio substantially different from a circle then the group is not a detected region.

6. A system as recited in claim 1, wherein the image is a digital image.

7. A system as recited in claim 1, wherein the image is a digitized version of a film image.

8. A system as recited in claim 1, wherein the eye confirmation module is to determine, for each of the detected regions, whether the detected region is part of a human eye.

9. A system as recited in claim 1, wherein the system is implemented in a computer.

10. A system as recited in claim 1, wherein the system is implemented in a camera.

11. A system as recited in claim 1, wherein the system is implemented in an image printing device.

12. A system as recited in claim 1, further comprising:

a skin color module to detect areas of skin color in the image and indicate the detected areas to the region detection module; and

wherein the region detection module is to search within the detected areas to detect regions that include pixels of the particular one or more colors.

13. A system as recited in claim 1, wherein the eye confirmation module comprises an SVM (Support Vector Machine) classifier to classify each of the detected regions as either part of an eye or not part of an eye.

14. A system as recited in claim 1, wherein the eye confirmation module comprises a multi-scale classifier to apply a window to the image and compare pixels within the window to an eye template, to alter the scale of the image, and then to repeat the application of the window to the scale-altered image and comparison to the eye template.

15. A system as recited in claim 1, wherein the pixel grouper further to group together pixels within a threshold distance of one another.

16. A method comprising:

receiving an image;

searching a set of areas of the image for candidate pixels of one or more colors, wherein the one or more colors comprise colors corresponding to red-eye;

combining the candidate pixels into a set of one or more pixel groups, wherein the combining comprises combining candidate pixels into the same group if the candidate pixels are adjacent one another;

for each pixel group in the set of one or more pixel groups, classifying the pixel group as being part of an eye or not part of an eye;

checking whether flash was used in capturing the image, wherein checking whether flash was used in capturing the image comprises checking whether a flash used flag is set in a header corresponding to the image; and

performing the searching, combining, and classifying only if flash was used in capturing the image.

17. A method as recited in claim 16, wherein the receiving comprises receiving the image from a camera.

18. A method as recited in claim 16, further comprising:

identifying areas within the image that are skin colored; and

using the identified areas as the set of areas.

19. A method as recited in claim 16, further comprising for each pixel group in the set of one or more pixel groups, prior to classifying the pixel group:

identifying the geometric shape of the pixel group;

determining whether the geometric shape is similar to the shape of an eye; and

leaving the pixel group as part of the set of one or more pixel groups if the geometric shape is similar to the shape of an eye, and otherwise removing the pixel group from the set.

20. A method as recited in claim 19, wherein the shape is similar to the shape of an eye if the geometric shape is more circular than rectangular and if an aspect ratio of the geometric shape differs from the aspect ratio of a circle by not greater than a particular amount.

21. A method as recited in claim 16, wherein the classifying comprises:

applying a window to the image and comparing pixels within the window to an eye template;

altering the scale of the image; and

repeating the applying and comparing based on the scale-altered image.

22. A method as recited in claim 21, wherein the repeating comprises repeating the applying and comparing based on the scale-altered image without altering the size of the eye template.

23. A method as recited in claim 16, further comprising removing, based on a set of rules, groups from the set of one or more pixel groups.

24. One or more computer readable media having stored thereon instructions that, when executed by one or more processors, causes the one or more processors to perform acts comprising:

receiving an image;

searching a set of areas of the image for candidate pixels of one or more colors, wherein the one or more colors comprise colors corresponding to red-eye;

combining the candidate pixels into a set of one or more pixel groups, wherein the combining comprises combining candidate pixels into the same group if the candidate pixels are adjacent one another,

for each pixel group in the set of one or more pixel groups, classifying the pixel group as being part of an eye or not part of an eye;

checking whether flash was used in capturing the image, wherein checking whether flash was used in capturing the image comprises checking whether a flash used flag is set in a header corresponding to the image; and

performing the searching, combining, and classifying only if flash was used in capturing the image.

25. One or more computer readable media as recited in claim 24, wherein the receiving comprises receiving the image from a camera.

26. One or more computer readable media as recited in claim 24, the instructions further causing the one or more processors to perform acts comprising:

identifying areas within the image that are skin colored; and

using the identified areas as the set of areas.

27. One or more computer readable media as recited in claim 24, the instructions further causing the one or more processors to perform acts comprising:

for each pixel group in the set of one or more pixel groups, prior to classifying the pixel group;

identifying the geometric shape of the pixel group;

determining whether the geometric shape is similar to the shape of an eye; and

leaving the pixel group as part of the set of one or more pixel groups if the geometric shape is similar to the shape of an eye, and otherwise removing the pixel group from the set.

28. One or more computer readable media as recited in claim 27, wherein the shape is similar to the shape of an eye if the geometric shape is more circular than rectangular and if an aspect ratio of the geometric shape differs from the aspect ratio of a circle by not greater than a particular amount.

29. One or more computer readable media as recited in claim 24, wherein the classifying comprises:

applying a window to the image and comparing pixels within the window to an eye template;

altering the scale of the image; and

repeating the applying and comparing based on the scale-altered image.

30. One or more computer readable media as recited in claim 29, wherein the repeating comprises repeating the applying and comparing based on the scale-altered image without altering the size of the eye template.

31. One or more computer readable media as recited in claim 24, the instructions further causing the one or more processors to perform acts comprising removing, based on a set of rules, groups from the set of one or more pixel groups.