Red-eye detection based on red region detection with eye confirmation

• US 20050058340A1
• Filed: 10/04/2004
• Published: 03/17/2005
• Est. Priority Date: 02/13/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Platt, John C., Yan, Jie, Zhang, Hong-Jiang, Chen, Xiangrong, Chen, Tong-Xian
• Granted Patent: US 7,636,470 B2
• Time in Patent Office: Days
• US Class Current: 382/165
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

Red-eye detection based on red region detection with eye confirmation initially identifies pixels that correspond to the color of red-eye within an image. A determination is then made as to whether these identified pixels and surrounding areas are part of an eye or not part of an eye. Those identified pixels that are determined to be part of an eye are the detected red-eye regions.

Claims

1. A system comprising:

a region detection module to detect regions of an image that include pixels of a particular one or more colors without requiring faces within the image to be previously detected; and

an eye confirmation module to receive the detected regions from the region detection module and identify, for each of the detected regions, whether the detected region is part of an eye.

2. A system as recited in claim 1, wherein the particular one or more colors are different shades of the same color.

3. A system as recited in claim 1, wherein the particular one or more colors are one or more shades of red.

4. A system as recited in claim 1, wherein the image is a digital image.

5. A system as recited in claim 1, wherein the image is a digitized version of a film image.

6. A system as recited in claim 1, wherein the eye confirmation module is to determine, for each of the detected regions, whether the detected region is part of a human eye.

7. A system as recited in claim 1, wherein the system is implemented in a computer.

8. A system as recited in claim 1, wherein the system is implemented in a camera.

9. A system as recited in claim 1, wherein the system is implemented in an image printing device.

10. A system as recited in claim 1, further comprising:

a skin color module to detect areas of skin color-in the image and indicate the detected areas to the region detection module; and

wherein the region detection module is to search within the detected areas to detect regions that include pixels of the particular one or more colors.

11. A system as recited in claim 1, wherein the eye confirmation module comprises an SVM (Support Vector Machine) classifier to classify each of the detected regions as either part of an eye or not part of an eye.

12. A system as recited in claim 1, wherein the eye confirmation module comprises a multi-scale classifier to apply a window to the image and compare pixels within the window to an eye template, to alter the scale of the image, and then to repeat the application of the window to the scale-altered image and comparison to the eye template.

13. A system as recited in claim 1, wherein the region detection module comprises a pixel identifier that is trained to colors associated with red-eye, and wherein the pixel identifier is to identify pixels within the region having colors that are close to the colors associated with red-eye.

14. A system as recited in claim 13, wherein the region detection module further comprises a pixel grouper coupled to receive the identified pixels from the pixel identifier and group together adjacent pixels.

15. A system as recited in claim 14, wherein the pixel grouper is further to group together pixels within a threshold distance of one another.

16. A method comprising:

receiving an image;

searching a set of areas of the image for candidate pixels of one or more colors, wherein the one or more colors comprise colors corresponding to red-eye;

combining the candidate pixels into a set of one or more pixel groups, wherein the combining comprises combining candidate pixels into the same group if the candidate pixels are adjacent one another; and

for each pixel group in the set of one or more pixel groups, classifying the pixel group as being part of an eye or not part of an eye.

17. A method as recited in claim 16, wherein the receiving comprises receiving the image from a camera.

18. A method as recited in claim 16, wherein the one or more colors comprises one or more shades of red.

19. A method as recited in claim 16, further comprising:

identifying areas within the image that are skin colored; and

using the identified areas as the set of areas.

20. A method as recited in claim 16, wherein the combining comprises combining two candidate pixels into the same pixel group if the two candidate pixels are within a threshold distance of each other.

21. A method as recited in claim 16, further comprising for each pixel group in the set of one or more pixel groups, prior to classifying the pixel group:

identifying the geometric shape of the pixel group;

determining whether the geometric shape is similar to the shape of an eye; and

leaving the pixel group as part of the set of one or more pixel groups if the geometric shape is similar to the shape of an eye, and otherwise removing the pixel group from the set.

22. A method as recited in claim 21, wherein the shape is similar to the shape of an eye of the geometric shape is more circular than rectangular and if an aspect ratio of the geometric shape differs from the aspect ratio of a circle by not greater than a particular amount.

23. A method as recited in claim 16, wherein the classifying comprises:

applying a window to the image and comparing pixels within the window to an eye template;

altering the scale of the image; and

repeating the applying and comparing based on the scale-altered image.

24. A method as recited in claim 23, wherein the repeating comprises repeating the applying and comparing based on the scale-altered image without altering the size of the eye template.

25. A method as recited in claim 16, further comprising removing, based on a set of rules, groups from the set of one or more pixel groups.

26. A method as recited in claim 16, further comprising:

checking whether flash was used in capturing the image; and

performing the searching, combining, and classifying only if flash was used in capturing the image.

27. One or more computer-readable memories containing a computer program that is executable by a processor to perform the method recited in claim 16.

28. A camera comprising:

an image capturer to capture an image; and

a red-eye detector including,

a region detector to detect regions of the image that include red pixels without requiring faces within the image to be previously detected, and

an eye confirmer to receive the detected regions from the region detector and identify, for each of the detected regions, whether the detected region is an eye.

29. A camera as recited in claim 28, wherein the image capturer comprises film.

30. A camera as recited in claim 28, wherein the image capturer comprises a charge coupled device (CCD).

31. A system comprising:

means for searching a set of areas of an image for candidate pixels of one or more colors, the one or more colors comprising colors corresponding to red-eye;

means for combining the candidate pixels into a set of one or more pixel groups, the means for combining combining two candidate pixels into the same group if the two candidate pixels are adjacent one another; and

means for classifying, for each pixel group in the set of one or more pixel groups, the pixel group as being part of an eye or not part of an eye.

32. A system as recited in claim 31, further comprising means for removing a group from the one or more pixel groups based on a set of rules.