Red-eye detection based on red region detection with eye confirmation

• US 7,636,470 B2
• Filed: 10/04/2004
• Issued: 12/22/2009
• Est. Priority Date: 02/13/2001
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Platt, John C., Yan, Jie, Zhang, Hong-Jiang, Chen, Xiangrong, Chen, Tong-Xian
• Primary Examiner(s): SHERALI, ISHRAT I
• Application Number:
  US10/958,136
• Publication Number:
  US 20050058340A1
• Time in Patent Office: 1,905 Days
• Field of Search: 382162-167, 382260-266, 382274-275, 382115-118, 382217-218, 358517-520, 358/527, 358/1.9, 348 77- 78, 348576-577, 348370-371, 348/560
• US Class Current: 382/165
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

Red-eye detection based on red region detection with eye confirmation initially identifies pixels that correspond to the color of red-eye within an image. A determination is then made as to whether these identified pixels and surrounding areas are part of an eye or not part of an eye. Those identified pixels that are determined to be part of an eye are the detected red-eye regions.

Claims

1. One or more computer-readable storage media encoded with instructions that, when executed, cause a processor to perform acts comprising:

receiving an image;

searching a set of areas of the image for candidate pixels of one or more colors, wherein the one or more colors comprise colors corresponding to red-eye;

combining the candidate pixels into a set of one or more pixel groups, wherein the combining comprises combining candidate pixels into the same group if the candidate pixels are adjacent one another;

filtering the pixel groups based on filling ratio, wherein the filling ratio comprises a number of red pixels in the pixel group divided by a product of horizontal and vertical aspects of the pixel group; and

for each pixel group in the set of one or more pixel groups, classifying the pixel group as being part of an eye or not part of an eye.

2. The one or more computer-readable storage media as recited in claim 1, wherein the receiving comprises receiving the image from a camera.

3. The one or more computer-readable storage media as recited in claim 1, wherein the one or more colors comprises one or more shades of red.

4. The one or more computer-readable storage media as recited in claim 1, further comprising:

identifying areas within the image that are skin colored; and

using the identified areas as the set of areas.

5. The one or more computer-readable storage media as recited in claim 1, wherein the combining comprises combining two candidate pixels into the same pixel group if the two candidate pixels are within a threshold distance of each other.

6. The one or more computer-readable storage media as recited in claim 1, further comprising for each pixel group in the set of one or more pixel groups, prior to classifying the pixel group:

identifying the geometric shape of the pixel group;

determining whether the geometric shape is similar to the shape of an eye; and

leaving the pixel group as part of the set of one or more pixel groups if the geometric shape is similar to the shape of an eye, and otherwise removing the pixel group from the set.

7. The one or more computer-readable storage media as recited in claim 6, wherein the shape is similar to the shape of an eye of the geometric shape is more circular than rectangular and if an aspect ratio of the geometric shape differs from the aspect ratio of a circle by not greater than a particular amount.

8. The one or more computer-readable storage media as recited in claim 1, wherein the classifying comprises:

applying a window to the image and comparing pixels within the window to an eye template;

altering the scale of the image; and

repeating the applying and comparing based on the scale-altered image.

9. The one or more computer-readable storage media as recited in claim 8, wherein the repeating comprises repeating the applying and comparing based on the scale-altered image without altering the size of the eye template.

10. The one or more computer-readable storage media as recited in claim 1, further encoded with instructions to perform an acts comprising removing, based on a set of rules, groups from the set of one or more pixel groups.

11. The one or more computer-readable storage media as recited in claim 1, further encoded with instructions to perform an acts comprising:

checking whether flash was used in capturing the image; and

performing the searching, combining, and classifying only if flash was used in capturing the image.

18. The one or more computer-readable storage media as recited in claim 1, wherein filtering the pixel groups based on filling ratio comprises determining that a pixel group having a filling ratio less than a threshold is not a red-eye region.

19. The one or more computer-readable storage media as recited in claim 1, further comprising filtering the pixel groups based on aspect ratio.

20. The one or more computer-readable storage media as recited in claim 19, wherein filtering the pixel groups based on aspect ratio comprises determining that a pixel group having an aspect ratio substantially different from a circle is not a red-eye region.

21. The one or more computer-readable storage media as recited in claim 19, wherein filtering the pixel groups based on aspect ratio comprises determining the aspect ratio of the pixel group by calculating a distance between rightmost and leftmost pixels in the pixel group defining a horizontal aspect, and a distance between uppermost and lowermost pixels in the group defining a vertical aspect, the aspect ratio being the horizontal aspect divided by the vertical aspect.

22. The one or more computer-readable storage media as recited in claim 19, wherein filtering the pixel groups based on aspect ratio comprises determining that an aspect ratio less than 0.5 or greater than 2.0 is not a red-eye region.

12. A system comprising:

means for searching a set of areas of an image for candidate pixels of one or more colors, the one or more colors comprising colors corresponding to red-eye;

means for combining the candidate pixels into a set of one or more pixel groups, the means for combining two candidate pixels into the same group if the two candidate pixels are adjacent one another;

means for filtering the pixel groups based on filling ratio, wherein the filling ratio comprises a number of red pixels in the pixel group divided by a product of horizontal and vertical aspects of the pixel group; and

means for classifying, for each pixel group in the set of one or more pixel groups, the pixel group as being part of an eye or not part of an eye.

13. A system as recited in claim 12, further comprising means for removing a group from the one or more pixel groups based on a set of rules.

14. A system as recited in claim 12, further comprising means for filtering the pixel groups based on aspect ratio.

15. A system as recited in claim 14, wherein the means for filtering the pixel groups based on aspect ratio is configured to determine that a pixel group having an aspect ratio substantially different from a circle is not a red-eye region.

16. A system as recited in claim 14, wherein the means for filtering the pixel groups based on aspect ratio is configured to determine the aspect ratio of the pixel group by calculating a distance between rightmost and leftmost pixels in the pixel group defining a horizontal aspect, and a distance between uppermost and lowermost pixels in the group defining a vertical aspect, the aspect ratio being the horizontal aspect divided by the vertical aspect.

17. A system as recited in claim 14, wherein the means for filtering the pixel groups based on aspect ratio is configured to determine that an aspect ratio less than 0.5 or greater than 2.0 is not a red-eye region.