DUAL-PHASE RED EYE CORRECTION

• US 20120301024A1
• Filed: 05/26/2011
• Published: 11/29/2012
• Est. Priority Date: 05/26/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wen, Fang, Sun, Jian, Yuan, Lu
• Granted Patent: US 8,571,271 B2
• Time in Patent Office: Days
• US Class Current: 382/167
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

A dual-phase approach to red eye correction may prevent overly aggressive or overly conservative red eye reduction. The dual-phase approach may include detecting an eye portion in a digital image. Once the eye portion is detected, the dual-phase approach may include the performance of a strong red eye correction for the eye portion when the eye portion includes a strong red eye. Otherwise, the dual-phase approach may include the performance of a weak red eye correction for the eye portion when the eye portion includes a weak red eye. The weak red eye may be distinguished from the strong red eye based a redness threshold that shows the weak red eye as having less redness hue than the strong red eye.

Claims

1. A computer-readable medium storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

detecting an eye portion in a digital image;

performing a strong red eye correction for the eye portion when the eye portion includes a strong red eye; and

performing a weak red eye correction for the eye portion when the eye portion includes a weak red eye that is at least distinguished from the strong red eye based a redness threshold as having less redness hue than the strong red eye.

2. The computer-readable medium of claim 1, wherein the detecting includes automatically separating the eye portion from a background of the digital image via object recognition or separating the eye portion from the background based at least on an estimation of interpupillary distance obtained from an manual input of eye locations in the digital image.

3. The computer-readable medium of claim 1, wherein the eye portion is determined to include a strong red eye when a redness hue of the eye portion is above the redness threshold.

4. The computer-readable medium of claim 1, wherein the eye portion is determined to include the strong red eye when a redness hue of the eye portion is above the redness threshold, each of one or more shape values of a region inside a red eye area of the eye portion is greater than a corresponding roundness threshold, and the region inside the red eye area has a centrality value that is greater than a corresponding centrality threshold.

5. The computer-readable medium of claim 3, wherein the eye portion is determined to includes the weak red eye when the redness hue of the eye portion is equal to or below the redness threshold, and a redness mean of a region inside the eye portion is closer in distance to a redness mean of a cluster of known weak red eye regions than a redness mean of a cluster of non-red eye regions.

6. The computer-readable medium of claim 5, further comprising performing no red eye correction for the eye portion when the redness hue of the eye portion is equal to or below the redness threshold and a redness mean of the region inside the eye portion is closer in distance to the redness mean of the cluster of non-red eye regions than the redness mean of the cluster of known weak red eye regions.

7. The computer-readable medium of claim 1, wherein the performing the strong red eye correction includes:

determining a strong red eye soft mask for a correction region that is larger than a strong red eye area in the eye portion;

obtaining a strong red eye confidence map for the correction region based at least on the strong red eye soft mask; and

correcting the strong red eye in the eye portion based at least on the strong red eye confidence map.

8. The computer-readable medium of claim 7, wherein the correcting the strong red eye includes correcting each pixel in the correction region using an algorithm that includes:

R′

=(1−

C_strong)*R+C_strong*min(G, B)=(1−

0.8*C_strong)*I+0.8*C_strong*min(G, B)

in which C_strong is the strong red eye confidence map, I={G, B} and R, G, and B are three color channels in a RGB color space.

9. The computer-readable medium of claim 7, wherein the correction region includes additional reddish pixels from outside the strong red eye area.

10. The computer-readable medium of claim 1, wherein performing the weak red eye correction includes:

determining a weak red eye soft mask for the eye portion;

obtaining a weak red eye confidence map for a weak red eye area based at least on the weak red eye soft mask; and

correcting the weak red eye in the eye portion based at least on the weak red eye confidence map and a target color.

11. The computer-readable medium of claim 10, wherein correcting the weak red eye includes correcting each pixel in the weak red eye area using an algorithm that includes:

I′

=(1−

C_weak)* I+C_weak*T

in which C_weak is the weak red eye confidence map, I={R, G, B}, T is the target color, and R, G, and B are three color channels in an RGB color space.

12. The computer-readable medium of claim 10, wherein the target color is obtained by converting a gray color that is derived from the RGB colors of the weak red eye area to a Lab color space and multiplying an L channel of the Lab color space by a predetermined factor.

13. A computer-implemented method, comprising:

detecting an eye portion in a digital image;

performing a strong red eye correction on a red eye in the eye portion at least when a redness hue of the eye portion is above a redness threshold; and

performing a weak red eye correction on the red eye when the redness hue of the eye portion is equal to or below the redness threshold, and a redness mean of a region inside the eye portion is closer in distance to a redness mean of a cluster of known weak red eye regions than a redness mean of a cluster of non-red eye regions.

14. The computer-implemented method of claim 13, further comprising performing no red eye correction on the red eye when the redness hue of the eye portion is equal to or below the redness threshold and a redness mean of the region inside the eye portion is closer in distance to the redness mean of the cluster of non-red eye regions than the redness mean of the cluster of known weak red eye regions.

15. The computer-implemented method of claim 13, further comprising:

determining one or more potential weak red eye areas in the eye portion;

selecting a potential weak red eye area that has a largest area and is closest to a center of the eye portion;

extracting the region from the selected potential weak red eye area; and

calculating the redness mean for the region.

16. The computer-implemented method of claim 13, wherein the performing the strong red eye correction includes performing the strong red eye correction on the red eye when the redness hue of the eye portion is above a redness threshold, each of one or more shape values of an additional region inside a strong red eye area in the eye portion is greater than a corresponding roundness threshold, and the additional region inside the red eye area has a centrality value that is greater than a corresponding centrality threshold.

17. The computer-implemented method of claim 16, furthering comprising:

generating a redness probability map for the eye portion;

generating a binary mask that locates one or more candidate red eye areas based at least one the redness probability map; and

selecting a candidate red eye area as the additional region.

18. A computing device, comprising:

one or more processors; and

a memory that includes a plurality of computer-executable components, the plurality of computer-executable components comprising;

an eye locator module that detects an eye portion in a digital image based at least on an estimation of interpupillary distance obtained from an manual input of eye locations in the digital image;

a strong red eye correction component that performs a strong red eye correction for the eye portion when the eye portion includes a strong red eye; and

a weak red eye correction component that performs a weak red eye correction for the eye portion when the eye portion includes a weak red eye that is at least distinguished from the strong red eye based a redness threshold as having less redness hue than the strong red eye.

19. The computing device of claim 18, wherein the strong red eye correction component includes:

a detection module that selects a candidate red eye area from the eye portion;

a validation module that determines that the strong red eye is present when an hue of the eye portion is above the redness threshold, each of one or more shape values of a region inside the candidate red eye area is greater than a corresponding roundness threshold, and the region inside the candidate red eye area has a centrality value that is greater than a corresponding centrality threshold; and

a correction module that corrects the strong red eye by determining a strong red eye soft mask for a correction region that is larger than the candidate red eye area, obtaining a strong red eye confidence map for the correction region based at least on the strong red eye soft mask, and correcting the strong red eye in the eye portion based at least on the strong red eye confidence map.

20. The computing device of claim 18, wherein the weak red eye correction component includes:

a detection module that selects a potential weak red eye area from one or more potential weak red eye areas in the eye portion;

a validation module that determines that the weak red eye is present when the redness hue of the eye portion is equal to or below the redness threshold, and a redness mean of a region inside the potential weak red eye area is closer in distance to a redness mean of a cluster of known weak red eye regions than a redness mean of a cluster of non-red eye region; and

a correction module that corrects the weak red eye by determining a weak red eye soft mask for the eye portion, obtaining a weak red eye confidence map for a weak red eye area based at least on the weak red eye soft mask, and correcting the weak red eye in the eye portion based at least on the weak red eye confidence map and a target color.