Modification of red-eye-effect in digital image

• US 7,224,850 B2
• Filed: 05/13/2003
• Issued: 05/29/2007
• Est. Priority Date: 05/13/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Mingjing, Zhang, Lei, Zhang, Hong-Jiang, Sun, Yanfeng
• Primary Examiner(s): Mancuso; Joseph
• Assistant Examiner(s): Allison; Andrae
• Application Number:
  US10/436,710
• Publication Number:
  US 20040228542A1
• Time in Patent Office: 1,477 Days
• Field of Search: 382/275, 382/167, 382/117, 348/207, 358/222
• US Class Current: 382/275
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

In one aspect, the present disclosure describes a process for automatic artifact compensation in a digital representation of an image. The process includes detecting, by a processor, regions corresponding to facial images within the digital representation; locating, by the processor, red-eye regions within the detected regions; and automatically modifying, by the processor, the located red-eye regions to provide a modified image.

Claims

1. A process at least partially implemented by a computer for automatic red-eye artifact modification in a digital representation of an image comprising:

detecting, by a processor, regions corresponding to facial images within the digital representation; and

locating, by a processor, red-eye regions within the detected regions, the locating comprising;

denoting red pixels as satisfying R>

50;

R/(R+G+B)>

0.40;

G/(R+G+B)<

0.31 and B/(R+G+B)>

0.36;

determining when RGB values for a region about a denoted red pixel satisfy conditions;

G/(R+G+B)>

0.4(Original) 0 or B/(R+G+B)>

0.45;

when the RGB values for the region satisfy the conditions, removing the region from a pool of red-eye candidate regions; and

when the RGB values for the region do not satisfy the conditions, maintaining the regions in the pool.

2. A process at least partially implemented by a computer for automatic red-eye artifact modification in a digital representation of an image comprising:

detecting, by a processor, regions corresponding to facial images within the digital representation;

locating, by the processor, red-eye regions within the detected regions, the locating comprising;

filtering pixels comprising the digital representation for RGB content to denote red pixels and to denote non-skin-tone pixels;

finding highlight pixels within the denoted red pixels;

grouping highlight pixels to form a pool of highlight areas;

eliminating groups not characteristic of red-eye regions from the pool;

winnowing groups by size to remove some groups from the pool;

winnowing groups based on comparison of colors of nearby pixels to remove additional groups from the pool;

merging groups to form red-eye candidate regions; and

second winnowing groups based on comparison of colors of nearby pixels to remove further groups from the pool; and

automatically modifying, by the processor the located red-eye regions to

provide a modified image.

3. The process of claim 2, further comprising, prior to detecting, removing from a pool of candidate pixels comprising the digital representation first pixels having colors indicative that the first regions do not represent facial images.

4. The process of claim 2, further comprising,

after automatically modifying:

displaying the modified image;

receiving an indication from a user of a region within the modified image for further analysis;

identifying a candidate red-eye region within the modified image; and

correcting the identified red-eye region in response to user input.

5. A system for image manipulation comprising:

a processor;

a memory coupled to the processor and configured to store data representing an image; and

an artifact modification module configured to cause the processor to access the data representing the image and to;

identify first pixels having a red content above a first threshold;

identify second pixels having coloration inconsistent with a skin color model by comparing pixel to a set of threshold values;

sort the first and second pixels to form a pool of red-eye candidate regions;

winnow the pool to eliminate red-eye candidate regions that (i) are not associated with nearby second pixels and (ii) are not associated with images corresponding to an image of an eye to provide red-eye regions; and

modify and blend the red-eye regions to provide a modified image; and

wherein the artifact modification module configured to modify and blend the red-eye regions is configured to;

decrease the red value within the red-eye region;

increase the values of green and blue within the red-eye region;

decrease contrast within the red-eye region to provide a modified red-eye region; and

blend the modified red-eye region with the original image linearly based on the distance from each pixel in the red-eye region to a nearest edge of the red-eye region.

6. The system of claim 5, wherein the artifact modification module is further configured to:

cause the modified image to be displayed;

accept user input designating at least a portion of the modified image;

identify first and second pixels using a modified first threshold and a modified set of threshold values to identify one or more additional red-eye candidate regions; and

remove and blend the one or more additional red-eye regions to provide an altered image.

7. The system of claim 5, wherein the processor is contained in an apparatus chosen from a group consisting of a digital camera, a digital image scanning device, a computer and a printer.

8. The system of claim 5, wherein the artifact modification module comprises computer code embodied in a storage medium.

9. The system of claim 5, wherein the artifact modification module comprises integrated circuitry.

10. The system of claim 5, wherein the artifact modification module is further configured to:

determine those regions within the data representing the image having characteristics associated with a face; and

identify first and second pixels only within those regions.

11. The system of claim 5, wherein the artifact modification module is further configured to:

filter pixels according to a skin tone color model to identify candidate pixels;

determine, among the candidate pixels, those regions within the data representing the image having characteristics associated with a face; and

identify first and second pixels only within those regions.

12. A process at least partially implemented by a computer for detection and modification of specific types of regions in a digital representation of an image comprising:

identifying first pixels in a first set of regions including red values above a first threshold;

identifying second pixels adjacent the first pixels having coloration inconsistent with a color model for human skin;

identifying a third set of regions taken from the second set that are bordered by regions having coloration consistent with the color model;

locating highlight regions within the third set of regions;

grouping the highlight regions into a pool of red-eye candidate regions;

determining red-eye regions from the pool which satisfy predetermined criteria; and

modifying and blending the determined red-eye regions to provide a modified image.

13. The process of claim 12, wherein determining comprises winnowing the third set to only include those regions whose images are consistent with being associated with an image of an eye.

14. The process of claim 12, further comprising:

displaying the digital representation with the third set demarcated; and

accepting user input selecting a fourth set having overlap with the third set in response to displaying, wherein the fourth set includes fewer regions than the third set.

15. The process of claim 12, further comprising:

displaying the digital representation with the third set demarcated; and

accepting user input selecting a fourth set having overlap with the third set in response to displaying, wherein the fourth set includes more regions than the third set.

16. The process of claim 12, further comprising:

displaying the digital representation with the third set demarcated; and

accepting user input selecting a fourth set having overlap with the third set in response to displaying; and

correcting for red-eye distortion in the fourth set.

17. A tangible computer readable medium having a plurality of instructions thereon that, when executed by a processor, cause the processor to:

automatically process a digital image to identify red-eye regions within the digital image, and wherein the plurality of instructions to cause the processor to process the image further comprise instruction to cause the processor to;

(a) identify first pixels in a first set of regions including red values above a first threshold;

(b) identify second pixels adjacent the first pixels having coloration inconsistent with a color model for human skin;

(c) identify a third set of regions taken from the second set that are bordered by regions having coloration consistent with the color model;

(d) locate highlight regions within the third set of regions;

(e) group the highlight regions into a pool of red-eye candidate regions;

(f) determine red-eye regions from the pool which satisfy predetermined criteria; and

(g) modify the red-eye regions and blend the modified red-eye regions to provide a modified image; and

modify the red-eye regions to provide a more natural appearance to the image.