Modification of red-eye-effect in digital image

• US 20040228542A1
• Filed: 05/13/2003
• Published: 11/18/2004
• Est. Priority Date: 05/13/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Mingjing, Zhang, Lei, Zhang, Hong-Jiang, Sun, Yanfeng
• Granted Patent: US 7,224,850 B2
• Time in Patent Office: Days
• US Class Current: 382/275
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

In one aspect, the present disclosure describes a process for automatic artifact compensation in a digital representation of an image. The process includes detecting, by a processor, regions corresponding to facial images within the digital representation; locating, by the processor, red-eye regions within the detected regions; and automatically modifying, by the processor, the located red-eye regions to provide a modified image.

Claims

1. A process for automatic red-eye artifact modification in a digital representation of an image comprising:

detecting, by a processor, regions corresponding to facial images within the digital representation;

locating, by the processor, red-eye regions within the detected regions; and

automatically modifying, by the processor, the located red-eye regions to provide a modified image.

2. The process of claim 1, further comprising, prior to detecting, removing from a pool of candidate pixels comprising the digital representation first pixels having colors indicative that the first regions do not represent facial images.

3. The process of claim 1, wherein locating includes:

analyzing colors of pixels to denote red pixels and to denote non-skin-color pixels;

when a red pixel is surrounded by pixels having non-skin-colors, removing that pixel from further analysis; and

when the red pixel is not surround by non-skin-color pixels, subjecting data corresponding to that pixel to further analysis.

4. The process of claim 1, wherein locating includes:

denoting red pixels as satisfying R>

50;

R/(R+G+B)>

0.40;

G/(R+G+B)<

0.31 and B/(R+G+B)<

0.36;

determining when RGB values for a region about a denoted red pixel satisfy conditions;

G/(R+G+B)>

0.40 or B/(R+G+B)>

0.45;

when the RGB values for the region satisfy the conditions, removing the region from a pool of red-eye candidate regions; and

when the RGB values for the region do not satisfy the conditions, maintaining the regions in the pool.

5. The process of claim 1, wherein locating red-eye regions comprises:

filtering pixels comprising the digital representation for RGB content to denote red pixels and to denote non-skin-tone pixels;

finding highlight pixels within the denoted red pixels;

grouping highlight pixels to form a pool of highlight areas;

eliminating groups not characteristic of red-eye regions from the pool;

winnowing groups by size to remove some groups from the pool;

winnowing groups based on comparison of colors of nearby pixels to remove additional groups from the pool;

merging groups to form red-eye candidate regions; and

second winnowing groups based on comparison of colors of nearby pixels to remove further groups from the pool.

6. The process of claim 1, further comprising, after automatically modifying:

displaying the modified image;

receiving an indication from a user of a region within the modified image for further analysis;

identifying a candidate red-eye region within the modified image; and

correcting the identified red-eye region in response to user input.

7. An article of manufacture comprising a storage medium having computer-readable code embodied thereon that is configured to cause one or more processors to:

identify a group of candidate red-eye regions within a digital representation of an image;

winnow the group to eliminate candidate red-eye regions not associated with an image resembling an eye;

modify a first set of red-eye regions and blend locations corresponding to the first set of red-eye regions with the image to provide a modified image;

display the modified image; and

modify and blend one or more additional red-eye regions in response to user input to provide another modified image.

8. The article of manufacture of claim 7, wherein the computer-readable code is further configured to cause the one or more processors to replace the first set of red-eye regions and the one or more additional red-eye regions with regions that blend with the another modified image.

9. The article of manufacture of claim 7, wherein the computer-readable code configured to identify comprises computer readable code configured to cause the one or more processors to:

filter pixels to identify (i) first pixels that are considered to be red pixels and (ii) second pixels that are non-skin-color pixels;

sort the red pixels to locate highlight pixels and to form groups of highlight pixels; and

winnow the groups based on size, neighboring pixel color and red-eye characteristics to form the group of candidate red-eye regions.

10. The article of manufacture of claim 7, wherein the computer-readable code configured to modify one or more additional red-eye regions comprises computer readable code configured to cause the one or more processors to process a designated portion of the modified image using one or more thresholds that are less constrained than the thresholds employed to identify and winnow.

11. The article of manufacture of claim 7, wherein the computer-readable code configured to identify comprises computer readable code configured to cause the one or more processors to:

determine portions of the image corresponding to a facial image and having coloration consistent with skin; and

winnow and modify only within the determined portions.

12. A process for causing one or more processors to automatically modify a digital image comprising:

determining first areas within the image having correspondence with facial images;

eliminating portions within the first areas that are not consistent with a skin-tone-model to provide second areas;

identifying red-eye regions within the second areas;

modifying the red-eye regions from the digital image; and

blending the modified red-eye regions with the digital image to provide a modified image.

13. The process of claim 12, wherein identifying comprises identifying regions having coloration and size characteristics consistent with red-eye regions.

14. The process of claim 12, wherein identifying comprises:

locating first pixels comprising red pixels in association with skin-colored areas;

grouping first pixels by identifying and grouping highlight pixels;

winnowing first pixel groups to provide second pixel groups by eliminating first pixel groups based on coloration- of nearby pixels, characteristics of pixel groups and size of pixel groups;

forming red-eye candidate regions from the second pixel groups; and

eliminating red-eye candidate regions not associated with an image of an eye.

15. The process of claim 12, further comprising:

displaying the modified image;

receiving an indication from a user of a region within the modified image for further analysis;

identifying a candidate red-eye region within the modified image in response to user input; and

modifying the identified red-eye region to alter a red-eye image therein and blending the identified red-eye region with the modified image to provide another modified image.

16. A system for image manipulation comprising:

a processor;

a memory coupled to the processor and configured to store data representing an image; and

an artifact modification module configured to cause the processor to access the data representing the image and to;

identify first pixels having a red content above a first threshold;

identify second pixels having coloration inconsistent with a skin color model by comparing pixels to a set of threshold values;

sort the first and second pixels to form a pool of red-eye candidate regions;

winnow the pool to eliminate red-eye candidate regions that (i) are not associated with nearby second pixels and (ii) are not associated with images corresponding to an image of an eye to provide red-eye regions; and

modify and blend the red-eye regions to provide a modified image.

17. The system of claim 16, wherein the artifact modification module is further configured to:

cause the modified image to be displayed;

accept user input designating at least a portion of the modified image;

identify first and second pixels using a modified first threshold and a modified set of threshold values to identify one or more additional red-eye candidate regions; and

remove and blend the one or more additional red-eye regions to provide an altered image.

18. The system of claim 16, wherein the processor is contained in an apparatus chosen from a group consisting of a digital camera, a digital image scanning device, a computer and a printer.

19. The system of claim 16, wherein the artifact modification module comprises computer code embodied in a storage medium.

20. The system of claim 16, wherein the artifact modification module comprises integrated circuitry.

21. The system of claim 16, wherein the artifact modification module is further configured to:

determine those regions within the data representing the image having characteristics associated with a face; and

identify first and second pixels only within those regions.

22. The system of claim 16, wherein the artifact modification module is further configured to:

filter pixels according to a skin tone color model to identify candidate pixels;

determine, among the candidate pixels, those regions within the data representing the image having characteristics associated with a face; and

identify first and second pixels only within those regions.

23. The system of claim 16, wherein the artifact modification module configured to modify and blend the red-eye regions is configured to:

decrease the red value within the red-eye region;

increase the values of green and blue within the red-eye region;

decrease contrast within the red-eye region to provide a modified red-eye region; and

blend the modified red-eye region with the original image linearly based on the distance from each pixel in the red-eye region to a nearest edge of the red-eye region.

24. A process for detection and modification of specific types of regions in a digital representation of an image comprising:

identifying first pixels in a first set of regions including red values above a first threshold;

identifying second pixels adjacent the first pixels having coloration inconsistent with a color model for human skin;

identifying a third set of regions taken from the second set that are bordered by regions having coloration consistent with the color model;

locating highlight regions within the third set of regions;

grouping the highlight regions into a pool of red-eye candidate regions;

determining red-eye regions from the pool which satisfy predetermined criteria; and

modifying and blending the determined red-eye regions to provide a modified image.

25. The process of claim 24, wherein determining comprises winnowing the third set to only include those regions whose images are consistent with being associated with an image of an eye.

26. The process of claim 24, further comprising:

displaying the digital representation with the third set demarcated; and

accepting user input selecting a fourth set having overlap with the third set in response to displaying, wherein the fourth set includes fewer regions than the third set.

27. The process of claim 24, further comprising:

displaying the digital representation with the third set demarcated; and

accepting user input selecting a fourth set having overlap with the third set in response to displaying, wherein the fourth set includes more regions than the third set.

28. The process of claim 24, further comprising:

displaying the digital representation with the third set demarcated; and

accepting user input selecting a fourth set having overlap with the third set in response to displaying; and

correcting for red-eye distortion in the fourth set.

29. A camera comprising:

an image capture device; and

a red-eye filter module configured to automatically identify red-eye regions and to automatically treat identified red-eye regions.

30. The camera of claim 29, further comprising:

a first module configured to eliminate first regions from consideration as red-eye regions, the first regions having color characteristics inconsistent with a skin-tone color model;

a second module configured to identify regions including image portions having attributes associated with images of faces, wherein the red-eye filter module evaluates image portions having attributes consistent with an image of an eye.

31. A computer readable medium having a plurality of instructions thereon that, when executed by a processor, cause the processor to:

automatically process a digital image to identify red-eye regions within the digital image; and

modify the red-eye regions to provide a more natural appearance to the image.

32. The computer readable medium of claim 31, wherein the plurality of instructions to cause the processor to process the image further comprise instruction to cause the processor to:

identify first pixels in a first set of regions including red values above a first threshold;

identify second pixels adjacent the first pixels having coloration inconsistent with a color model for human skin; and

identify red-eye regions including at least some second pixels.

33. The computer readable medium of claim 31, wherein the plurality of instructions to cause the processor to process the image further comprise instruction to cause the processor to:

identify first pixels in a first set of regions including red values above a first threshold;

identify second pixels adjacent the first pixels having coloration inconsistent with a color model for human skin;

identify a third set of regions taken from the second set that are bordered by regions having coloration consistent with the color model;

locate highlight regions within the third set of regions;

group the highlight regions into a pool of red-eye candidate regions;

determine red-eye regions from the pool which satisfy predetermined criteria; and

modify the red-eye regions and blend the modified red-eye regions to provide a modified image.