PARTIAL FACE DETECTOR RED-EYE FILTER METHOD AND APPARATUS

• US 20100053362A1
• Filed: 08/31/2009
• Published: 03/04/2010
• Est. Priority Date: 08/05/2003
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Ireland Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Petrescu, Stefan, Ciuc, Mihai, Zamfir, Adrian, Capata, Adrian, Nanu, Florin, Gangea, Mihnea, Drimbarean, Alexandru, Bigioi, Petronel, Corcoran, Peter, Pososin, Alexei
• Granted Patent: US 9,412,007 B2
• Time in Patent Office: Days
• US Class Current: 348/222.100
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

A digital camera has an integral flash and stores and displays a digital image. Under certain conditions, a flash photograph taken with the camera may result in a red-eye phenomenon due to a reflection within an eye of a subject of the photograph. A digital apparatus has a red-eye filter which analyzes at least one partial face region identified within the digital image for the red-eye phenomenon and modifies the image to eliminate the red-eye phenomenon by changing the red area to black. The modification of the image is enabled when a photograph is taken under conditions indicative of the red-eye phenomenon. The modification is subject to anti-falsing analysis which further examines the area around the red-eye area for indicia of the eye of the subject. The detection and correction can be optimized for performance and quality by operating on subsample versions of the image when appropriate.

Claims

1. A portable digital image capturing device having no photographic film, comprising:

a flash for providing illumination during image acquisition;

an optical system including a lens and an image sensor for capturing a digital image;

a partial face detector for identifying one or more partial face regions within the digital image; and

a red-eye filter for modifying an area within the image indicative of a red-eye phenomenon based on an analysis of a subsample representation comprising one or more partial face regions identified within the image.

2. The device of claim 1, wherein said red-eye filter is adapted based on a type of at least one of said one or more partial face regions identified within the digital image.

3. The device of claim 1, wherein the analysis is performed at least in part for determining said area.

4. The device of claim 1, wherein the analysis is performed at least in part for determining said modifying.

5. The device of claim 1, wherein at least one partial face region within the digital image is not among said one or more partial face regions identified within the digital image that are analyzed.

6. The device of claim 1, wherein said analysis is performed in part on a full resolution partial face region and in part on a subsample resolution of at least one different partial face region.

7. The device of claim 1, further comprising a module for changing the degree of said subsampling.

8. The device of claim 1, wherein said subsample representation is determined using spline interpolation.

9. The device of claim 1, wherein said subsample representation is determined using bi-cubic interpolation.

10. The device of claim 1, wherein said modifying the area is performed on a full resolution of at least one partial face region within the digital image.

11. The device of claim 1, wherein said red-eye filter comprises a plurality of sub-filters.

12. The device according to claim 11, wherein said subsampling for said sub-filters operating on selected regions of said image is determined by image size, a suspected red eye region size, filter computation complexity, empirical success rate of said sub-filter, empirical false detection rate of said sub-filter, falsing probability of said sub-filter, relations between suspected red eye regions, or results of previous analysis of one or more other sub-filters, or combinations thereof.

13. The device of claim 1, further comprising:

a pixel locator for locating pixels having a color indicative of the red-eye phenomenon;

a shape analyzer for determining if a grouping of at least a portion of the pixels located by the pixel locator comprise a shape indicative of the red-eye phenomenon; and

a pixel modifier for modifying the color of the pixels within the grouping.

14. The device of claim 13, further comprising a falsing analyzer for further processing the digital image in a vicinity of the grouping for details indicative of an eye, and for enabling the pixel modifier in response thereto.

15. The device of claim 13, further comprising an exposure analyzer for determining if the digital image was acquired in a condition indicative of the red-eye phenomenon.

16. A method of capturing and processing within a portable device a digital image including one or more partial face regions not exhibiting any red eye defect, comprising:

capturing a digital image with a portable device that includes a lens and an image sensor;

providing flash illumination during capturing of the digital image with the portable device;

identifying one or more partial face regions within the image each including at least one eye;

analyzing a subsample representation of at least one of said one or more partial face regions within the digital image, while foregoing within the digital image analysis of at least one other partial face region not including an eye; and

modifying an area within the at least one partial face region that is determined to be indicative of a red-eye phenomenon based on said analyzing.

17. The method of claim 16, further comprising adapting the modifying based on a type of said at least one partial face region.

18. The method of claim 16, further comprising determining said area based at least in part on the analyzing.

19. The method of claim 16, further comprising determining said modifying based at least in part on the analyzing.

20. The method of claim 16, wherein said analyzing is performed in part on a full resolution partial face region and in part on a subsample resolution of at least one different partial face region.

21. The method of claim 16, further comprising changing the degree of said subsampling.

22. The method of claim 16, further comprising determining said subsample representation using spline interpolation.

23. The method of claim 16, further comprising determining said subsample representation using bi-cubic interpolation.

24. The method of claim 16, wherein said modifying the area is performed on a full resolution of at least one partial face region within the image.

25. The method of claim 16, wherein the analyzing and modifying comprise:

locating pixels having a color indicative of the red-eye phenomenon;

determining if a grouping of at least a portion of the pixels located by the pixel locator comprise a shape indicative of the red-eye phenomenon; and

modifying the color of the pixels within the grouping.

26. One or more processor-readable storage devices having code embedded therein for programming a processor to perform a method of capturing and processing within a portable device a digital image including one or more partial face regions not exhibiting any red eye defect, the method comprising:

capturing a digital image with a portable device that includes a lens and an image sensor;

providing flash illumination during capturing of the digital image with the portable device;

identifying one or more partial face regions within the image each including at least one eye;

analyzing a subsample representation of at least one of said one or more partial face regions within the digital image, while foregoing within the digital image analysis of at least one other partial face region not including an eye; and

modifying an area within the at least one partial face region that is determined to be indicative of a red-eye phenomenon based on said analyzing.

27. The one or more processor-readable storage devices of claim 26, wherein the method further comprises adapting the modifying based on a type of said at least one partial face region.

28. The one or more processor-readable storage devices of claim 26, wherein the method further comprises determining said area based at least in part on the analyzing.

29. The one or more processor-readable storage devices of claim 26, wherein the method further comprises determining said modifying based at least in part on the analyzing.

30. The one or more processor-readable storage devices of claim 26, wherein said analyzing is performed in part on a full resolution partial face region and in part on a subsample resolution of at least one different partial face region.

31. The one or more processor-readable storage devices of claim 26, wherein the method further comprises changing the degree of said subsampling.

32. The one or more processor-readable storage devices of claim 26, wherein the method further comprises determining said subsample representation using spline interpolation.

33. The one or more processor-readable storage devices of claim 26, wherein the method further comprises determining said subsample representation using bi-cubic interpolation.

34. The one or more processor-readable storage devices of claim 26, wherein said modifying the area is performed on a full resolution of at least one partial face region within the image.

35. The one or more processor-readable storage devices of claim 26, wherein the analyzing and modifying comprise:

locating pixels having a color indicative of the red-eye phenomenon;

determining if a grouping of at least a portion of the pixels located by the pixel locator comprise a shape indicative of the red-eye phenomenon; and

modifying the color of the pixels within the grouping.