Partial face detector red-eye filter method and apparatus

• US 9,412,007 B2
• Filed: 08/31/2009
• Issued: 08/09/2016
• Est. Priority Date: 08/05/2003
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: Nanu, Florin, Steinberg, Eran, Corcoran, Peter, Pososin, Alexei, Bigioi, Petronel, Drimbarean, Alexandru, Petrescu, Stefan, Gangea, Mihnea, Capata, Adrian, Ciuc, Mihai, Zamfir, Adrian
• Primary Examiner(s): Haskins, Twyler
• Assistant Examiner(s): BHUIYAN, FAYEZ A
• Application Number:
  US12/551,282
• Publication Number:
  US 20100053362A1
• Time in Patent Office: 2,535 Days
• Field of Search: 348/224.1, 382/103, 382/167
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

A digital camera has an integral flash and stores and displays a digital image. Under certain conditions, a flash photograph taken with the camera may result in a red-eye phenomenon due to a reflection within an eye of a subject of the photograph. A digital apparatus has a red-eye filter which analyzes at least one partial face region identified within the digital image for the red-eye phenomenon and modifies the image to eliminate the red-eye phenomenon by changing the red area to black. The modification of the image is enabled when a photograph is taken under conditions indicative of the red-eye phenomenon. The modification is subject to anti-falsing analysis which further examines the area around the red-eye area for indicia of the eye of the subject. The detection and correction can be optimized for performance and quality by operating on subsample versions of the image when appropriate.

Claims

1. A device, comprising:

an integral flash for providing illumination during image acquisition;

a digital image capturing apparatus for capturing a digital image, wherein

the device is configured to;

use a red-eye filter configured to determine a grouping of image pixels, in the digital image, containing image pixels having a color indicative of a red-eye phenomenon and a shape indicative of the red-eye phenomenon;

determine whether other image pixels, of the digital image, that are located around the grouping of image pixels have the color indicative of the red-eye phenomenon;

determine whether a substantially white area of pixels is found in vicinity of the grouping of image pixels; and

modify the color of the image pixels in the grouping in response to a set of conditions being satisfied, wherein the set of conditions includes both;

that the other image pixels do not have the color indicative of the red-eye phenomenon; and

that a substantially white area of pixels is found in the vicinity of the grouping of image pixels.

2. The device of claim 1, wherein the color indicative of the red-eye phenomenon is a red color.

3. The device of claim 2, wherein the modifying the color of the image pixels in the grouping comprises replacing the red color of the image pixels with a black color.

4. The device of claim 1, wherein the red-eye filter is configured to determine the grouping based in part on a full resolution data of the digital image and in part on a subsample resolution data of the digital image.

5. The device of claim 4, further comprising a module for generating the subsample resolution data of the digital image.

6. The device of claim 5, wherein generating the subsample resolution data of the digital image is determined using a spline interpolation.

7. The device of claim 5, wherein generating the subsample resolution data of the digital image is determined using a bi-cubic interpolation.

8. The device of claim 1, wherein said modifying the color of the image pixels in the grouping is performed on a full resolution data of the image pixels in the grouping.

9. The device of claim 4, wherein said red-eye filter comprises a plurality of sub-filters.

10. The device according to claim 9, wherein each of said plurality of sub-filters is configured to generate the subsample resolution data of the digital image based on one or more of:

an image size, a suspected red eye region size, a filter computation complexity, an empirical success rate of said sub-filter, an empirical false detection rate of said sub-filter, a falsing probability of said sub-filter, relations between suspected red eye regions, or results of previous analysis of one or more other sub-filters.

11. The device of claim 1, further comprising:

a pixel locator for locating pixels having the color indicative of the red-eye phenomenon;

a shape analyzer for determining if the grouping of at least a portion of the pixels located by the pixel locator comprises the shape indicative of the red-eye phenomenon; and

a pixel modifier for modifying the color of the pixels within the grouping.

12. The device of claim 11, further comprising a falsing analyzer for further processing the digital image in a vicinity of the grouping for details indicative of the eye, and for enabling the pixel modifier in response thereto.

13. The device of claim 11, further comprising an exposure analyzer for determining if the digital image was captured in a condition indicative of the red-eye phenomenon.

14. A method, comprising:

using a red-eye filter to determine a grouping of image pixels, in a digital image, containing image pixels having a color indicative of a red-eye phenomenon and a shape indicative of the red-eye phenomenon;

determining whether other image pixels, of the digital image, that are located around the grouping of image pixels have the color indicative of the red-eye phenomenon;

determining whether a substantially white area of pixels is found in vicinity of the grouping of image pixels; and

modifying the color of the image pixels in the grouping in response to a set of conditions being satisfied,

wherein the set of conditions includes both;

that the other image pixels do not have the color indicative of the red-eye phenomenon; and

that a substantially white area of pixels is found in the vicinity of the grouping of image pixels; and

wherein the method is performed by one or more computing devices.

15. The method of claim 14, wherein the color indicative of the red-eye phenomenon is a red color.

16. The method of claim 14, wherein the grouping is performed based in part on a full resolution data of the digital image and in part on a subsample resolution data of the digital image.

17. The method of claim 16, further comprising generating the subsample resolution data of the digital image.

18. The method of claim 16, further comprising generating the subsample resolution data of the digital image using a spline interpolation.

19. The method of claim 16, further comprising generating the subsample resolution data of the digital image using a bi-cubic interpolation.

20. The method of claim 14, wherein said modifying the color of the image pixels in the grouping is performed on a full resolution data of the image pixels in the grouping.

21. A non-transitory computer-readable storage medium, storing one or more instructions which, when executed by one or more processors, cause the one or more processors to perform a sequence of operations, comprising:

using a red-eye filter to determine a grouping of image pixels, in a digital image, containing image pixels having a color indicative of a red-eye phenomenon and a shape indicative of the red-eye phenomenon;

determining whether other image pixels, of the digital image, that are located around the grouping of image pixels have the color indicative of the red-eye phenomenon;

determining whether a substantially white area of pixels is found in the vicinity of the grouping of image pixels; and

modifying the color of the image pixels in the grouping in response to a set of conditions being satisfied;

wherein the set of conditions includes both;

that the other image pixels do not have the color indicative of the red-eye phenomenon; and

that a substantially white area of pixels is found in the vicinity of the grouping of image pixels.

22. The non-transitory computer-readable storage medium of claim 21, wherein the color indicative of the red-eye phenomenon is a red color.

23. The non-transitory computer-readable storage medium of claim 21, wherein the grouping is performed in part on full resolution data of the digital image and in part on subsample resolution data of the digital image.

24. The non-transitory computer-readable storage medium of claim 23, comprising additional instructions which, when executed by the one or more processors, cause the one or more processors to perform:

generating the subsample resolution data of the digital image.

25. The non-transitory computer-readable storage medium of claim 23, comprising additional instructions which, when executed by the one or more processors, cause the one or more processors to perform:

generating the subsample resolution data of the digital image using a spline interpolation.

26. The non-transitory computer-readable storage medium of claim 23, comprising additional instructions which, when executed by the one or more processors, cause the one or more processors to perform an additional operation comprising:

generating the subsample resolution data of the digital image using a bi-cubic interpolation.

27. The non-transitory computer-readable storage medium of claim 21, wherein the color of the image pixels in the grouping is modified on a full resolution of the image pixels in the grouping.