Red-eye filter method and apparatus

• US 8,493,466 B2
• Filed: 08/07/2012
• Issued: 07/23/2013
• Est. Priority Date: 08/05/2003
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Corcoran, Peter, Bigioi, Petronel, DeLuca, Michael J.
• Primary Examiner(s): LAM, HUNG H
• Application Number:
  US13/569,171
• Publication Number:
  US 20130033624A1
• Time in Patent Office: 350 Days
• Field of Search: 348/222.1, 348/239
• US Class Current: 348/222.1
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 9/70 for colour killing

Abstract

A digital image acquisition system having no photographic film, such as a digital camera, has a flash unit for providing illumination during image capture and a red-eye filter for detecting a region within a captured image indicative of a red-eye phenomenon, the detection being based upon a comparison of the captured image and a reference image of nominally the same scene taken without flash. In the embodiment the reference image is a preview image of lower pixel resolution than the captured image, the filter matching the pixel resolutions of the captured and reference images by up-sampling the preview image and/or sub-sampling the captured image. The filter also aligns at least portions of the captured image and reference image prior to comparison to allow for, e.g. movement in the subject.

Claims

1. A portable digital camera device having no photographic film, comprising:

a digital image capturing apparatus including an image sensor for capturing a main image including a face region,

focusing means configured to acquire multiple images at least approximately of said same scene as the main image including said face region and at different focus distances and at different degrees of focus based on the different focus distances;

processing means configured to analyze the different degrees of focus based on the different focus distances and to determine an approximate distance to the face and to focus the face on the one or more image sensors;

a dedicated hardware subsystem which is provided image data directly from the image sensor;

a face filter for determining a group of pixels that include the face region within the image data;

a subsampling module for changing a degree of subsampling based on a complexity of calculation for said face filter; and

a main system memory which receives raw image data and metadata as implemented by the dedicated hardware subsystem; and

wherein the main system memory is configured to receive metadata that comprises one or more identified face regions within the main image.

2. The portable digital camera device of claim 1, wherein the multiple images comprise a subsampled version of approximately the main image.

3. The portable digital camera device of claim 2, wherein the multiple images comprise a subsample resolution representation of approximately the main image.

4. The portable digital camera device of claim 1, wherein said changing the degree of said subsampling is determined empirically.

5. The portable digital camera device of claim 1, wherein said changing the degree of said subsampling is determined based on a size of said face region.

6. The portable digital camera device of claim 1, wherein said changing the degree of said subsampling is determined based on a relative size of said face region.

7. The portable digital camera device of claim 1, wherein said changing the degree of said subsampling is determined based on a region size.

8. The portable digital camera device of claim 3, wherein said subsample resolution representation is determined using spline interpolation.

9. The portable digital camera device of claim 3, wherein said subsample resolution representation is determined using bi-cubic interpolation.

10. The portable digital camera of claim 1, wherein one or more of the multiple images comprise at least one precapture image.

11. The portable digital camera of claim 10, wherein the precapture image is captured just before the main image is captured.

12. The portable digital camera of claim 1, further comprising one or more further filters for correcting exposure, focus or composition of the main image, or combinations thereof.

13. The portable digital camera of claim 1, further comprising a filter for aligning two or more of the multiple images.

14. The portable digital camera of claim 13, wherein the aligning comprises estimating camera or object movement, or both, in one or more intervals between capture of the two or more of the multiple images.

15. A method of focusing a face region onto an image sensor of a digital image capturing apparatus, comprising:

focusing the face region onto the image sensor including acquiring multiple images at least approximately of said same scene as the main image including said face region and at different focus distances and at different degrees of focus based on the different focus distances;

analyzing the different degrees of focus based on the different focus distances and determining an approximate distance to the face for focusing the face region onto the image sensor;

providing image data to a dedicated hardware subsystem directly from the image sensor;

determining a group of pixels that include the face region within the image data;

changing a degree of subsampling based on a complexity of calculation for said face filter; and

providing raw image data and metadata to a main memory as implemented by the dedicated hardware subsystem; and

wherein the metadata comprises one or more identified face regions within the main image.

16. The method of claim 15, further comprising determining empirically said changing the degree of said subsampling.

17. The method of claim 15, further comprising determining said changing the degree of said subsampling based on a size of said face region.

18. The method of claim 15, further comprising determining said changing the degree of said subsampling based on a relative size of said face region.

19. The method of claim 15, further comprising determining said changing the degree of said subsampling based on a region size.

20. The method of claim 15, further comprising capturing one or more precapture images just before capturing the main image.

21. The method of claim 15, further comprising aligning two or more of the multiple images.

22. The method of claim 21, wherein the aligning comprises estimating camera or object movement, or both, in one or more intervals between capture of the two or more of the multiple images.

23. One or more non-transitory processor-readable media having code embedded therein for programming one or more processors to perform a method of focusing a face region onto an image sensor of a digital image capturing apparatus, wherein the method comprises:

focusing the face region onto the image sensor including acquiring multiple images at least approximately of said same scene as the main image including said face region and at different focus distances and at different degrees of focus based on the different focus distances;

analyzing the different degrees of focus based on the different focus distances and determining an approximate distance to the face for focusing the face region onto the image sensor;

providing image data to a dedicated hardware subsystem directly from the image sensor;

determining a group of pixels that include the face region within the image data;

changing a degree of subsampling based on a complexity of calculation for said face filter; and

providing raw image data and metadata to a main memory as implemented by the dedicated hardware subsystem; and

wherein the metadata comprises one or more identified face regions within the main image.

24. The one or more non-transitory processor-readable media of claim 23, further comprising determining empirically said changing the degree of said subsampling.

25. The one or more non-transitory processor-readable media of claim 23, further comprising determining said changing the degree of said subsampling based on a size of said face region.

26. The one or more non-transitory processor-readable media of claim 23, further comprising determining said changing the degree of said subsampling based on a relative size of said face region.

27. The one or more non-transitory processor-readable media of claim 23, further comprising determining said changing the degree of said subsampling based on a region size.

28. The one or more non-transitory processor-readable media of claim 23, further comprising capturing one or more precapture images just before capturing the main image.

29. The one or more non-transitory processor-readable media of claim 23, further comprising aligning two or more of the multiple images.

30. The one or more non-transitory processor-readable media of claim 29, wherein the aligning comprises estimating camera or object movement, or both, in one or more intervals between capture of the two or more of the multiple images.