Red-eye filter method and apparatus

• US 8,698,916 B2
• Filed: 10/22/2012
• Issued: 04/15/2014
• Est. Priority Date: 08/05/2003
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Corcoran, Peter, Bigioi, Petronel, DeLuca, Michael J
• Primary Examiner(s): LAM, HUNG H
• Application Number:
  US13/656,738
• Publication Number:
  US 20130044243A1
• Time in Patent Office: 540 Days
• Field of Search: 348/222.1, 348/239
• US Class Current: 348/222.1
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 9/70 for colour killing

Abstract

A digital image acquisition system having no photographic film, such as a digital camera, has a flash unit for providing illumination during image capture and a red-eye filter for detecting a region within a captured image indicative of a red-eye phenomenon, the detection being based upon a comparison of the captured image and a reference image of nominally the same scene taken without flash. In the embodiment the reference image is a preview image of lower pixel resolution than the captured image, the filter matching the pixel resolutions of the captured and reference images by up-sampling the preview image and/or sub-sampling the captured image. The filter also aligns at least portions of the captured image and reference image prior to comparison to allow for, e.g. movement in the subject.

Claims

1. A portable digital camera device having no photographic film, comprising:

a digital image capturing apparatus including an image sensor for capturing a main image including a face, and for acquiring reference image data at least approximately of said same scene as the main image including said face;

a dedicated hardware subsystem which is provided reference image data directly from the image sensor;

a face filter for identifying a face region within the main image based on analysis of the reference image data;

a main system memory which receives raw image data and metadata as implemented by the dedicated hardware subsystem, the main system memory being configured to receive metadata that comprises one or more identified face regions within the main image; and

a subsampling module for changing a degree of subsampling that is determined based on a complexity of calculation for said face filter.

2. The portable digital camera device of claim 1, wherein the reference image data comprises a relatively low resolution compared with the main image.

3. The portable digital camera device of claim 2, wherein the reference image data comprises a subsample representation of the main image.

4. The portable digital camera device of claim 3, wherein the subsample representation comprises a subsample resolution representation.

5. The portable digital camera device of claim 3, wherein said changing the degree of said subsampling is determined empirically.

6. The portable digital camera device of claim 3, wherein said changing the degree of said subsampling is determined based on a size of said image.

7. The portable digital camera device of claim 3, wherein said changing the degree of said subsampling is determined based on a region size.

8. The portable digital camera device of claim 3, wherein said subsample resolution representation is determined using spline interpolation.

9. The portable digital camera device of claim 3, wherein said subsample resolution representation is determined using bi-cubic interpolation.

10. The portable digital camera of claim 1, wherein the reference-image data comprises precapture image data.

11. The portable digital camera of claim 10, wherein the precapture image data is acquired just before the main image is captured.

12. The portable digital camera of claim 1, further comprising one or more further filters for determining exposure, focus or composition of the main image, or combinations thereof, based on analysis of reference image data.

13. The portable digital camera of claim 1, further comprising an alignment filter for aligning one or more reference images with the main image.

14. The portable digital camera of claim 13, wherein the aligning comprises estimating camera or object movement, or both, in one or more intervals between capture of the main and reference images.

15. The portable digital camera of claim 13, further comprising a resolution matching filter for upsampling the one or more reference images or downsampling the main image or both.

16. A method of correcting an image defect within a digital image, comprising:

acquiring, with optics and an image sensor, a main image and reference image data of at least approximately the same scene as the main image;

providing the reference image data directly from the image sensor to a dedicated hardware subsystem;

identifying a face within the reference image data;

identifying the same face in the main image based on metadata associated with the face identified in the reference image data;

receiving raw image data at a main image memory, along with metadata as implemented by the dedicated hardware subsystem, the metadata comprising one or more candidate face regions identified in the reference image data each associated with having a certain enhanced probability of appearing in the main image;

subsampling certain image data to generate certain reference image data; and

determining a degree of said subsampling based on a complexity of calculation for said identifying said same face.

17. The method of claim 16, wherein the reference image data comprises a relatively low resolution compared with the main image.

18. The method of claim 17, wherein the reference image data comprises a subsample representation of the main image.

19. The method of claim 18, wherein the subsample representation comprises a subsample resolution representation.

20. The method of claim 18, wherein said changing the degree of said subsampling comprises determining said degree of subsampling empirically.

21. The method of claim 18, wherein said changing the degree of said subsampling comprising determining said degree of subsampling based on a size of said image.

22. The method of claim 18, wherein said changing the degree of said subsampling comprises determining said degree of subsampling based on a region size.

23. The method of claim 18, further comprising determining said subsample resolution representation using spline interpolation.

24. The method of claim 18, further comprising determining said subsample resolution representation using bi-cubic interpolation.

25. The method of claim 16, wherein the reference image data comprises precapture image data.

26. The method of claim 25, further comprising acquired the precapture image data just before capturing the main image.

27. The method of claim 16, further comprising applying one or more further filters for determining exposure, focus or composition of the main image, or combinations thereof, based on analysis of reference image data.

28. The method of claim 16, further comprising aligning one or more reference images with the main image.

29. The method of claim 28, wherein the aligning comprises estimating camera or object movement, or both, in one or more intervals between capture of the main and reference images.

30. The method of claim 28, further comprising applying a resolution matching filter for upsampling the one or more reference images or downsampling the main image or both.

31. One or more non-transitory digital storage media having code embodied therein for programming one or more processors of a portable digital image acquisition device that includes an image sensor for acquiring digital images and reference image data to perform a method that involves identifying faces within the image data, wherein the method comprises:

providing reference image data directly from the image sensor to a dedicated hardware subsystem;

identifying a face within the reference image data;

identifying the same face in the main image based on metadata associated with the face identified in the reference image data;

receiving raw image data at a main image memory, along with metadata as implemented by the dedicated hardware subsystem, the metadata comprising one or more candidate face regions identified in the reference image data each associated with having a certain enhanced probability of appearing in the main image;

subsampling main image data to generate reference image data; and

determining a degree of said subsampling based on a complexity of calculation for said identifying said same face.

32. The one or more non-transitory digital storage media of claim 31, wherein the reference image data comprises a relatively low resolution compared with the main image.

33. The one or more non-transitory digital storage media of claim 32, wherein the reference image data comprises a subsample representation of the main image.

34. The one or more non-transitory digital storage media of claim 33, wherein the subsample representation comprises a subsample resolution representation.

35. The one or more non-transitory digital storage media of claim 33, wherein said changing the degree of said subsampling comprises determining said degree of subsampling empirically.

36. The one or more non-transitory digital storage media of claim 33, wherein said changing the degree of said subsampling comprising determining said degree of subsampling based on a size of said image.

37. The one or more non-transitory digital storage media of claim 33, wherein said changing the degree of said subsampling comprises determining said degree of subsampling based on a region size.

38. The one or more non-transitory digital storage media of claim 33, further comprising determining said subsample resolution representation using spline interpolation.

39. The one or more non-transitory digital storage media of claim 33, further comprising determining said subsample resolution representation using bi-cubic interpolation.

40. The one or more non-transitory digital storage media of claim 31, wherein the reference image data comprises precapture image data.

41. The one or more non-transitory digital storage media of claim 40, further comprising acquired the precapture image data just before capturing the main image.

42. The one or more non-transitory digital storage media of claim 31, further comprising applying one or more further filters for determining exposure, focus or composition of the main image, or combinations thereof, based on analysis of reference image data.

43. The one or more non-transitory digital storage media of claim 31, further comprising aligning one or more reference images with the main image.

44. The one or more non-transitory digital storage media of claim 43, wherein the aligning comprises estimating camera or object movement, or both, in one or more intervals between capture of the main and reference images.

45. The one or more non-transitory digital storage media of claim 43, further comprising applying a resolution matching filter for upsampling the one or more reference images or downsampling the main image or both.