Red-eye filter method and apparatus

• US 7,973,828 B2
• Filed: 02/17/2010
• Issued: 07/05/2011
• Est. Priority Date: 10/09/1997
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: DeLuca, Michael J, Steinberg, Eran, Prilutsky, Yury, Bigioi, Petronel, Corcoran, Peter
• Primary Examiner(s): LAM, HUNG H
• Application Number:
  US12/707,531
• Publication Number:
  US 20100295959A1
• Time in Patent Office: 503 Days
• Field of Search: 348/222.1, 348/239
• US Class Current: 348/222.1
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 9/70 for colour killing

Abstract

A digital image acquisition system having no photographic film, such as a digital camera, has a flash unit for providing illumination during image capture and a red-eye filter for detecting a region within a captured image indicative of a red-eye phenomenon, the detection being based upon a comparison of the captured image and a reference image of nominally the same scene taken without flash. In the embodiment the reference image is a preview image of lower pixel resolution than the captured image, the filter matching the pixel resolutions of the captured and reference images by up-sampling the preview image and/or sub-sampling the captured image. The filter also aligns at least portions of the captured image and reference image prior to comparison to allow for, e.g. movement in the subject.

Claims

1. A portable digital camera-enabled device having no photographic film, comprising:

(a) an integral flash for providing illumination during image acquisition;

(b) a digital image capturing apparatus including

(i) at least one lens;

(ii) a main image sensor for acquiring a main image of a scene including at least one real face (contra an image of a face), and

(iii) a separate sensor for acquiring a reference image including a representation nominally of said same scene as the main image including said at least one real face;

(c) a face filter for correcting a face defect within the main image acquired by the main image sensor based on analysis of the reference image acquired by the separate sensor including said representation nominally of said same scene as the main image including said at least one real face;

(c) a dedicated hardware subsystem which is provided reference image data from the separate sensor; and

(d) a main system memory which receives raw image data and metadata as implemented by the dedicated hardware subsystem; and

(f) wherein the main system memory is configured to receive meta data that comprises identified defect areas determined within at least one image of said scene including at least one real face.

2. The portable digital camera-enabled device of claim 1, wherein the reference image comprises a low resolution image including a subsample representation.

3. The portable digital camera-enabled device of claim 2, wherein the reference image comprises a subsample resolution representation.

4. The portable digital camera-enabled device of claim 2, wherein the reference image comprises a segmented representation of the main image.

5. The portable digital camera-enabled device of claim 2, further comprising a module for changing a degree of subsampling.

6. The portable digital camera-enabled device of claim 1, wherein the reference image comprises a precapture image captured just before the main image is captured.

7. The portable digital camera-enabled device of claim 1, further comprising one or more further filters for correcting exposure, focus or composition of the main image, or combinations thereof, based on analysis of the reference image acquired by the separate sensor.

8. The portable digital camera-enabled device of claim 1, further comprising a filter for aligning the reference image with the main image.

9. The portable digital camera-enabled device of claim 8, wherein the aligning comprises estimating camera or object movement, or both, in one or more intervals between capture of the main and reference images.

10. The portable digital camera-enabled device of claim 1, wherein the reference is generated in parallel with the main image.

11. A method of correcting a face defect within a digital image, comprising:

providing a portable digital camera-enabled device including a main image sensor for capturing a main image and a separate sensor for acquiring a reference image temporally proximate to acquisition of the main image;

providing illumination during main image acquisition with a flash;

acquiring with the main image sensor a main image of a scene including at least one real face (contra an image of a face);

acquiring with the separate sensor a reference image temporally proximate to acquisition of the main image and including a representation nominally of said same scene as the main image including said at least one real face;

correcting a face defect within the main image acquired by the main image sensor based on an analysis of the reference image acquired by the separate sensor including the representation nominally of said same scene as the main image including said at least one real face;

providing reference image data to a dedicated hardware subsystem from the separate sensor; and

providing raw image data and meta data to a main system memory, the meta data comprising identified defect areas determined within at least one image of said scene including at least one real face.

12. The method of claim 11, wherein the reference image comprises a low resolution image including a subsample representation.

13. The method of claim 12, wherein the reference image comprises a subsample resolution representation.

14. The method of claim 12, wherein the reference image comprises a segmented representation of the main image.

15. The method of claim 12, further comprising changing a degree of subsampling.

16. The method of claim 11, wherein the reference image comprises a precapture image captured just before the main image is captured.

17. The method of claim 11, further comprising correcting exposure, focus or composition of the main image, or combinations thereof, based on analysis of the reference image acquired by the separate sensor.

18. The method of claim 11, further comprising aligning the reference image with the main image.

19. The method of claim 18, wherein the aligning comprises estimating camera or object movement, or both, in one or more intervals between capture of the main and reference images.

20. The method of claim 11, further comprising generating the reference image in parallel with the main image.

21. One or more processor-readable digital storage media having code embodied therein for programming one or more processors of a portable digital camera-enabled device that includes a main image sensor and a separate reference image sensor to perform a method of correcting a face defect within a digital image, wherein the method comprises:

providing illumination during main image acquisition with a flash;

acquiring with the main image sensor a main image of a scene including at least one real face (contra an image of a face);

acquiring with the separate sensor a reference image temporally proximate to acquisition of the main image and including a representation nominally of said same scene as the main image including said at least one real face;

correcting a face defect within the main image acquired by the main image sensor based on an analysis of the reference image acquired by the separate sensor including the representation nominally of said same scene as the main image including said at least one real face;

providing reference image data to a dedicated hardware subsystem from the separate sensor; and

providing raw image data and meta data to a main system memory, the meta data comprising identified defect areas determined within at least one image of said scene including at least one real face.

22. The one or more processor-readable digital storage media of claim 21, wherein the reference image comprises a low resolution image including a subsample representation.

23. The one or more processor-readable digital storage media of claim 22, wherein the reference image comprises a subsample resolution representation.

24. The one or more processor-readable digital storage media of claim 22, wherein the reference image comprises a segmented representation of the main image.

25. The one or more processor-readable digital storage media of claim 22, wherein the method further comprises changing a degree of subsampling.

26. The one or more processor-readable digital storage media of claim 21, wherein the reference image comprises a precapture image captured just before the main image is captured.

27. The one or more processor-readable digital storage media of claim 21, wherein the method further comprises correcting exposure, focus or composition of the main image, or combinations thereof, based on analysis of the reference image acquired by the separate sensor.

28. The one or more processor-readable digital storage media of claim 21, wherein the method further comprises aligning the reference image with the main image.

29. The one or more processor-readable digital storage media of claim 28, wherein the aligning comprises estimating camera or object movement, or both, in one or more intervals between capture of the main and reference images.

30. The one or more processor-readable digital storage media of claim 21, wherein the method further comprises generating the reference image in parallel with the main image.