Red-eye filter method and apparatus

• US 8,885,074 B2
• Filed: 03/18/2014
• Issued: 11/11/2014
• Est. Priority Date: 08/05/2003
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Corcoran, Peter, Bigioi, Petronel, DeLuca, Michael J.
• Primary Examiner(s): LAM, HUNG H
• Application Number:
  US14/218,369
• Publication Number:
  US 20140198238A1
• Time in Patent Office: 238 Days
• Field of Search: 348/239, 348/241, 348/267, 348/276, 348/277, 348/340, 348/335, 348345-350, 348353-354
• US Class Current: 348/241
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 9/70 for colour killing

Abstract

A digital image acquisition system having no photographic film, such as a digital camera, has a flash unit for providing illumination during image capture and a red-eye filter for detecting a region within a captured image indicative of a red-eye phenomenon, the detection being based upon a comparison of the captured image and a reference image of nominally the same scene taken without flash. In the embodiment the reference image is a preview image of lower pixel resolution than the captured image, the filter matching the pixel resolutions of the captured and reference images by up-sampling the preview image and/or sub-sampling the captured image. The filter also aligns at least portions of the captured image and reference image prior to comparison to allow for, e.g. movement in the subject.

Claims

1. A digital camera device, comprising:

an image sensor configured to capture a main image of a scene and a preview image of the scene;

wherein the main image is captured with flash turned on;

wherein the preview image is captured with flash turned off;

a red-eye filter configured to;

determine whether a first region of the main image has a shape indicative of a red-eye phenomenon;

determine whether a second region of the preview image, that corresponds to the first region of the main image, has the shape indicative of the red-eye phenomenon; and

in response to determining that both the first region of the main image and the second region of the preview image have the shape indicative of the red-eye phenomenon, designate the first region of the main image as indicative of the red-eye phenomenon if the second region of the preview image does not have a color indicative of the red-eye phenomenon.

2. The digital camera device of claim 1, wherein the red-eye filter is further configured to:

determine whether the first region of the main image has the color indicative of the red-eye phenomenon;

determine whether the second region of the preview image has the color indicative of the red-eye phenomenon; and

in response to determining that both the first region of the main image and the second region of the preview image have the color indicative of the red-eye phenomenon, determine that the first region of the main image does not have the red-eye phenomenon.

3. The digital camera device of claim 1, wherein the red-eye filter is further configured to:

align the preview image with the main image by transforming the preview image to correlate the preview image with the main image based on measurable characteristics of both the preview image and the main image;

wherein the measurable characteristics include any one of;

a color, texture, or edge characteristics; and

wherein the transforming comprises any one of;

an image shifting, an image shearing, an image affine transformation.

4. The digital camera device of claim 1,

wherein the main image is a full-resolution image;

and

wherein the preview image is captured with a lower resolution than the main image.

5. The digital camera device of claim 1,

wherein the preview image comprises a subsample representation of the main image;

and

wherein the subsample representation comprises a subsample resolution representation.

6. The digital camera device of claim 5,

wherein the image capturing apparatus is further configured to change the subsample resolution representation by changing a degree of a subsampling with which the preview image is captured;

and

wherein changing the degree of subsampling is determined by any one of;

an empirical method, a size of the preview image, a size of the second region, a spline interpolation method, or a bi-cubic interpolation.

7. The digital camera device of claim 1,

wherein the image capturing apparatus is further configured to change the subsample resolution representation of the preview image by performing any one of:

sub-sampling, or up-sampling.

8. A method, comprising:

capturing a main image of a scene and a preview image of the scene;

wherein the main image is captured with flash turned on;

wherein the preview image is captured with flash turned off;

determining whether a first region of the main image has a shape indicative of a red-eye phenomenon;

determining whether a second region of the preview image, that corresponds to the first region of the main image, has the shape indicative of the red-eye phenomenon; and

in response to determining that both the first region of the main image and the second region of the preview image have the shape indicative of the red-eye phenomenon, designating the first region of the main image as indicative of the red-eye phenomenon if the second region of the preview image does not have a color indicative of the red-ey phenomenon;

wherein the method is performed by one or more computing devices.

9. The method of claim 8, further comprising:

determining whether the first region of the main image has the color indicative of the red-eye phenomenon;

determining whether the second region of the preview image has the color indicative of the red-eye phenomenon; and

in response to determining that both the first region of the main image and the second region of the preview image have the color indicative of the red-eye phenomenon, determining that the first region of the main image does not have the red-eye phenomenon.

10. The method of claim 8, further comprising:

aligning the preview image with the main image by transforming the preview image to correlate the preview image with the main image based on measurable characteristics of both the preview image and the main image;

wherein the measurable characteristics include any one of;

a color, texture, or edge characteristics; and

wherein the transforming comprises any one of;

an image shifting, an image shearing, an image affine transformation.

11. The method of claim 8,

wherein the main image is a full-resolution image;

and

wherein the preview image is captured with a lower resolution than the main image.

12. The method of claim 8,

wherein the preview image comprises a subsample representation of the main image;

and

wherein the subsample representation comprises a subsample resolution representation.

13. The method of claim 12,

wherein the image capturing apparatus is further configured to change the subsample resolution representation by changing a degree of a subsampling with which the preview image is captured;

and

wherein changing the degree of subsampling is determined by any one of;

an empirical method, a size of the preview image, a size of the second region, a spline interpolation method, or a bi-cubic interpolation.

14. The method of claim 8,

wherein the image capturing apparatus is further configured to change the subsample resolution representation of the preview image by performing any one of:

sub-sampling, or up-sampling.

15. One or more non-transitory computer-readable storage medium, storing one or more instructions which, when executed by one or more processors, cause performance of:

capturing a main image of a scene and a preview image of the scene;

wherein the main image is captured with flash turned on;

wherein the preview image is captured with flash turned off;

determining whether a first region of the main image has a shape indicative of a red-eye phenomenon;

determining whether a second region of the preview image, that corresponds to the first region of the main image, has the shape indicative of the red-eye phenomenon; and

in response to determining that both the first region of the main image and the second region of the preview image have the shape indicative of the red-eye phenomenon, designating the first region of the main image as indicative of the red-eye phenomenon if the second region of the preview image does not have a color indicative of the red-ey phenomenon.

16. The one or more non-transitory computer-readable storage medium of claim 15, storing additional instructions which, when executed, cause performance of:

determining whether the first region of the main image has the color indicative of the red-eye phenomenon;

determining whether the second region of the preview image has the color indicative of the red-eye phenomenon; and

in response to determining that both the first region of the main image and the second region of the preview image have the color indicative of the red-eye phenomenon, determining that the first region of the main image does not have the red-eye phenomenon.

17. The one or more non-transitory computer-readable storage medium of claim 15, storing additional instructions which, when executed, cause performance of:

aligning the preview image with the main image by transforming the preview image to correlate the preview image with the main image based on measurable characteristics of both the preview image and the main image;

wherein the measurable characteristics include any one of;

a color, texture, or edge characteristics; and

wherein the transforming comprises any one of;

an image shifting, an image shearing, an image affine transformation.

18. The one or more non-transitory computer-readable storage medium of claim 15,

wherein the main image is a full-resolution image;

and

wherein the preview image is captured with a lower resolution than the main image.

19. The one or more non-transitory computer-readable storage medium of claim 15,

wherein the preview image comprises a subsample representation of the main image;

and

wherein the subsample representation comprises a subsample resolution representation.

20. The one or more non-transitory computer-readable storage medium of claim 15,

wherein the image capturing apparatus is further configured to change the subsample resolution representation by changing a degree of a subsampling with which the preview image is captured;

and

wherein changing the degree of subsampling is determined by any one of;

an empirical method, a size of the preview image, a size of the second region, a spline interpolation method, or a bi-cubic interpolation.