Method and apparatus for red-eye detection using preview or other reference images

• US 8,320,641 B2
• Filed: 06/19/2008
• Issued: 11/27/2012
• Est. Priority Date: 10/28/2004
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Corcoran, Peter, Prilutsky, Yury, Bigioi, Petronel, Zamfir, Adrian, Nanu, Florin, Drimbarean, Alexandru
• Primary Examiner(s): DRENNAN, BARRY T
• Application Number:
  US12/142,134
• Publication Number:
  US 20080317339A1
• Time in Patent Office: 1,622 Days
• Field of Search: None
• US Class Current: 382/117
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 2207/20008 Globally adaptive
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

A method for red-eye detection in an acquired digital image acquiring one or more preview or other reference images without a flash. Any red regions that exist within the one or more reference images are determined. A main image is acquired with a flash of approximately a same scene as the one or more reference images. The main image is analyzed to determine any candidate red eye defect regions that exist within the main image. Any red regions determined to exist within the one or more reference images are compared with any candidate red eye defect regions determined to exist within the main image. Any candidate red eye defect regions within the main image corresponding to red regions determined also to exist within the one or more reference images are removed as candidate red eye defect regions.

Claims

1. A method for red-eye detection in an acquired digital image comprising:

acquiring one or more reference images without a flash;

determining any red regions that exist within said one or more reference images;

applying an image analysis prefilter to the one or more reference images;

applying an image correction analyzer to the one or more reference images based on information gained from the applying of the image analysis prefilter;

adapting a chain of multiple image compensation filters based on information gained from the applying of the image correction analyzer, including segmenting the one or more reference images and adapting corrective image processing to be applied differently to each of multiple regions, and adapting a global pixel transformation to be applied uniformly to the entire image;

acquiring a main image with a flash of approximately a same scene as said one or more reference images;

analyzing the main image to determine any candidate red eye defect regions that exist within the main image;

comparing said any red regions determined to exist within said one or more reference images with said any candidate red eye defect regions determined to exist within said main image; and

removing as candidate red eye defect regions within said main image any corresponding red regions determined also to exist within said one or more reference images; and

applying shape analysis and falsing analysis filters to remaining candidate red eye regions at least including region size and eye shape filters, and eye region falsing filters, preview image filter and metadata based filters; and

removing any remaining candidate red eye regions determined to be physically incompatible with known eye defects to have a location within a region that is physically incompatible with known eye defects or to have a high falsing likelihood, or combinations thereof.

2. A method according to claim 1 wherein said analyzing comprises applying a chain of multiple red-eye filters to said main image.

3. A method according to claim 2 wherein said red-eye filter chain comprises:

(i) a pixel locator and segmentor;

(ii) a shape analyzer including multiple shape filters;

(iii) a falsing analyzer including multiple falsing filters; and

(iv) a pixel modifier.

4. A method according to claim 3 wherein said pixel locator and segmentor includes a pixel transformer.

5. A method according to claim 1 further comprising correcting a third acquired image based on any red eye defect regions determined in said main image.

6. A method according to claim 5 wherein said third acquired image comprises a sub-sampled copy of said main image.

7. A method according to claim 1 wherein the analyzing comprises recognizing one or more faces or types of faces, or combinations thereof, in the main image.

8. A method according to claim 1, further comprising correcting said main image based on analysis of at least one reference image and determining a degree of blur;

a degree of dust contamination;

a color imbalance;

a white imbalance;

a gamma error;

a texture characteristic error;

or noise characteristics, or combinations thereof.

9. A method according to claim 8, further comprising recognizing a same face or type of face, or both, in a second image as in said main image, and applying one or more same corrective processes to said second image as said main image.

10. A method according to claim 1, further comprising determining correcting to apply to said main image based on analysis of at least one reference image, wherein said correcting comprises:

(i) contrast normalization and image sharpening;

(ii) image color adjustment and tone scaling;

(iii) exposure adjustment and digital fill flash;

(iv) brightness adjustment with color space matching;

(v) image auto-gamma determination with image enhancement;

(vi) image enhancement;

or

(vii) face based image enhancement, or combinations thereof.

11. A method as claimed in claim 10, comprising:

responsive to determining to apply correcting to the main image, also determining that such correcting cannot be beneficially applied to said main image, and disabling said correcting and providing an indication of such to a user.

12. A method as claimed in claim 10, wherein said determining to apply correcting includes interacting with a user to determine the corrections to be made.

13. A method according to claim 10, wherein said determining to apply correcting includes performing a color space transformation.

14. A method as claimed in claim 10, comprising determining a sequence in which to apply more than one correcting action to said main image.

15. A method as claimed in claim 1 in which the method is performed in whole or in part within a portable digital camera.

16. A method as claimed in claim 1, wherein said one or more reference images comprises one or more preview images.

17. A digital image acquisition device, comprising one or more optics and a sensor for acquiring digital images including main images and relatively low resolution reference images, a processor, and a processor readable medium having stored thereon digital code for programming the processor to perform a method of red eye detection in a digital image, wherein the method comprises

acquiring one or more reference images without a flash;

determining any red regions that exist within said one or more reference images;

applying an image analysis prefilter to the one or more reference images;

applying an image correction analyzer to the one or more reference images based on information gained from the applying of the image analysis prefilter;

adapting a chain of multiple image compensation filters based on information gained from the applying of the image correction analyzer, including segmenting the one or more reference images and adapting corrective image processing to be applied differently to each of multiple regions, and adapting a global pixel transformation to be applied uniformly to the entire image;

acquiring a main image with a flash of approximately a same scene as said one or more reference images;

analyzing the main image to determine any candidate red eye defect regions that exist within the main image;

comparing said any red regions determined to exist within said one or more reference images with said any candidate red eye defect regions determined to exist within said main image; and

removing as candidate red eye defect regions within said main image any corresponding red regions determined also to exist within said one or more reference images; and

applying shape analysis and falsing analysis filters to remaining candidate red eye regions at least including region size and eye shape filters, and eye region falsing filters, preview image filter and metadata based filters; and

removing any remaining candidate red eye regions determined to be physically incompatible with known eye defects to have a location within a region that is physically incompatible with known eye defects or to have a high falsing likelihood, or combinations thereof.

18. A device according to claim 17 wherein said analyzing comprises applying a chain of multiple red-eye filters to said main image.

19. A device according to claim 18 wherein said red-eye filter chain comprises:

(i) a pixel locator and segmentor;

(ii) a shape analyzer including multiple shape filters;

(iii) a falsing analyzer including multiple falsing filters; and

(iv) a pixel modifier.

20. A device according to claim 19 wherein said pixel locator and segmentor includes a pixel transformer.

21. A device according to claim 17 wherein the method further comprises correcting a third acquired image based on any red eye defect regions determined in said main image.

22. A device according to claim 21 wherein said third acquired image comprises a sub-sampled copy of said main image.

23. A device according to claim 17 wherein the analyzing comprises recognizing one or more faces or types of faces, or combinations thereof, in the main image.

24. A device according to claim 17, wherein the method further comprises correcting said main image based on analysis of at least one reference image and determining a degree of blur;

a degree of dust contamination;

a color imbalance;

a white imbalance;

a gamma error;

a texture characteristic error;

or noise characteristics, or combinations thereof.

25. A device according to claim 24, wherein the method further comprises recognizing a same face or type of face, or both, in a second image as in said main image, and applying one or more same corrective processes to said second image as said main image.

26. A device according to claim 17, wherein the method further comprises determining correcting to apply to said main image based on analysis of at least one reference image, wherein said correcting comprises:

(i) contrast normalization and image sharpening;

(ii) image color adjustment and tone scaling;

(iii) exposure adjustment and digital fill flash;

(iv) brightness adjustment with color space matching;

(v) image auto-gamma determination with image enhancement;

(vi) image enhancement;

or

(vii) face based image enhancement, or combinations thereof.

27. A device as claimed in claim 26, wherein the method further comprises, responsive to determining to apply correcting to the main image, also determining that such correcting cannot be beneficially applied to said main image, and disabling said correcting and providing an indication of such to a user.

28. A device as claimed in claim 26, wherein said determining to apply correcting includes interacting with a user to determine the corrections to be made.

29. A device according to claim 26, wherein said determining to apply correcting includes performing a color space transformation.

30. A device as claimed in claim 26, wherein the method further comprises determining a sequence in which to apply more than one correcting action to said main image.

31. A device as claimed in claim 17 in which the method is performed in whole or in part within a portable digital camera.

32. A device as claimed in claim 17, wherein the one or more reference images comprise one or more preview images.