In-camera based method of detecting defect eye with high accuracy

• US 8,081,254 B2
• Filed: 08/14/2008
• Issued: 12/20/2011
• Est. Priority Date: 08/14/2008
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Nanu, Florin, Corcoran, Peter
• Primary Examiner(s): Tran, Sinh
• Assistant Examiner(s): GEBRIEL, SELAM T
• Application Number:
  US12/191,304
• Publication Number:
  US 20100039520A1
• Time in Patent Office: 1,223 Days
• Field of Search: 348/240, 348/362, 348/367, 348/373, 348/370, 396/88, 396/155, 396/157, 396/158, 396/161, 396/177
• US Class Current: 348/370
• CPC Class Codes: G03B 15/05 Combinations of cameras with electronic flash apparatus; Electronic flash units
• CPC Class Codes: G03B 19/023 Multi-image cameras
• CPC Class Codes: G03B 2215/0503 Built-in units
• CPC Class Codes: G03B 2215/0571 With second light source
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

A portable digital image acquisition device includes multiple lenses and/or multiple flashes. A main digital image and first and second reference images are acquired. The first and second reference images are acquired with different flash-lens combinations that have different flash-lens distances. One or more flash eye defects are detected and corrected in the main image based on analysis of the first and second reference images.

Claims

1. A portable digital image acquisition device, comprising:

(a) a lens and a sensor for acquiring a main digital image and first and second reference images;

(b) first and second light sources disposed at different distances from the lens for illuminating one or more objects within an acquired scene;

(c) a memory for storing the main digital image;

(d) one or more non-transitory processor-readable media having digital code embedded therein for programming a processor to perform a method of determining and correcting a flash eye defect in the main digital image using the first and second reference images, wherein the method comprises;

(i) acquiring the first and second reference images respectively using the first and second light sources;

(ii) normalizing the first and second reference images to generate normalized first and second reference images;

(iii) analyzing one or more differences between the first and second normalized reference images;

(iv) determining and correcting the flash eye defect within the main image based on the analyzing to generate a corrected main image; and

(v) storing, transmitting, communicating, displaying, or projecting the corrected main image or a further processed version, or combinations thereof.

2. The device of claim 1, wherein the analyzing comprises subtracting the first and second reference images.

3. The device of claim 1, wherein the normalizing comprises luminance normalizing.

4. The device of claim 1, wherein the flash eye defect comprises a red eye defect.

5. The device of claim 1, wherein the normalizing comprises re-sizing one or both of the first and second reference images.

6. The device of claim 1, wherein the normalizing comprises aligning the first and second reference images.

7. The device of claim 6, wherein the aligning comprises aligning one or more eyes within the first and second images.

8. The device of claim 1, wherein the differences comprise color differences.

9. The device of claim 1, wherein the differences comprise brightness differences.

10. The device of claim 1, wherein the first and second reference images are each acquired using relatively low-intensity pre-flashes compared with an intensity of a flash used in acquiring the main digital image.

11. The device of claim 1, wherein the analyzing takes into account the specific different distances from the lens of the first and second light sources.

12. The device of claim 11, wherein the analyzing further takes into account distance to one or more objects, a gaze angle, an ambient lighting condition, color of an iris or a skin tone of a face within the acquired scene, or combinations thereof.

13. The device of claim 1, wherein the first and second preview images are acquired consecutively prior to acquiring the main digital image.

14. A method of determining and correcting a flash eye defect in a main digital image using first and second reference images, wherein the method comprises using a processor-based digital image acquisition device to perform the following:

acquiring first and second reference images respectively using first and second light sources disposed at different distances from a camera lens;

normalizing the first and second reference images to generate normalized first and second reference images;

analyzing one or more differences between the first and second normalized reference images;

determining and correcting a flash eye defect within the main image based on the analyzing to generate a corrected main image; and

storing, transmitting, communicating, displaying, or projecting the corrected main image or a further processed version, or combinations thereof.

15. The method of claim 14, wherein the analyzing comprises subtracting the first and second reference images.

16. The method of claim 14, wherein the normalizing comprises luminance normalizing.

17. The method of claim 14, wherein the flash eye defect comprises a red eye defect.

18. The method of claim 14, wherein the normalizing comprises re-sizing one or both of the first and second reference images.

19. The method of claim 14, wherein the normalizing comprises aligning the first and second reference images.

20. The method of claim 19, wherein the aligning comprises aligning one or more eyes within the first and second images.

21. The method of claim 14, wherein the differences comprise color differences.

22. The method of claim 14, wherein the differences comprise brightness differences.

23. The method of claim 14, wherein the first and second reference images are each acquired using relatively low-intensity pre-flashes compared with an intensity of a flash used in acquiring the main digital image.

24. The method of claim 14, wherein the analyzing takes into account the specific different distances from the lens of the first and second light sources.

25. The method of claim 24, wherein the analyzing further takes into account distance to one or more objects, a gaze angle, an ambient lighting condition, color of an iris or a skin tone of a face within the acquired scene, or combinations thereof.

26. The method of claim 14, wherein the first and second preview images are acquired consecutively prior to acquiring the main digital image.

27. One or more non-transitory processor-readable media having digital code embedded therein for programming a processor to perform a method of determining and correcting a flash eye defect in a main digital image using first and second reference images acquired with first and second light sources respectively disposed different distances from a camera lens, wherein the method comprises:

normalizing the first and second reference images to generate normalized first and second reference images;

analyzing one or more differences between the first and second normalized reference images;

determining and correcting the flash eye defect within the main image based on the analyzing to generate a corrected main image; and

storing, transmitting, communicating, displaying, or projecting the corrected main image or a further processed version, or combinations thereof.

28. The one or more media of claim 27, wherein the analyzing comprises subtracting the first and second reference images.

29. The one or more media of claim 27, wherein the normalizing comprises luminance normalizing.

30. The one or more media of claim 27, wherein the flash eye defect comprises a red eye defect.

31. The one or more media of claim 27, wherein the normalizing comprises re-sizing one or both of the first and second reference images.

32. The one or more media of claim 27, wherein the normalizing comprises aligning the first and second reference images.

33. The one or more media of claim 32, wherein the aligning comprises aligning one or more eyes within the first and second images.

34. The one or more media of claim 27, wherein the differences comprise color differences.

35. The one or more media of claim 27, wherein the differences comprise brightness differences.

36. The one or more media of claim 27, wherein the first and second reference images are each acquired using relatively low-intensity pre-flashes compared with an intensity of a flash used in acquiring the main digital image.

37. The one or more media of claim 27, wherein the analyzing takes into account the specific different distances from the lens of the first and second light sources.

38. The one or more media of claim 37, wherein the analyzing further takes into account distance to one or more objects, a gaze angle, an ambient lighting condition, color of an iris or a skin tone of a face within the acquired scene, or combinations thereof.

39. The one or more media of claim 27, wherein the first and second preview images are acquired consecutively prior to acquiring the main digital image.