Method and apparatus of correcting hybrid flash artifacts in digital images

• US 7,865,036 B2
• Filed: 09/14/2009
• Issued: 01/04/2011
• Est. Priority Date: 11/18/2005
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Ciuc, Mihai, Capata, Adrian, Nanu, Florin, Corcoran, Peter
• Primary Examiner(s): Do; Anh Hong
• Application Number:
  US12/558,859
• Publication Number:
  US 20100040284A1
• Time in Patent Office: 477 Days
• Field of Search: 382/275, 382/167, 382/165, 382/103, 382/117, 382/209, 382/254, 382/173, 348/241, 348/222.1
• US Class Current: 382/275
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal

Abstract

A method for digital image eye artifact detection and correction include identifying one or more candidate red-eye defect regions in an acquired image. For one or more candidate red-eye regions, a seed pixels and/or a region of pixels having a high intensity value in the vicinity of the candidate red-eye region is identified. The shape, roundness or other eye-related characteristic of a combined hybrid region including the candidate red-eye region and the region of high intensity pixels is analyzed. Based on the analysis of the eye-related characteristic of the combined hybrid region, it is determined whether to apply flash artifact correction, including red eye correction of the candidate red-eye region and/or correction of the region of high intensity pixels.

Claims

1. A digital camera, comprising:

(a) a lens;

(b) an image sensor;

(c) a processor and

(d) a computer readable medium having computer readable code embodied therein for programming the processor to perform a method of digital image artifact correction, the method comprising;

acquiring a digital image, including capturing a digital image using the lens and

the image sensor or receiving a digital image captured by an optical system of another device, or combinations thereof;

identifying a candidate red-eye defect region in said image;

identifying a region of high intensity pixels having at least a threshold intensity value within a sub-region of the image that also includes the candidate red-eye region;

analyzing an eye-related characteristic of the sub-region that comprises a combined hybrid region including said candidate red-eye region and said region of high intensity pixels;

identifying said combined hybrid region as a flash artifact region based on said analyzing of said eye-related characteristic; and

applying flash artifact correction to said flash artifact region.

2. The digital camera of claim 1, the method further comprising defining a bounding box around said candidate red-eye defect region.

3. The digital camera of claim 1, wherein identifying said region of high intensity pixels comprises identifying a seed high intensity pixel by locating said seed high intensity pixel within said bounding box.

4. The digital camera of claim 3, wherein said seed pixel has a yellowness above a pre-determined threshold and a redness below a pre-determined threshold.

5. The digital camera of claim 3, the method further comprising defining said region of high intensity pixels around said seed pixel.

6. The digital camera of claim 5, wherein said analyzing comprises calculating a difference in roundness between said candidate red-eye region and said combined region.

7. The digital camera of claim 6, wherein said red-eye correction is applied when said roundness of the combined hybrid region is greater than a threshold value.

8. The digital camera of claim 6, the method further comprising determining to apply said red-eye correction when a roundness of the combined hybrid region is greater than a roundness of the candidate red-eye region by a threshold amount.

9. The digital camera of claim 5, the method further comprising determining to not apply correction when said region of high intensity pixels comprises greater than a threshold area.

10. The digital camera of claim 9, wherein said area is determined as a relative function to a size of said bounding box.

11. The digital camera of claim 5, the method further comprising determining a yellowness and a non-pinkness of said region of high intensity pixels.

12. The digital camera of claim 11, wherein said acquired image is in LAB color space, and wherein the method further comprises measuring an average b value of said region of high intensity pixels and determining a difference between an average a value and the average b value of said region of high intensity pixels.

13. The digital camera of claim 5, wherein said analyzing further comprises analyzing said combined hybrid region for the presence of a glint, and determining to not correct said region of high intensity pixels responsive to the presence of glint.

14. The digital camera of claim 5, the method further comprising correcting said region of high intensity pixels by selecting one or more pixel values from a corrected red-eye region and employing said pixel values to correct said region of high intensity pixels.

15. The digital camera of claim 14, wherein said selected pixel values are taken from pixels having L and b values falling within a median for the corrected red-eye region.

16. The digital camera of claim 1, the method further comprising determining to not apply correction when an average b value of said region of high intensity pixels exceeds a relatively low threshold or if a difference between average a and b values is lower than a pre-determined threshold.

17. The digital camera of claim 1, the method further comprising converting said acquired image to one of RGB, YCC or Lb color space formats, or combinations thereof.

18. The digital camera of claim 1, wherein said analyzing of said acquired image is performed in Luminance chrominance color space and said region of high intensity pixels has a luminance value greater than a luminance threshold, and blue-yellow chrominance values greater than a chrominance threshold and a red-green value less than a red-green threshold.

19. The digital camera of claim 1, the method further comprising:

filtering said red-eye candidate regions to confirm or reject said regions as red-eye defect regions; and

selecting a subset of said rejected red-eye candidate regions for analysis as hybrid regions.

20. The digital camera of claim 1, the method being implemented as part of an image acquisition process.

21. The digital camera of claim 1, the method being implemented as part of a playback option in said digital camera.

22. The digital camera of claim 1, the method being implemented to run as a background process in said digital camera.

23. The digital camera of claim 1, the method being implemented within a general purpose computing device and wherein said acquiring comprises receiving said digital image from said digital camera.

24. The digital camera of claim 1, wherein said flash artifact correction includes red eye correction of said candidate red-eye region.

25. The digital camera of claim 24, wherein said flash artifact correction further comprises correction of said region of high intensity pixels.

26. The digital camera of claim 24, wherein correcting said region of high intensity pixels comprises using corrected pixel values based on said candidate red-eye region.

27. The digital camera of claim 24, wherein results of correcting said candidate red-eye region and said region of high intensity pixels are combined in such a manner as to obfuscate a seam between the regions.

28. The digital camera of claim 24, the method further comprising smoothing a seam region between said candidate red-eye region and said region of high intensity pixels.

29. The digital camera of claim 1, wherein said eye-related characteristic comprises shape.

30. The digital camera of claim 1, wherein said eye-related characteristic comprises roundness.

31. The digital camera of claim 1, wherein said eye-related characteristic comprises relative pupil size.

32. A digital camera, comprising:

(a) a lens,

(b) an image sensor,

(c)a processor and

(d) a computer readable medium having computer readable code embodied therein for programming the processor to perform a method of digital image artifact correction, the method comprising;

acquiring a digital image, including capturing a digital image using the lens and

the image sensor or receiving a digital image captured by an optical system of another device, or combinations thereof;

identifying a candidate red-eye defect region in said image;

identifying a seed pixel having a high intensity value within a sub-region of the image that also includes the candidate red-eye region;

analyzing an eye-related characteristic of the sub-region that comprises a combined hybrid region including said candidate red-eye region and said seed pixel;

identifying said combined hybrid region as a flash artifact region based on said analyzing of said eye-related characteristic; and

applying flash artifact correction to said flash artifact region.

33. The digital camera of claim 32, wherein said seed pixel has a yellowness above a pre-determined threshold and a redness below a pre-determined threshold.

34. The digital camera of claim 32, the method further comprising:

filtering said red-eye candidate regions to confirm or reject said regions as red-eye defect regions; and

selecting a subset of said rejected red-eye candidate regions for analysis as hybrid regions.

35. The digital camera of claim 32, the method being implemented as part of an image acquisition process.

36. The digital camera of claim 32, the method being implemented as part of a playback option in said digital camera.

37. The digital camera of claim 32, the method being implemented to run as a background process in said digital camera.

38. The digital camera of claim 32, the method being implemented within a general purpose computing device and wherein said acquiring comprises receiving said digital image at said digital camera.

39. The digital camera of claim 32, wherein the analyzing comprises checking whether an average b value exceeds a relatively low threshold.

40. The digital camera of claim 39, wherein the analyzing comprises checking whether a difference between an average a value and the average b value is lower than a given threshold.

41. The digital camera of claim 32, wherein said flash artifact correction includes red eye correction of said candidate red-eye region.

42. The digital camera of claim 32, wherein said flash artifact correction further comprises correction of a second region that includes said seed pixel.