Flash artifact eye defect correction in blurred images using anisotropic blurring

• US 8,180,173 B2
• Filed: 09/21/2007
• Issued: 05/15/2012
• Est. Priority Date: 09/21/2007
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Capata, Adrian, Drimbarean, Alexandru, Nanu, Florin
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): HEIDEMANN, JASON E
• Application Number:
  US11/859,164
• Publication Number:
  US 20090080796A1
• Time in Patent Office: 1,698 Days
• Field of Search: None
• US Class Current: 382/275
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection

Abstract

A method and apparatus for providing defect correction in blurred images is disclosed. For one embodiment of the invention, a digital image is acquired. One or more candidate defect regions in the image are identified. The candidate defect regions are corrected and responsive to the image being blurred, the corrected regions of the image are blurred. For one embodiment of the invention, a digital image is acquired and one or more candidate eye defect regions in the image are identified. The candidate eye defect regions are corrected and if the image is blurred, the corrected regions of the image are blurred accordingly.

Claims

1. A method for digital image flash artefact eye defect correction comprising:

acquiring a digital image;

identifying one or more candidate eye defect regions in the image comprising one or more flash artefact regions including pixels in one or more pupil regions that are red, yellow, golden or white, or combinations thereof;

determining whether the image is blurred;

correcting the one or more candidate eye defect regions, including replacing said red, yellow, golden or white pixels with corrected pixels to form one or more corrected regions, wherein the corrected regions comprise probabilistically uncertain defect pixels utilizing between 10% and 70% information derived from pixels of the acquired digital image, as well as probabilistically certain pixels not utilizing information from pixels of the acquired digital image, and uncorrected glint pixels having a luminance above a certain threshold to be classified as certain glint pixels; and

responsive to the image being blurred, anisotropically blurring the one or more corrected regions of the image.

2. The method of claim 1 wherein the determining further comprises determining a degree of blur of the image.

3. The method of claim 2 wherein the blurring further comprises blurring the corrected regions of the image according to the degree of blur.

4. The method of claim 2 wherein the degree of blur is a function of an edginess factor or a glint factor or a combination thereof.

5. The method of claim 4 further comprising applying an edge detector to pixels of an annular region defined between a border of the candidate eye defect region and a surrounding boundary region, to ascertain the edginess factor.

6. The method of claim 5 wherein applying the edge detector comprises determining a number of groups of pixels within the annular region having a change in intensity of greater than a threshold value with respect to neighbouring groups of pixels in the annular region, as a percentage of a total number of groups of pixels in the annular region.

7. The method of claim 6 wherein the group of pixels comprises at least one pixel.

8. The method of claim 7 wherein the annular region comprises a thickness of at least 2 pixels.

9. The method of claim 6 further comprising determining a size of a glint area as a percentage of the candidate eye defect region to ascertain the glint factor.

10. The method of claim 1 further comprising

obtaining a copy of the candidate eye defect regions;

wherein said correcting comprises correcting the copy; and

wherein said blurring comprises blurring the corrected copy.

11. The method of claim 10 further comprising blending the corrected copy into corresponding regions of the acquired image.

12. The method of claim 11, further comprising determining to neither correct, blur nor blend candidate eye defect regions with a luminance above a certain threshold.

13. The method of claim 11, wherein the identifying further comprises labeling the candidate eye defect regions as one of certain eye defect regions, non-certain eye defect regions, or holes within the candidate eye defect regions.

14. The method of claim 11 wherein the blending further comprises blending certain eye defect regions into the image using corresponding corrected and blurred regions.

15. The method of claim 11 wherein the blending further comprises blending the certain eye defect regions into the image using between 30% to 70% of the corresponding corrected and blurred regions.

16. The method of claim 11 wherein the blending further comprises blending the holes within candidate eye defect regions into the image using between 50% to 90% of the corresponding corrected and blurred regions.

17. The method of claim 1, wherein at least the identifying and correcting are performed by a hardware component.

18. A digital image processing device including a processor arranged to correct digital image flash artefact eye defects, comprising:

a flash artefact eye defect correction component configured both to identify one or more candidate eye defect regions in the image comprising one or more flash artefact regions including pixels in one or more pupil regions that are red, yellow, golden or white, or combinations thereof; and

to correct the one or more candidate eye defect regions, including replacing said red, yellow, golden or white pixels with corrected pixels to form one or more corrected regions, wherein the one or more corrected regions comprise probabilistically uncertain defect pixels utilizing between 10% and 70% information derived from pixels of the acquired digital image, as well as probabilistically certain pixels not utilizing information from pixels of the acquired digital image, and uncorrected glint pixels having a luminance above a certain threshold to be classified as certain glint pixels; and

an anisotropic blurring component configured to determine whether the image is blurred; and

, responsive to the image being blurred, anisotropically blurring the one or more corrected regions of the image.

19. The digital image processing device of claim 18, wherein the blurring component comprises an anisotropic diffusion blur component arranged to blur the corrected regions of the image.

20. The digital image device of claim 18, wherein the blurring component comprises an edge detector for determining a degree of blur of the image.

21. The digital image device of claim 20 wherein the edge detector comprises a Prewitt edge detector.

22. The device of claim 18, wherein at least the identifying and correcting are performed by a hardware component.

23. One or more non-transitory processor readable hardware components having code embedded therein for programming one or more processors to perform a method for digital image flash artefact eye defect correction, wherein the method comprises:

identifying one or more candidate eye defect regions in said image comprising one or more flash artefact regions including pixels in one or more pupil regions that are red, yellow, golden or white, or combinations thereof;

determining whether said image is blurred;

correcting said one or more candidate eye defect regions, including replacing said red, yellow, golden or white pixels with corrected pixels to form one or more corrected regions, wherein the corrected regions comprise probabilistically uncertain defect pixels utilizing between 10% and 70% information derived from pixels of the acquired digital image, as well as probabilistically certain pixels not utilizing information from pixels of the acquired digital image, and uncorrected glint pixels having a luminance above a certain threshold to be classified as certain glint pixels; and

responsive to said image being blurred, anisotropically blurring said one or more corrected regions of said image.

24. The one or more processor readable media of claim 23, wherein said determining further comprises determining a degree of blur of said image.

25. The one or more processor readable media of claim 24, wherein said blurring further comprises blurring said corrected regions of said image according to said degree of blur.

26. The one or more processor readable media of claim 24, wherein said degree of blur is a function of an edginess factor or a glint factor or a combination thereof.

27. The one or more processor readable media of claim 26, wherein the method further comprises applying an edge detector to pixels of an annular region defined between a border of said candidate eye defect region and a surrounding boundary region, to ascertain the edginess factor.

28. The one or more processor readable media of claim 27, wherein applying said edge detector comprises determining a number of groups of pixels within said annular region having a change in intensity of greater than a threshold value with respect to neighbouring groups of pixels in said annular region, as a percentage of a total number of groups of pixels in said annular region.

29. The one or more processor readable media of claim 28, wherein said group of pixels comprises at least one pixel.

30. The one or more processor readable media of claim 29, wherein said annular region comprises a thickness of at least 2 pixels.

31. The one or more processor readable media of claim 28, wherein the method further comprises determining a size of a glint area as a percentage of said candidate eye defect region to ascertain the glint factor.

32. The one or more processor readable media of claim 23, wherein the method further comprises:

obtaining a copy of said candidate eye defect regions;

wherein said correcting comprises correcting said copy; and

wherein said blurring comprises blurring said corrected copy.

33. The one or more processor readable media of claim 32, wherein the method further comprises blending said corrected copy into corresponding regions of said acquired image.

34. The one or more processor readable media of claim 33, wherein the method further comprises determining to neither correct, blur nor blend candidate eye defect regions with a luminance above a certain threshold.

35. The one or more processor readable media of claim 33, wherein the identifying further comprises labeling said candidate eye defect regions as one of certain eye defect regions, non-certain eye defect regions, or holes within said candidate eye defect regions.

36. The one or more processor readable media of claim 33, wherein said blending further comprises blending said certain eye defect regions into said image using said corresponding corrected and blurred regions.

37. The one or more processor readable media of claim 33, wherein said blending further comprises blending said non-certain eye defect regions into said image using between 30% to 70% of said corresponding corrected and blurred regions.

38. The one or more processor readable media of claim 33, wherein said blending further comprises blending holes within said candidate eye defect regions into said image using between 50% to 90% of said corresponding corrected and blurred regions.

39. One or more processor-readable media as in claim 23, wherein at least the identifying and correcting are performed by a hardware component.

40. A digital image acquisition device, comprising:

a camera including a flash for acquiring a digital image;

a processor;

a memory having code embedded therein for programming the processor to perform a method for digital image flash artefact eye defect correction, wherein the method comprises;

identifying one or more candidate eye defect regions in said digital image comprising one or more flash artefact regions each including pixels in a pupil region that are red, yellow, golden or white, or combinations thereof;

determining whether said digital image is blurred;

correcting said one or more candidate eye defect regions, including replacing said red, yellow, golden or white pixels with corrected pixels to form one or more corrected regions, wherein the corrected regions comprise probabilistically uncertain defect pixels utilizing between 10% and 70% information derived from pixels of the acquired digital image, as well as probabilistically certain pixels not utilizing information from pixels of the acquired digital image, and uncorrected glint pixels having a luminance above a certain threshold to be classified as certain glint pixels; and

responsive to said image being blurred, anisotropically blurring said one or more corrected regions of said digital image.

41. The device of claim 40, wherein said determining further comprises determining a degree of blur of said image.

42. The device of claim 41, wherein said blurring further comprises blurring said corrected regions of said image according to said degree of blur.

43. The device of claim 41, wherein said degree of blur is a function of an edginess factor or a glint factor or a combination thereof.

44. The device of claim 43, wherein the method further comprises applying an edge detector to pixels of an annular region defined between a border of said candidate eye defect region and a surrounding boundary region, to ascertain the edginess factor.

45. The device of claim 44, wherein applying said edge detector comprises determining a number of groups of pixels within said annular region having a change in intensity of greater than a threshold value with respect to neighbouring groups of pixels in said annular region, as a percentage of a total number of groups of pixels in said annular region.

46. The device of claim 45, wherein said group of pixels comprises at least one pixel.

47. The device of claim 46, wherein said annular region comprises a thickness of at least 2 pixels.

48. The device of claim 45, wherein the method further comprises determining a size of a glint area as a percentage of said candidate eye defect region to ascertain the glint factor.

49. The device of claim 40, wherein the method further comprises:

obtaining a copy of said candidate eye defect regions;

wherein said correcting comprises correcting said copy; and

wherein said blurring comprises blurring said corrected copy.

50. The device of claim 49, wherein the method further comprises blending said corrected copy into corresponding regions of said acquired image.

51. The device of claim 40, wherein the method further comprises determining to neither correct, blur nor blend candidate eye defect regions with a luminance above a certain threshold.

52. The device of claim 40, wherein the identifying further comprises labeling said candidate eye defect regions as one of certain eye defect regions, non-certain eye defect regions, or holes within said candidate eye defect regions.

53. The device of claim 40, wherein said blending further comprises blending said certain eye defect regions into said image using said corresponding corrected and blurred regions.

54. The device of claim 40, wherein said blending further comprises blending said non-certain eye defect regions into said image using between 30% to 70% of said corresponding corrected and blurred regions.

55. The device of claim 40, wherein said blending further comprises blending holes within said candidate eye defect regions into said image using between 50% to 90% of said corresponding corrected and blurred regions.

56. The device of claim 40, wherein at least the identifying and correcting are performed by a hardware component.