Method and apparatus of correcting hybrid flash artifacts in digital images

• US 7,599,577 B2
• Filed: 11/18/2005
• Issued: 10/06/2009
• Est. Priority Date: 11/18/2005
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Ciuc, Mihai, Capata, Adrian, Nanu, Florin, Corcoran, Peter
• Primary Examiner(s): Do; Anh Hong
• Application Number:
  US11/282,955
• Publication Number:
  US 20070116380A1
• Time in Patent Office: 1,418 Days
• Field of Search: 382/275, 382/167, 396/569, 396/570, 355/40, 348/370, 348/576, 348/222.1, 348/E5.001
• US Class Current: 382/275
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal

Abstract

A method for digital image eye artifact detection and correction include identifying one or more candidate red-eye defect regions in an acquired image. For one or more candidate red-eye regions, a seed pixels and/or a region of pixels having a high intensity value in the vicinity of the candidate red-eye region is identified. The shape, roundness or other eye-related characteristic of a combined hybrid region including the candidate red-eye region and the region of high intensity pixels is analyzed. Based on the analysis of the eye-related characteristic of the combined hybrid region, it is determined whether to apply flash artifact correction, including red eye correction of the candidate red-eye region and/or correction of the region of high intensity pixels.

Claims

1. A method for digital image artifact correction comprising using a digital image acquisition device or external image processing device, or a combination thereof, including a processor that is to perform the method, wherein the method further comprises:

acquiring a digital image;

identifying a candidate red-eye defect region in said image;

identifying a region of high intensity pixels having at least a threshold intensity value in a vicinity of said candidate red-eye region;

analyzing an eye-related characteristic of a combined hybrid region including said candidate red-eye region and said region of high intensity pixels; and

identifying said combined hybrid region as a flash artifact region based on said analyzing of said eye-related characteristic; and

applying flash artifact correction to said flash artifact region.

2. A method as claimed in claim 1, further comprising defining a bounding box that contains said candidate red-eye defect region.

3. A method as claimed in claim 2, wherein identifying said region of high intensity pixels comprises identifying a seed high intensity pixel located within said bounding box.

4. A method as claimed in claim 3, wherein said seed pixel has a yellowness above a pre-determined threshold and a redness below a pre-determined threshold.

5. A method as claimed in claim 3, further comprising defining said region of high intensity pixels around said seed pixel.

6. A method as claimed in claim 5, wherein said analyzing comprises calculating a difference in roundness between said candidate red-eye region and said combined hybrid region.

7. A method as claimed in claim 6, further comprising determining to apply said red-eye correction when a roundness of the combined hybrid region is greater than a threshold value.

8. A method as claimed in claim 6, further comprising determining to apply said red-eye correction when a roundness of the combined hybrid region is greater than a roundness of the candidate red-eye region by a threshold amount.

9. A method as claimed in claim 5, further comprising determining to not apply correction when said region of high intensity pixels comprises greater than a threshold area.

10. A method as claimed in claim 9, wherein said area is determined as a relative function to a size of said bounding box.

11. A method according to claim 5, further comprising determining a yellowness and a non-pinkness of said region of high intensity pixels.

12. A method as claimed in claim 11, wherein said acquired image is in LAB color space, and wherein the method further comprises measuring an average b value of said region of high intensity pixels and determining a difference between an average a value and the average b value of said region of high intensity pixels.

13. A method as claimed in claim 5, wherein said analyzing further comprises analyzing said combined hybrid region for the presence of a glint, and determining to not correct said region of high intensity pixels responsive to the presence of glint.

14. A method as claimed in claim 5, further comprising correcting said region of high intensity pixels by selecting one or more pixel values from a corrected red-eye region and employing said pixel values to correct said region of high intensity pixels.

15. A method as claimed in claim 14, wherein said selected pixel values are taken from pixels having L and b values falling within a median for the corrected red-eye region.

16. A method as claimed in claim 1, further comprising determining to not apply correction when an average b value of said region of high intensity pixels exceeds a relatively low threshold or if a difference between average a and b values is lower than a pre-determined threshold.

17. A method as claimed in claim 1, further comprising convening said acquired image to one of RGB, YCC or Lab color space formats, or combinations thereof.

18. A method as claimed in claim 1, wherein said analyzing of said acquired image is performed in Luminance-chrominance color space and said region of high intensity pixels has a luminance value greater than a luminance threshold, and blue-yellow chrominance values greater than a chrominance threshold and a red-green value less than a red-green threshold.

19. A method as claimed in claim 1, further comprising:

filtering said red-eye candidate regions to confirm or reject said regions as red-eye defect regions; and

selecting a subset of said rejected red-eye candidate regions for analysis as hybrid regions.

20. A method as claimed in claim 1, and implemented within a digital image acquisition device.

21. A method as claimed in claim 20, and implemented as part of an image acquisition process.

22. A method as claimed in claim 20, and implemented as part of a playback option in said digital image acquisition device.

23. A method as claimed in claim 1, and implemented to run as a background process in a digital image acquisition device.

24. A method as claimed in claim 1, and implemented within a general purpose computing device and wherein said acquiring comprises receiving said digital image from a digital image acquisition device.

25. A method as claimed in claim 1, wherein said flash artifact correction includes red eye correction of said candidate red-eye region.

26. A method as claimed in claim 25, wherein said flash artifact correction further comprises correction of said region of high intensity pixels.

27. A method as claimed in claim 25, wherein correcting said region of high intensity pixels comprises using corrected pixel values based on said candidate red-eye region.

28. A method as claimed in claim 25, wherein results of correcting said candidate red-eye region and said region of high intensity pixels are combined in such a manner as to obfuscate a seam between the regions.

29. A method as claimed in claim 25, further comprising smoothing a seam region between said candidate red-eye region and said region of high intensity pixels.

30. A method as claimed in claim 1, wherein said eye-related characteristic comprises shape.

31. A method as claimed in claim 1, wherein said eye-related characteristic comprises roundness.

32. A method as claimed in claim 1, wherein said eye-related characteristic comprises relative pupil size.

33. A method for digital image artifact correction, comprising using a digital image acquisition device or external image processing device, or a combination thereof, including a processor that is to perform the method, wherein the method further comprises:

acquiring a digital image;

identifying a candidate red-eye defect region in said image;

identifying a seed pixel having a high intensity value in the vicinity of said candidate red-eye region;

analyzing an eye-related characteristic of a combined hybrid region including said candidate red-eye region and said seed pixel;

identifying said combined hybrid region as a flash artifact region based on said analyzing of said eye-related characteristic; and

applying flash artifact correction to said flash artifact region.

34. A method as claimed in claim 33, wherein said seed pixel has a yellowness above a pre-determined threshold and a redness below a pre-determined threshold.

35. A method as claimed in claim 33, further comprising:

filtering said red-eye candidate regions to confirm or reject said regions as red-eye defect regions; and

selecting a subset of said rejected red-eye candidate regions for analysis as hybrid regions.

36. A method as claimed in claim 33, and implemented within a digital image acquisition device.

37. A method as claimed in claim 36, and implemented as part of an image acquisition process.

38. A method as claimed in claim 36, and implemented as part of a playback option in said digital image acquisition device.

39. A method as claimed in claim 33, and implemented to run as a background process in a digital image acquisition device.

40. A method as claimed in claim 33, and implemented within a general purpose computing device and wherein said acquiring comprises receiving said digital image from a digital image acquisition device.

41. A method as claimed in claim 33, wherein the analyzing comprises checking whether an average b value exceeds a relatively low threshold.

42. A method as claimed in claim 41, wherein the analyzing comprises checking whether a difference between an average a value and the average b value is lower than a given threshold.

43. A method as claimed in claim 33, wherein said flash artifact correction includes red eye correction of said candidate red-eye region.

44. A method as claimed in claim 43, wherein said flash artifact correction further comprises correction of a second region that includes said seed pixel.

45. A computer readable medium having computer readable code embodied therein for programming one or more processors to perform a method of digital image artifact correction, the method comprising:

acquiring a digital image;

identifying a candidate red-eye defect region in said image;

identifying a region of high intensity pixels having at least a threshold intensity value in a vicinity of said candidate red-eye region;

analyzing an eye-related characteristic of a combined hybrid region including said candidate red-eye region and said region of high intensity pixels; and

identifying said combined hybrid region as a flash artifact region based on said analyzing of said eye-related characteristic; and

applying flash artifact correction to said flash artifact region.

46. The computer readable medium as claimed in claim 45, the method further comprising defining a bounding box around said candidate red-eye defect region.

47. The computer readable medium as claimed in claim 45, wherein identifying said region of high intensity pixels comprises identifying a seed high intensity pixel by locating said seed high intensity pixel within said bounding box.

48. The computer readable medium as claimed in claim 47, wherein said seed pixel has a yellowness above a pre-determined threshold and a redness below a pre-determined threshold.

49. The computer readable medium as claimed in claim 47, the method further comprising defining said region of high intensity pixels around said seed pixel.

50. The computer readable medium as claimed in claim 49, wherein said analyzing comprises calculating a difference in roundness between said candidate red-eye region and said combined region.

51. The computer readable medium as claimed in claim 50, wherein said red-eye correction is applied when said roundness of the combined hybrid region is greater than a threshold value.

52. The computer readable medium as claimed in claim 50, the method further comprising determining to apply said red-eye correction when a roundness of the combined hybrid region is greater than a roundness of the candidate red-eye region by a threshold amount.

53. The computer readable medium as claimed in claim 49, the method further comprising determining to not apply correction when said region of high intensity pixels comprises greater than a threshold area.

54. The computer readable medium as claimed in claim 53, wherein said area is determined as a relative function to a size of said bounding box.

55. The computer readable medium according to claim 49, the method further comprising determining a yellowness and a non-pinkness of said region of high intensity pixels.

56. The computer readable medium as claimed in claim 55, wherein said acquired image is in LAB color space, and wherein the method further comprises measuring an average b value of said region of high intensity pixels and determining a difference between an average a value and the average b value of said region of high intensity pixels.

57. The computer readable medium as claimed in claim 49, wherein said analyzing Further comprises analyzing said combined hybrid region for the presence of a glint, and determining to not correct said region of high intensity pixels responsive to the presence of glint.

58. The computer readable medium as claimed in claim 49, the method further comprising correcting said region of high intensity pixels by selecting one or more pixel values from a corrected red-eye region and employing said pixel values to correct said region of high intensity pixels.

59. The computer readable medium as claimed in claim 58, wherein said selected pixel values are taken from pixels having L and b values falling within a median for the corrected red-eye region.

60. The computer readable medium as claimed in claim 45, the method further comprising determining to not apply correction when an average b value of said region of high intensity pixels exceeds a relatively low threshold or if a difference between average a and b values is lower than a pre-determined threshold.

61. The computer readable medium as claimed in claim 45, the method further comprising converting said acquired image to one of RGB, YCC or Lb color space formats, or combinations thereof.

62. The computer readable medium as claimed in claim 45, wherein said analyzing of said acquired image is performed in Luminance chrominance color space and said region of high intensity pixels has a luminance value greater than a luminance threshold, and blue-yellow chrominance values greater than a chrominance threshold and a red-green value less than a red-green threshold.

63. The computer readable medium as claimed in claim 45, the method further comprising:

filtering said red-eye candidate regions to confirm or reject said regions as red-eye defect regions; and

selecting a subset of said rejected red-eye candidate regions for analysis as hybrid regions.

64. The computer readable medium as claimed in claim 45, and implemented within a digital image acquisition device.

65. The computer readable medium as claimed in claim 64, the method being implemented as part of an image acquisition process.

66. The computer readable medium as claimed in claim 64, the method being implemented as pail of a playback option in said digital image acquisition device.

67. The computer readable medium as claimed in claim 45, the method being implemented to run as a background process in a digital image acquisition device.

68. The computer readable medium as claimed in claim 45, the method being implemented within a general purpose computing device and wherein said acquiring comprises receiving said digital image from a digital image acquisition device.

69. The computer readable medium as claimed in claim 45, wherein said flash artifact correction includes red eye correction of said candidate red-eye region.

70. The computer readable medium as claimed in claim 69, wherein said flash artifact correction further comprises correction of said region of high intensity pixels.

71. The computer readable medium as claimed in claim 69, wherein correcting said region of high intensity pixels comprises using corrected pixel values based on said candidate red-eye region.

72. The computer readable medium as claimed in claim 69, wherein results of correcting said candidate red-eye region and said region of high intensity pixels are combined in such a manner as to obfuscate a seam between the regions.

73. The computer readable medium as claimed in claim 69, the method further comprising smoothing a seam region between said candidate red-eye region and said region of high intensity pixels.

74. The computer readable medium as claimed in claim 45, wherein said eye-related characteristic comprises shape.

75. The computer readable medium as claimed in claim 45, wherein said eye-related characteristic comprises roundness.

76. The computer readable medium as claimed in claim 45, wherein said eye-related characteristic comprises relative pupil size.

77. A computer readable medium having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of digital image artifact correction, the method comprising:

acquiring a digital image;

identifying a candidate red-eye defect region in said image;

identifying a seed pixel having a high intensity value in the vicinity of said candidate red-eye region;

analyzing an eye-related characteristic of a combined hybrid region including said candidate red-eye region and said seed pixel;

identifying said combined hybrid region as a flash artifact region based on said analyzing of said eye-related characteristic; and

applying flash artifact correction to said flash artifact region.

78. The computer readable medium as claimed in claim 77, wherein said seed pixel has a yellowness above a pre-determined threshold and a redness below a pre-determined threshold.

79. The computer readable medium as claimed in claim 77, the method further comprising:

filtering said red-eye candidate regions to confirm or reject said regions as red-eye defect regions; and

selecting a subset of said rejected red-eye candidate regions for analysis as hybrid regions.

80. The computer readable medium as claimed in claim 77, and implemented within a digital image acquisition device.

81. The computer readable medium as claimed in claim 80, the method being implemented as part of an image acquisition process.

82. The computer readable medium as claimed in claim 80, the method being implemented as part of a playback option in said digital image acquisition device.

83. The computer readable medium as claimed in claim 80, the method being implemented to run as a background process in a digital image acquisition device.

84. The computer readable medium as claimed in claim 80, the method being implemented within a general purpose computing device and wherein said acquiring comprises receiving said digital image from a digital image acquisition device.

85. The computer readable medium as claimed in claim 77, wherein the analyzing comprises checking whether an average b value exceeds a relatively low threshold.

86. The computer readable medium as claimed in claim 85, wherein the analyzing comprises checking whether a difference between an average a value and the average b value is lower than a given threshold.

87. The computer readable medium as claimed in claim 77, wherein said flash artifact correction includes red eye correction of said candidate red-eye region.

88. The computer readable medium as claimed in claim 77, wherein said flash artifact correction further comprises correction of a second region that includes said seed pixel.

89. An apparatus configured for correcting digital image artifact, comprising:

means for acquiring a digital image;

means for identifying a candidate red-eye defect region in said image;

means for identifying a region of high intensity pixels having at least a threshold intensity value in a vicinity of said candidate red-eye region;

means for analyzing an eye-related characteristic of a combined hybrid region including said candidate red-eye region and said region of high intensity pixels;

means for identifying said combined hybrid region as a flash artifact region based on said analyzing of said eye-related characteristic; and

means for applying flash artifact correction to said flash artifact region.

90. An apparatus as claimed in claim 89, further comprising means for defining a bounding box around said candidate red-eye defect region.

91. An apparatus as claimed in claim 89, wherein said means for analyzing further comprises means for analyzing said combined hybrid region for the presence of a glint, and further comprising means for determining to not correct said region of high intensity pixels responsive to the presence of glint.

92. An apparatus as claimed in claim 89, further comprising:

means for filtering said red-eye candidate regions to confirm or reject said regions as red-eye defect regions; and

means for selecting a subset of said rejected red-eye candidate regions for analysis as hybrid regions.

93. An apparatus as claimed in claim 89, wherein said flash artifact correction includes red eye correction of said candidate red-eye region.

94. An apparatus configured for digital image artifact correction, comprising:

means acquiring a digital image;

means for identifying a candidate red-eye defect region in said image;

means for identifying a seed pixel having a high intensity value in the vicinity of said candidate red-eye region;

means for analyzing an eye-related characteristic of a combined hybrid region including said candidate red-eye region and said seed pixel;

means for identifying said combined hybrid region as a flash artifact region based on said analyzing of said eye-related characteristic; and

means for applying flash artifact correction to said flash artifact region.