Detection and correction of flash artifacts from airborne particulates

• US 8,073,286 B2
• Filed: 08/09/2007
• Issued: 12/06/2011
• Est. Priority Date: 08/09/2006
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: David, Emilian, Drimbarean, Alexandru, Corcoran, Peter, Steinberg, Eran, Vertan, Constantin, Zamfir, Adrian
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US11/836,744
• Publication Number:
  US 20080075385A1
• Time in Patent Office: 1,580 Days
• Field of Search: 382/167, 382/254, 382/264, 382/274, 382/275, 382/312, 382/299, 382/305, 348/222.1, 348/223.1, 348/224.1, 348/371, 358/3.26, 358/3.27, 358/515, 358/518
• US Class Current: 382/275
• CPC Class Codes: G06T 2207/20032 Median filtering
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation

Abstract

Flash image orb artifacts arising from specular reflections from airborne particles are corrected. A specific location is detected within a digital image of a flash image airborne particle artifact (orb). A defined curved plane shape is electronically identified within the image. Luminances are analyzed of pixels within the identified shape to assess whether the shape in fact corresponds to an orb. The digital image is corrected by removing the orb. One or more pixel values are adjusted inside the orb, and one or more edge pixel values of the orb are also adjusted.

Claims

1. A method of correcting airborne particle artifacts from digital images, comprising:

detecting a specific location within a digital image of an airborne particle artifact (orb), including;

electronically identifying a defined curved plane shape within the image; and

analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

correcting the digital image by removing the orb, including;

adjusting one or more pixel luminances inside the orb;

adjusting one or more edge pixel luminances of the orb; and

sub-sampling an original image to generate a sub-sampled image, wherein the sub-sampling comprises reducing angular resolution.

2. The method of claim 1, further comprising cropping an original image to generate a crop that includes the orb.

3. The method of claim 2, further comprising smoothing the crop.

4. The method of claim 3, further comprising median-filtering the crop.

5. The method of claim 2, further comprising transforming the crop to polar coordinates.

6. The method of claim 5, wherein the transforming is performed only for a valid region corresponding approximately to a largest circle that can be fit within the crop.

7. The method of claim 2, further comprising selecting a crop dimension that is less than twice an orb dimension.

8. The method of claim 1, wherein the luminance analyzing comprises comparing gray level luminance values in crossing an orb margin in the radial direction.

9. The method of claim 8, wherein the correcting comprises reducing a brightness of inside pixels and edge pixels in accordance with the comparing.

10. The method of claim 1, wherein the defined curved plane shape approximately comprises a circle.

11. The method of claim 1 wherein the adjusting of pixel values includes applying an in-painting technique to correct saturated pixels.

12. The method of claim 11, wherein a determination of predefined accuracy of an original pixel value is deemed insufficient without said in-painting techniques being applied.

13. The method of claim 1 wherein the adjusting of pixel values includes adjusting pixel luminance values.

14. The method of claim 13, wherein the adjusting of pixel luminance values does not include adjusting pixel chrominance values.

15. A method of correcting airborne particle artifacts from digital images, comprising:

detecting a specific location within a digital image of an airborne particle artifact (orb), including;

electronically identifying a defined curved plane shape within the image; and

analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

correcting the digital image by removing the orb, including;

adjusting one or more pixel luminances inside the orb; and

adjusting one or more edge pixel luminances of the orb cropping an original image to generate a crop that includes the orb, transforming the crop to polar coordinates; and

obtaining an angular resolution in the polar coordinates according to the relation;

16. The method of claim 15, computing the following along the angular direction:

17. The method of claim 16, further comprising determining a derivative of I(r), and determining an orb margin based thereon.

18. The method of claim 17, further comprising applying a moving average filter to the derivative, and finding local minima.

19. The method of claim 17, further comprising determining a derivative of I(r), and determining a sign of the derivative, and determining an orb margin based thereon.

20. The method of claim 19, further comprising determining a confidence measure M(x,y) according to the following:

21. The method of claim 20, further comprising determining an orb center at a minimum confidence value.

22. The method of claim 15, further comprising sub-sampling an original image to generate a sub-sampled image.

23. The method of claim 22, wherein the sub-sampling comprises reducing angular resolution.

24. The method of claim 23, wherein the sub-sampling comprises performing one or more calculations with a reduced interval of radial values.

25. A method of correcting a flash image orb artifact arising from specular reflections from airborne particles, comprising:

receiving user input as to general location (crop) within a digital image of a flash image airborne particle artifact (orb);

detecting a more specific location within the digital image of the orb, including;

electronically identifying a defined curved plane shape within the crop; and

analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

correcting the digital image by removing the orb, including;

adjusting one or more pixel values inside the orb; and

adjusting one or more edge pixel values of the orb,

wherein the detecting comprises;

(i) detecting an area of interest containing the orb; and

(ii) detecting an orb binary mask, and

wherein the method further comprises computing a gray level histogram of the area of interest.

26. The method of claim 25, further comprising sub-sampling an original image to generate a sub-sampled image.

27. The method of claim 25, further comprising:

(1) thresholding according to Otsu method; and

(2) checking orb location and shape.

28. The method of claim 27, further comprising reducing gray level range including determining that orb is not ok.

29. The method of claim 25, further comprising segmenting image in at least two classes.

30. The method of claim 29, further comprising identifying orb class and selecting a desired orb segmentation.

31. The method of claim 25, wherein the user input comprises a point within the orb.

32. The method of claim 31, further comprising:

(1) computing an orb kernel and kernel margins;

(2) computing a contrast-enhanced area of interest;

(3) for each point in the orb kernel, finding same line/column org margins;

(4) filtering a set of orb margin points;

(5) estimating a circular orb center location to generate an estimated orb; and

(6) computing an area of interest around the estimated orb.

33. The method of claim 25, wherein the user input comprises a rectangular region of interest containing the orb.

34. The method of claim 25, wherein the defined curved plane shape approximately comprises a circle.

35. The method of claim 25 wherein the adjusting of pixel values includes applying an in-painting technique to correct saturated pixels.

36. The method of claim 35, wherein a determination of predefined accuracy of an original pixel value is deemed insufficient without said in-painting techniques being applied.

37. The method of claim 25 wherein the adjusting of pixel values includes adjusting pixel luminance values.

38. The method of claim 37, wherein the adjusting of pixel luminance values does not include adjusting pixel chrominance values.

39. A method of correcting a flash image orb artifact arising from specular reflections from airborne particles, comprising:

receiving user input as to general location (crop) within a digital image of a flash image airborne particle artifact (orb);

detecting a more specific location within the digital image of the orb, including;

electronically identifying a defined curved plane shape within the crop; and

analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

correcting the digital image by removing the orb, including;

adjusting one or more pixel values inside the orb; and

adjusting one or more edge pixel values of the orb,

wherein the user input comprises a rectangular region of interest containing the orb,

wherein an orb margin is not more than 50 pixels from a border of the rectangular region of interest.

40. The method of claim 39, wherein the detecting comprises:

(i) detecting an area of interest containing the orb; and

(ii) detecting an orb binary mask.

41. The method of claim 39, further comprising computing a gray level histogram of the area of interest.

42. A method of correcting a flash image orb artifact arising from specular reflections from airborne articles, comprising:

receiving user input as to general location (crop) within a digital image of a flash image airborne particle artifact (orb);

detecting a more specific location within the digital image of the orb, including;

electronically identifying a defined curved plane shape within the crop; and

analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

correcting the digital image by removing the orb, including;

adjusting one or more pixel values inside the orb; and

adjusting one or more edge pixel values of the orb,

wherein the user input comprises a rectangular region of interest containing the orb,

wherein the method further comprises;

extracting contours within the rectangular region of interest;

eliminating vertical and horizontal oriented contour points;

finding potential orb region and potential orb contour;

estimating orb region size and circular orb center location to generate an estimated orb; and

computing an area of interest around the estimated orb.

43. One or more non-transitory, processor-readable media having program code embodied therein for programming one or more processors to perform a method of correcting a flash image orb artifact arising from specular reflections from airborne particles, the method comprising:

detecting a specific location within a digital image of an airborne particle artifact (orb), including;

electronically identifying a defined curved plane shape within the image; and

analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

correcting the digital image by removing the orb, including;

adjusting one or more pixel luminances inside the orb;

adjusting one or more edge pixel luminances of the orb; and

sub-sampling an original image to generate a sub-sampled image, wherein the sub-sampling comprises reducing angular resolution.

44. The one or more non-transitory processor-readable media of claim 43, the method further comprising cropping an original image to generate a crop that includes the orb.

45. The one or more non-transitory processor-readable media of claim 44, the method further comprising smoothing the crop.

46. The one or more non-transitory processor-readable media of claim 45, further comprising median-filtering the crop.

47. The one or more non-transitory processor-readable media of claim 44, further comprising transforming the crop to polar coordinates.

48. The one or more non-transitory processor-readable media of claim 47, wherein the transforming is performed only for a valid region corresponding approximately to a largest circle that can be fit within the crop.

49. The one or more non-transitory processor-readable media of claim 43, further comprising selecting a crop dimension that is less than twice an orb dimension.

50. The one or more non-transitory processor-readable media of claim 43, wherein the luminance analyzing comprises comparing gray level luminance values in crossing an orb margin in the radial direction.

51. The one or more non-transitory processor-readable media of claim 43, further comprising sub-sampling an original image to generate a sub-sampled image.

52. The one or more non-transitory processor-readable media of claim 43, wherein the defined curved plane shape approximately comprises a circle.

53. The one or more non-transitory processor-readable media of claim 43, wherein the adjusting of pixel values includes applying an in-painting technique to correct saturated pixels.

54. The one or more non-transitory processor-readable media of claim 43, wherein the adjusting of pixel values includes adjusting pixel luminance values.

55. One or more non-transitory, processor-readable media having program code embodied therein for programming one or more processors to perform a method of correcting a flash image orb artifact arising from specular reflections from airborne particles, the method comprising:

detecting a specific location within a digital image of an airborne particle artifact orb), including;

electronically identifying a defined curved plane shape within the image; and

analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

correcting the digital image by removing the orb, including;

adjusting one or more pixel luminances inside the orb; and

adjusting one or more edge pixel luminances of the orb cropping an original image to generate a crop that includes the orb, transforming the crop to polar coordinates; and

obtaining an angular resolution in the polar coordinates according to the relation;

56. One or more non-transitory, processor-readable media having program code embodied therein for programming one or more processors to perform a method of correcting a flash image orb artifact arising from specular reflections from airborne particles, the method comprising:

receiving user input as to general location (crop) within a digital image of a flash image airborne particle artifact (orb);

detecting a more specific location within the digital image of the orb, including;

electronically identifying a defined curved plane shape within the crop; and

analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

correcting the digital image by removing the orb, including;

adjusting one or more pixel values inside the orb; and

adjusting one or more edge pixel values of the orb,

wherein the detecting comprises;

detecting an area of interest containing the orb; and

detecting an orb binary mask, and

wherein the method further comprises computing a gray level histogram of the area of interest.

57. The one or more non-transitory processor-readable media of claim 56, wherein the detecting comprises:

(i) detecting an area of interest containing the orb; and

(ii) detecting an orb binary mask.

58. The one or more non-transitory processor-readable media of claim 57, further comprising sub-sampling an original image to generate a sub-sampled image.

59. The one or more non-transitory processor-readable media of claim 57, further comprising computing a gray level histogram of the area of interest.

60. The one or more non-transitory processor-readable media of claim 56, wherein the user input comprises a point within the orb.

61. The one or more non-transitory processor-readable media of claim 56, wherein the user input comprises a rectangular region of interest containing the orb.

62. The one or more non-transitory processor-readable media of claim 56, wherein the defined curved plane shape approximately comprises a circle.

63. The one or more non-transitory processor-readable media of claim 56 wherein the adjusting of pixel values includes applying an in-painting technique to correct saturated pixels.

64. The one or more non-transitory processor-readable media of claim 56 wherein the adjusting of pixel values includes adjusting pixel luminance values.

65. One or more non-transitory, processor-readable media having program code embodied therein for programming one or more processors to perform a method of correcting a flash image orb artifact arising from specular reflections from airborne particles, the method comprising:

receiving user input as to general location (crop) within a digital image of a flash image airborne particle artifact (orb);

detecting a more specific location within the digital image of the orb, including;

electronically identifying a defined curved plane shape within the crop; and

analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

correcting the digital image by removing the orb, including;

adjusting one or more pixel values inside the orb; and

adjusting one or more edge pixel values of the orb,

wherein the user input comprises a rectangular region of interest containing the orb,

wherein an orb margin is not more than 50 pixels from a border of the rectangular region of interest.

66. One or more non-transitory, processor-readable media having program code embodied therein for programming one or more processors to perform a method of correcting a flash image orb artifact arising from specular reflections from airborne particles, the method comprising:

receiving user input as to general location (crop) within a digital image of a flash image airborne particle artifact (orb);

detecting a more specific location within the digital image of the orb, including;

electronically identifying a defined curved plane shape within the crop; and

analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

correcting the digital image by removing the orb, including;

adjusting one or more pixel values inside the orb; and

adjusting one or more edge pixel values of the orb,

wherein the user input comprises a rectangular region of interest containing the orb,

wherein the method further comprises;

extracting contours within the rectangular region of interest;

eliminating vertical and horizontal oriented contour points;

finding potential orb region and potential orb contour;

estimating orb region size and circular orb center location to generate an estimated orb; and

computing an area of interest around the estimated orb.