Detection and Correction of Flash Artifacts from Airborne Particulates

• US 20080075385A1
• Filed: 08/09/2007
• Published: 03/27/2008
• Est. Priority Date: 08/09/2006
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: David, Emilian, Steinberg, Eran, Zamfir, Adrian, Drimbarean, Alexandru, Corcoran, Peter, Vertan, Constantin
• Granted Patent: US 8,073,286 B2
• Time in Patent Office: Days
• US Class Current: 382/275
• CPC Class Codes: G06T 2207/20032 Median filtering
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation

Abstract

Flash image orb artifacts arising from specular reflections from airborne particles are corrected. A specific location is detected within a digital image of a flash image airborne particle artifact (orb). A defined curved plane shape is electronically identified within the image. Luminances are analyzed of pixels within the identified shape to assess whether the shape in fact corresponds to an orb. The digital image is corrected by removing the orb. One or more pixel values are adjusted inside the orb, and one or more edge pixel values of the orb are also adjusted.

Claims

1. A method of correcting a flash image orb artifact arising from specular reflections from airborne particles, comprising:

(a) detecting a specific location within a digital image of a flash image airborne particle artifact (orb), including;

(i) electronically identifying a defined curved plane shape within the image; and

(ii) analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

(b) correcting the digital image by removing the orb, including;

(iii) adjusting one or more pixel values inside the orb; and

(iv) adjusting one or more edge pixel values of the orb.

2. The method of claim 1, further comprising cropping an original image to generate a crop that includes the orb.

3. The method of claim 2, further comprising smoothing the crop.

4. The method of claim 3, further comprising median-filtering the crop.

5. The method of claim 2, further comprising transforming the crop to polar coordinates.

6. The method of claim 5, wherein the transforming is performed for a valid region corresponding approximately to a largest circle that can be fit within the crop.

7. The method of claim 5, further comprising obtaining an angular resolution in the polar coordinates according to the relation:

8. The method of claim 7, computing the following along the angular direction:

9. The method of claim 8, further comprising determining a derivative of I(r), and determining an orb margin based thereon.

10. The method of claim 9, further comprising applying a moving average filter to the derivative, and finding local minima.

11. The method of claim 9, further comprising determining a derivative of I(r), and determining a sign of the derivative, and determining an orb margin based thereon.

12. The method of claim 11, further comprising determining a confidence measure M(x,y) according to the following:

13. The method of claim 12, further comprising determining an orb center at a minimum confidence value.

14. The method of claim 2, further comprising selecting a crop dimension that is less than twice an orb dimension.

15. The method of claim 1, wherein the luminance analyzing comprises comparing gray level luminance values in crossing an orb margin in the radial direction.

16. The method of claim 15, wherein the correcting comprises reducing a brightness of inside pixels and edge pixels in accordance with the comparing.

17. The method of claim 1, further comprising sub-sampling an original image to generate a sub-sampled image.

18. The method of claim 17, wherein the sub-sampling comprises reducing angular resolution.

19. The method of claim 18, wherein the sub-sampling comprises performing one or more calculations with a reduced interval of radial values.

20. The method of claim 1, wherein the defined curved plane shape approximately comprises a circle.

21. The method of claim 1 wherein the adjusting of pixel values includes applying an in-painting technique to correct saturated pixels.

22. The method of claim 21, wherein a determination of predefined accuracy of an original pixel value is deemed insufficient without said in-painting techniques being applied.

23. The method of claim 1 wherein the adjusting of pixel values includes adjusting pixel luminance values

24. The method of claim 23, wherein the adjusting of pixel luminance values does not include adjusting pixel chrominance values.

25. A method of correcting a flash image orb artifact arising from specular reflections from airborne particles, comprising:

(a) receiving user input as to general location (crop) within a digital image of a flash image airborne particle artifact (orb);

(b) detecting a more specific location within the digital image of the orb, including;

(i) electronically identifying a defined curved plane shape within the crop; and

(ii) analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

(c) correcting the digital image by removing the orb, including;

(iii) adjusting one or more pixel values inside the orb; and

(iv) adjusting one or more edge pixel values of the orb.

26. The method of claim 25, wherein the detecting comprises:

(i) detecting an area of interest containing the orb; and

(ii) detecting an orb binary mask.

27. The method of claim 26, further comprising sub-sampling an original image to generate a sub-sampled image.

28. The method of claim 26, further comprising computing a gray level histogram of the area of interest.

29. The method of claim 28, further comprising:

(1) thresholding according to Otsu method; and

(2) checking orb location and shape.

30. The method of claim 29, further comprising reducing gray level range including determining that orb is not ok.

31. The method of claim 28, further comprising segmenting image in at least two classes.

32. The method of claim 31, further comprising identifying orb class and selecting a desired orb segmentation.

33. The method of claim 25, wherein the user input comprises a point within the orb.

34. The method of claim 33, further comprising:

(1) computing an orb kernel and kernel margins;

(2) computing a contrast-enhanced area of interest;

(3) for each point in the orb kernel, finding same line/column org margins;

(4) filtering a set of orb margin points;

(5) estimating a circular orb center location to generate an estimated orb; and

(6) computing an area of interest around the estimated orb.

35. The method of claim 25, wherein the user input comprises a rectangular region of interest containing the orb.

36. The method of claim 35, wherein an orb margin is not more than 50 pixels from a border of the rectangular region of interest.

37. The method of claim 35, further comprising:

(1) extracting contours within the rectangular region of interest;

(2) eliminating vertical and horizontal oriented contour points;

(3) finding potential orb region and potential orb contour;

(4) estimating orb region size and circular orb center location to generate an estimated orb; and

(5) computing an area of interest around the estimated orb.

38. The method of claim 25, wherein the defined curved plane shape approximately comprises a circle.

39. The method of claim 25 wherein the adjusting of pixel values includes applying an in-painting technique to correct saturated pixels.

40. The method of claim 39, wherein a determination of predefined accuracy of an original pixel value is deemed insufficient without said in-painting techniques being applied.

41. The method of claim 25 wherein the adjusting of pixel values includes adjusting pixel luminance values

42. The method of claim 41, wherein the adjusting of pixel luminance values does not include adjusting pixel chrominance values.

43. One or more processor-readable media having program code embodied therein for programming one or more processors to perform a method of correcting a flash image orb artifact arising from specular reflections from airborne particles, the method comprising:

(a) detecting a specific location within a digital image of a flash image airborne particle artifact (orb), including;

(i) electronically identifying a defined curved plane shape within the image; and

(ii) analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

(b) correcting the digital image by removing the orb, including;

(iii) adjusting one or more pixel values inside the orb; and

(iv) adjusting one or more edge pixel values of the orb.

44. The one or more media of claim 43, the method further comprising cropping an original image to generate a crop that includes the orb.

45. The one or more media of claim 44, the method further comprising smoothing the crop.

46. The one or more media of claim 45, further comprising median-filtering the crop.

47. The one or more media of claim 44, further comprising transforming the crop to polar coordinates.

48. The one or more media of claim 47, wherein the transforming is performed only for a valid region corresponding approximately to a largest circle that can be fit within the crop.

49. The one or more media of claim 47, further comprising obtaining an angular resolution in the polar coordinates according to the relation:

50. The one or more media of claim 43, further comprising selecting a crop dimension that is less than twice an orb dimension.

51. The one or more media of claim 43, wherein the luminance analyzing comprises comparing gray level luminance values in crossing an orb margin in the radial direction.

52. The one or more media of claim 43, further comprising sub-sampling an original image to generate a sub-sampled image.

53. The one or more media of claim 43, wherein the defined curved plane shape approximately comprises a circle.

54. The one or more media of claim 43, wherein the adjusting of pixel values includes applying an in-painting technique to correct saturated pixels.

55. The one or more media of claim 43, wherein the adjusting of pixel values includes adjusting pixel luminance values

56. One or more processor-readable media having program code embodied therein for programming one or more processors to perform a method of correcting a flash image orb artifact arising from specular reflections from airborne particles, the method comprising:

(a) receiving user input as to general location (crop) within a digital image of a flash image airborne particle artifact (orb);

(b) detecting a more specific location within the digital image of the orb, including;

(i) electronically identifying a defined curved plane shape within the crop; and

(ii) analyzing luminances of pixels within the identified shape to assess whether said shape in fact corresponds to an orb; and

(c) correcting the digital image by removing the orb, including;

(iii) adjusting one or more pixel values inside the orb; and

(iv) adjusting one or more edge pixel values of the orb.

57. The one or more media of claim 56, wherein the detecting comprises:

(i) detecting an area of interest containing the orb; and

(ii) detecting an orb binary mask.

58. The one or more media of claim 57, further comprising sub-sampling an original image to generate a sub-sampled image.

59. The one or more media of claim 57, further comprising computing a gray level histogram of the area of interest.

60. The one or more media of claim 56, wherein the user input comprises a point within the orb.

61. The one or more media of claim 56, wherein the user input comprises a rectangular region of interest containing the orb.

62. The one or more media of claim 56, wherein the defined curved plane shape approximately comprises a circle.

63. The one or more media of claim 56 wherein the adjusting of pixel values includes applying an in-painting technique to correct saturated pixels.

64. The one or more media of claim 56 wherein the adjusting of pixel values includes adjusting pixel luminance values