Method for automatic detection and correction of halo artifacts in images

• US 20090161983A1
• Filed: 12/21/2007
• Published: 06/25/2009
• Est. Priority Date: 12/21/2007
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: SONY CORPORATION AND SONY ELECTRONICS, INC.
• Inventors: Ciurea, Florian
• Granted Patent: US 8,090,214 B2
• Time in Patent Office: Days
• US Class Current: 382/275
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/003 Deblurring; Sharpening vibration or motion blur correction for cameras comprising an electronic image sensor H04N23/682
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

A method of automatically detecting and correcting halo artifacts within a processed image is described. The method computes a two-dimensional (2D) gradient field of the original image and a 2D gradient field of the processed image. Each gradient field includes a gradient vector corresponding to each pixel. To detect halo artifacts, the gradient vector at each pixel of the original image is compared to the gradient vector at the corresponding pixel of the processed image. A halo artifact is determined to exist at a given pixel if a direction of the two corresponding gradient vectors differs by at least a specified threshold. To correct the halo artifacts, a composite gradient field is generated using one of three correction methods. A final image is generated by integrating the newly generated composite gradient field using known integration methods from a 2D gradient field such as ones based on the Fast Fourier Transform.

Claims

1. A method of detecting halo artifacts in an image, the method comprising:

a. determining a first two-dimensional gradient field for an image, wherein the first gradient field includes a plurality of first gradient vectors, one first gradient vector associated with each pixel in the image;

b. determining a second two-dimensional gradient field for the processed image, wherein the second gradient field includes a plurality of second gradient vectors, one second gradient vector associated with each pixel in the processed image, wherein a first pixel in the image corresponds to a first pixel in the processed image;

c. calculating a gradient angle for each pixel, wherein a first gradient angle is an angle between a first gradient vector and a second gradient vector associated with the first pixel; and

d. calculating a relative angle for each pixel, wherein a first relative angle for the first pixel is the minimum between the first gradient angle and 360 degrees minus the first gradient angle.

2. The method of claim 1 wherein the first gradient angle between the first gradient vector and the second gradient vector is the absolute value.

3. The method of claim 1 wherein the first gradient field is determined according to a scale that identifies a pixel separation distance used in the determination of each first gradient vector.

4. The method of claim 3 wherein the second gradient field is determined according to the scale.

5. The method of claim 4 wherein the scale increases as noise in the image increases.

6. The method of claim 1 wherein the image is a grayscale image.

7. The method of claim 1 wherein if the image is a color image, the method further comprises converting the image to a grayscale image before determining the first two-dimensional gradient field.

8. The method of claim 1 wherein a halo artifact value at each pixel is defined as the relative angle at the pixel.

9. The method of claim 1 wherein a halo artifact value at each pixel is defined as a magnitude of the second gradient vector at the pixel

10. A method of correcting halo artifacts in an image, the method comprising:

a. determining a first two-dimensional gradient field for an image, wherein the first gradient field includes a plurality of first gradient vectors, each first gradient vector includes a first magnitude and a first direction;

b. determining a second two-dimensional gradient field for the processed image, wherein the second gradient field includes a plurality of second gradient vectors, each second gradient vector includes a second magnitude and a second direction, further wherein each second gradient vector corresponds to a specific first gradient vector, thereby forming a first gradient vector and second gradient vector pair;

c. generating a third two-dimensional gradient field, wherein the third gradient field includes a plurality of third gradient vectors, each third gradient vector corresponds to a specific first gradient vector and second gradient vector pair, and each third gradient vector includes a third magnitude equal to the second magnitude of the corresponding second gradient vector and a third direction equal to the first direction of the corresponding first gradient vector; and

d. integrating the third gradient field thereby generating a corrected image.

11. The method of claim 10 wherein integrating the third gradient field comprises integrating the third gradient field according to known integrating methods from a two-dimensional gradient field based on a Fast Fourier Transform.

12. The method of claim 10 wherein the plurality of first gradient vectors includes one first gradient vector for each pixel in the image.

13. The method of claim 12 wherein the plurality of second gradient vectors includes one second gradient vector for each pixel in the processed image, wherein a first pixel in the image corresponds to a first pixel in the processed image.

14. The method of claim 13 wherein one first gradient vector and second gradient vector pair corresponds to each pixel.

15. The method of claim 10 wherein the first gradient field is determined according to a scale that identifies a pixel separation distance used in the determination of each first gradient vector.

16. The method of claim 15 wherein the second gradient field is determined according to the scale.

17. The method of claim 10 wherein the image is a grayscale image.

18. The method of claim 10 wherein if the image is a color image, the method further comprises converting the image to a grayscale image before determining a first two-dimensional gradient field.

19. A method of detecting and correcting halo artifacts in an image, the method comprising:

a. determining a first two-dimensional gradient field for an image, wherein the first gradient field includes a plurality of first gradient vectors, each first gradient vector includes a first magnitude and a first direction;

b. determining a second two-dimensional gradient field for the processed image, wherein the second gradient field includes a plurality of second gradient vectors, each second gradient vector includes a second magnitude and a second direction;

c. quantifying a halo artifact value at each pixel;

d. generating a third two-dimensional gradient field, wherein the third gradient field includes a plurality of third gradient vectors, each third gradient vector includes a third magnitude and a third direction, further wherein if the halo artifact value at a given pixel is less than a predetermined value, then the third magnitude equals the second magnitude and the third direction equals the second direction at the given pixel, and if the halo artifact value at the given pixel is greater than or equal to the predetermined value, then the third magnitude equals the second magnitude and the third direction equals the first direction at the given pixel; and

e. integrating the third gradient field thereby generating a corrected image.

20. The method of claim 19 wherein quantifying a halo artifact value at each pixel comprises:

a. calculating a gradient angle for each pixel, wherein the gradient angle defines a difference in the direction between the first gradient vector and the second gradient vector at the given pixel; and

b. calculating a relative angle for each pixel, wherein the relative angle for the given pixel is the minimum between the gradient angle at the given pixel and 360 degrees minus the gradient angle at the given pixel, further wherein the relative angle equals the halo artifact value at each pixel.

21. The method of claim 19 wherein each second gradient vector corresponds to a specific first gradient vector, thereby forming a first gradient vector and second gradient vector pair, and each third gradient vector corresponds to a specific first gradient vector and second gradient vector pair.

22. The method of claim 19 wherein one first gradient vector is associated with each pixel in the image and one second gradient vector is associated with each pixel in the processed image, further wherein a first pixel in the image corresponds to a first pixel in the processed image.

23. The method of claim 22 wherein a first gradient angle is an angle between a first gradient vector and a second gradient vector associated with the first pixel.

24. The method of claim 19 wherein integrating the third gradient field comprises integrating the third gradient filed according to known integration methods from a two-dimensional gradient field based on a Fast Fourier Transform.

25. The method of claim 19 wherein the first gradient field is determined according to a scale that identifies a pixel separation distance used in the determination of each first gradient vector.

26. The method of claim 25 wherein the second gradient field is determined according to the scale.

27. The method of claim 19 wherein the image is a grayscale image.

28. The method of claim 19 wherein if the image is a color image, the method further comprises converting the image to a grayscale image before determining a first two-dimensional gradient field.

29. A method of detecting and correcting halo artifacts in an image, the method comprising:

a. determining a first two-dimensional gradient field for an image, wherein the first gradient field includes a plurality of first gradient vectors, each first gradient vector includes a first magnitude and a first direction;

b. determining a second two-dimensional gradient field for the processed image, wherein the second gradient field includes a plurality of second gradient vectors, each second gradient vector includes a second magnitude and a second direction;

c. quantifying a halo artifact value at each pixel;

d. generating a third two-dimensional gradient field, wherein the third gradient field includes a plurality of third gradient vectors, each third gradient vector includes a third magnitude and a third direction, further wherein if the halo artifact value at a given pixel is less than a predetermined value, then the third magnitude equals the second magnitude and the third direction equals the first direction at the given pixel, and if the halo artifact value at the given pixel is greater than or equal to the predetermined value, then the third magnitude equals the first magnitude and the third direction equals the first direction at the given pixel; and

e. integrating the third gradient field thereby generating a corrected image.

30. The method of claim 29 wherein quantifying the halo artifact value at each pixel comprises:

a. calculating a gradient angle for each pixel, wherein the gradient angle defines a difference in the direction between the first gradient vector and the second gradient vector at the given pixel; and

b. calculating a relative angle for each pixel, wherein the relative angle for the given pixel is the minimum between the gradient angle at the given pixel and 360 degrees minus the gradient angle at the given pixel, further wherein the relative angle equals the halo artifact value at each pixel.

31. The method of claim 30 wherein the predetermined value is 90 degrees.

32. The method of claim 29 wherein each second gradient vector corresponds to a specific first gradient vector, thereby forming a first gradient vector and second gradient vector pair, and each third gradient vector corresponds to a specific first gradient vector and second gradient vector pair.

33. The method of claim 29 wherein one first gradient vector is associated with each pixel in the image and one second gradient vector is associated with each pixel in the processed image, further wherein a first pixel in the image corresponds to a first pixel in the processed image.

34. The method of claim 33 wherein a first gradient angle is an angle between a first gradient vector and a second gradient vector associated with the first pixel.

35. The method of claim 29 wherein integrating the third gradient field comprises integrating the third gradient field according to known integration methods from a two-dimensional gradient field based on a Fast Fourier Transform.

36. The method of claim 29 wherein the first gradient field is determined according to a scale that identifies a pixel separation distance used in the determination of each first gradient vector.

37. The method of claim 36 wherein the second gradient field is determined according to the scale.

38. The method of claim 29 wherein the image is a grayscale image.

39. The method of claim 29 wherein if the image is a color image, the method further comprises converting the image to a grayscale image before determining a first two-dimensional gradient field.

40. An apparatus for detecting halo artifacts in an image, the apparatus comprising:

a. an application for;

i. determining a first two-dimensional gradient field for an image, wherein the first gradient field includes a plurality of first gradient vectors, each first gradient vector includes a first magnitude and a first direction;

ii. determining a second two-dimensional gradient field for the processed image, wherein the second gradient field includes a plurality of second gradient vectors, each second gradient vector includes a second magnitude and a second direction;

iii. quantifying a halo artifact value at each pixel;

iv. generating a third two-dimensional gradient field, wherein the third gradient field includes a plurality of third gradient vectors, each third gradient vector includes a third magnitude and a third direction, further wherein if the halo artifact value at a given pixel is less than a predetermined value, then the third magnitude equals the second magnitude and the third direction equals the second direction at the given pixel, and if the halo artifact value at the given pixel is greater than or equal to the predetermined value, then the third magnitude equals the second magnitude and the third direction equals the first direction at the given pixel; and

v. integrating the third gradient field thereby generating a corrected image; and

b. a processor configured for executing the application; and

c. a memory coupled to the processor, the memory configured for temporarily storing data for execution by the processor.

41. An apparatus for correcting halo artifacts in an image, the apparatus comprising:

a. an application for;

i. determining a first two-dimensional gradient field for an image, wherein the first gradient field includes a plurality of first gradient vectors, each first gradient vector includes a first magnitude and a first direction;

ii. determining a second two-dimensional gradient field for the processed image, wherein the second gradient field includes a plurality of second gradient vectors, each second gradient vector includes a second magnitude and a second direction, further wherein each second gradient vector corresponds to a specific first gradient vector, thereby forming a first gradient vector and second gradient vector pair;

iii. generating a third two-dimensional gradient field, wherein the third gradient field includes a plurality of third gradient vectors, each third gradient vector corresponds to a specific first gradient vector and second gradient vector pair, and each third gradient vector includes a third magnitude equal to the second magnitude of the corresponding second gradient vector and a third direction equal to the first direction of the corresponding first gradient vector; and

iv. integrating the third gradient field thereby generating a corrected image; and

b. a processor configured for executing the application; and

c. a memory coupled to the processor, the memory configured for temporarily storing data for execution by the processor.

42. An apparatus for detecting halo artifacts in an image, the apparatus comprising:

a. an application for;

i. determining a first two-dimensional gradient field for an image, wherein the first gradient field includes a plurality of first gradient vectors, each first gradient vector includes a first magnitude and a first direction;

ii. determining a second two-dimensional gradient field for the processed image, wherein the second gradient field includes a plurality of second gradient vectors, each second gradient vector includes a second magnitude and a second direction;

iii. quantifying a halo artifact value at each pixel;

iv. generating a third two-dimensional gradient field, wherein the third gradient field includes a plurality of third gradient vectors, each third gradient vector includes a third magnitude and a third direction, further wherein if the halo artifact value at a given pixel is less than a predetermined value, then the third magnitude equals the second magnitude and the third direction equals the first direction at the given pixel, and if the halo artifact value at the given pixel is greater than or equal to the predetermined value, then the third magnitude equals the first magnitude and the third direction equals the first direction at the given pixel; and

v. integrating the third gradient field thereby generating a corrected image; and

b. a processor configured for executing the application; and

c. a memory coupled to the processor, the memory configured for temporarily storing data for execution by the processor.