Filter for combined de-ringing and edge sharpening

• US 20030081854A1
• Filed: 06/12/2001
• Published: 05/01/2003
• Est. Priority Date: 06/12/2001
• Status: Active Grant

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Deshpande, Sachin G.
• Granted Patent: US 7,003,173 B2
• Time in Patent Office: Days
• US Class Current: 382/261
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 5/208 for compensating for attenuation of high frequency components, e.g. crispening, aperture distortion correction
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Abstract

A filter for post-processing digital images and videos, having an edge mapper, a pixel sorter, and an adaptive filter that simultaneously performs de-ringing and edge sharpening. The combined filter is computationally simpler than known methods and can achieve removal of ringing artifacts and sharpening of true edges at the same time. The present invention also includes a preferred method of simultaneously de-ringing and edge sharpening digital images and videos.

Claims

1. A filter system for post-processing a digital image, said digital image having a plurality of visual-edge pixels and a plurality of visual non-edge pixels, said filter system comprising:

(a) an edge mapper for producing a binary map of said visual edge pixels and said visual non-edge pixels;

(b) a pixel sorter comprising;

(i) said pixel sorter for reading said binary map; and

(ii) said pixel sorter for assigning to each pixel a type of filtration to be provided by said filter system;

(c) an adaptive filter for receiving output from said pixel sorter; and

(d) said adaptive filter comprising;

(i) a de-ringing module for post-processing said visual non-edge pixels; and

(ii) an edge sharpener for post-processing said edge pixels.

2. The filter system of claim 1, said edge mapper further comprising:

(a) an edge detector comprising;

(i) said edge detector for calculating intensity gradients for each pixel in said digital image;

(ii) said edge detector for assigning a first edge value to each edge pixel based on said intensity gradients; and

(iii) said edge detector for assigning a second edge value to each non-edge pixel based on said intensity gradients; and

(b) a memory storage array for storing said first edge value for each edge pixel and for storing said second edge value for each non-edge pixel.

3. The filter system of claim 2, wherein said edge detector uses at least one edge detection operator to calculate said intensity gradients selected from the group of edge detection operators consisting of:

(a) a Sobel edge detection operator;

(b) a Prewitt edge detection operator; and

(c) a Roberts edge detection operator.

4. The filter system of claim 2, wherein said edge detector uses Roberts edge detection operators H₁ and H₂ of the form:

5. The filter system of claim 2, said edge detector for executing an edge value subroutine for calculating said first edge value for each visual edge pixel and said second edge value for each visual non-edge pixel, said edge value subroutine further comprising:

(a) at least one edge detection operator for calculating intensity gradients for each pixel in said digital image;

(b) an i variable for storing a horizontal coordinate of each pixel in said digital image;

(c) a j variable for storing a vertical coordinate of each pixel in said digital image;

(d) g_H1(i, j) and g_H2(i, j) variables for storing intensity gradients calculated by said at least one edge detection operator;

(e) an EdgeStrength(i, j) variable for storing an average of said intensity gradients for each pixel in said digital image;

(f) an EdgeThreshold variable for storing a selectable threshold value for defining a true visual edge containing said visual edge pixels; and

(g) an EdgeValue(i, j) variable for storing said first edge value for each visual edge pixel and said second edge value for each visual non-edge pixel;

(h) wherein said edge value subroutine is defined as;

EdgeStrength(i, j) = (|g_H1(i, j)| + |g_H2(i, j)|)/2;

if (EdgeStrength(i, j) >

EdgeThreshold){EdgeValue(i, j) = 1;

}else{EdgeValue(i, j) = 0;

}.

6. The filter system of claim 2, said pixel sorter further comprising:

(a) a first comparator for sorting said visual edge pixels from said visual non-edge pixels;

(b) a selector comprising;

(i) said selector receiving output from said first comparator; and

(ii) said selector designating a kernel of pixels near each pixel being processed; and

(c) a second comparator comprising;

(i) said second comparator receiving output from said selector; and

(ii) said second comparator assigning types of filtration to each pixel being processed based at least in part on a sum of first edge values and second edge values of said pixels in said kernel of pixels.

7. The filter system of claim 6, wherein said kernel of pixels is a grid of pixels in which said pixel being processed is a center pixel in said grid of pixels.

8. The filter system of claim 6, wherein said second comparator is for applying de-ringing filtration by said de-ringing module to said pixel being processed if said pixel being processed is a visual non-edge pixel and said sum of first edge values and second edge values of said pixels in said kernel of pixels is less than a predetermined threshold value defining a true visual edge.

9. The filter system of claim 6, wherein said first comparator is for applying edge sharpening filtration by said edge sharpener to said pixel being processed if said pixel being processed is a visual edge pixel.

10. The filter system of claim 6, said second comparator applying no de-ringing filter and no edge sharpener to said pixel being processed if said pixel being processed is a visual non-edge pixel based on said second edge value and said sum of first edge values and second edge values of said pixels in said kernel of pixels is not less than a predetermined threshold value defining a true visual edge.

11. The filter system of claim 6, said filter system further comprising:

(a) a grayscaler;

(b) said grayscaler for summing grayscale values of all visual edge pixels in said kernel of pixels; and

(c) said grayscaler summing grayscale values of all visual non-edge pixels in said kernel of pixels.

12. The filter system of claim 6, said adaptive filter for executing a grayscale subroutine for counting said number of visual edge pixels in said kernel of pixels, for summing grayscale values of all visual edge pixels in said kernel of pixels, and for summing grayscale values of all visual non-edge pixels in said kernel of pixels, said grayscale subroutine further comprising:

(a) an i variable for storing a horizontal coordinate of each pixel in said digital image;

(b) a j variable for storing a vertical coordinate of each pixel in said digital image;

(c) an ix integer variable for counting a horizontal distance away from said i variable;

(d) an jy integer variable for counting a vertical distance away from said j variable;

(e) an X integer variable for defining a length of a horizontal edge of said kernel of pixels;

(f) a Y integer variable for defining a length of a vertical edge of said kernel of pixels;

(g) a Kernel variable for storing a count of pixels in said kernel of pixels obtained by multiplying said X integer by said Y integer;

(h) a NonEdgeGrayscaleSum variable for storing said sum of grayscale values of all visual non-edge pixels in said kernel of pixels;

(i) an EdgeGrayscaleSum variable for storing said sum of grayscale values of all visual edge pixels in said kernel of pixels;

(j) an EdgeValue(i, j) variable for storing said first edge value for each visual edge pixel and for storing said second edge value for each visual non-edge pixel;

(k) a Grayscale(i, j) variable for storing a grayscale value of each pixel located at coordinates i and j in said digital image; and

(l) a SumEdgePixels variable for counting a number of said visual edge pixels in said kernel of pixels;

(m) wherein said grayscale subroutine is defined as;

Kernel = (2 * X +

     1) * (2 * Y +

     1);

NonEdgeGrayscaleSum = EdgeGrayscaleSum = 0;

for(ix =−

X;

ix <

= X;

ix++)for(jy =−

Y;

jy <

= Y;

jy++){NonEdgeGrayscaleSum += (1 −

EdgeValue(i + ix, j + jy)) *GrayScale(i + ix, j + jy);

EdgeGrayscaleSum += EdgeValue(i + ix, j + jy) * GrayScale(i +ix, j + jy);

SumEdgePixels += EdgeValue(i + ix, j + jy);

}.

13. The filter system of claim 1, said de-ringing module further comprising a weighting module;

said weighting module altering a grayscale value of each visual non-edge pixel for final display in direct proportion to an average grayscale value of all visual non-edge pixels in a kernel of pixels.

14. The filter system of claim 13, said average grayscale value further comprising:

(a) a sum of grayscale values from said all visual non-edge pixels in said kernel of pixels;

(b) said sum divided by a number of said all visual non-edge pixels in a kernel of pixels.

15. The filter system of claim 14, said weighting module for executing a weighting subroutine for altering a grayscale value of each visual non-edge pixel for final display in proportion to an average grayscale value of all visual non-edge pixels in said kernel of pixels, said weighting subroutine further comprising:

(a) a FinalGrayScale(i, j) variable;

(b) said FinalGrayScale(i, j) variable storing a grayscale value for final display of each pixel being processed;

(c) a Kernel variable for storing a count of pixels in said kernel of pixels;

(d) a SumEdgePixels variable for counting a number of said visual edge pixels in said kernel of pixels;

(e) a NonEdgeGrayscaleSum variable for storing said sum of grayscale values of all visual non-edge pixels in said kernel of pixels; and

(f) wherein said weighting subroutine is defined as;

FinalGrayScale(i, j)=(1/(Kernel−

SumEdgePixels))* NonEdgeGrayscaleSum.

16. The filter system of claim 1, said edge sharpener further comprising an unsharp masking module, said unsharp masking module adding a high pass filtered image of said digital image to said digital image.

17. The filter system of claim 16, said unsharp masking module sharpening visual edges in said digital image by an edge sharpening factor λ

.

18. The filter system of claim 16, said high pass filtered image being obtained by subtracting a low pass filtered image of said digital image from a scaled version of said digital image.

19. The filter system of claim 18, said low pass filtered image for each pixel being processed further comprising:

(a) an EdgeGrayscaleSum variable for storing a sum of grayscale values of all visual edge pixels in a kernel of pixels surrounding said pixel being processed;

(b) a SumEdgePixels variable for counting a number of pixels representing a visual edge in said kernel of pixels surrounding said pixel being processed; and

(c) said low pass filtered image for said pixel being processed being the ratio EdgeGrayscaleSum/SumEdgePixels.

20. The filter system of claim 16, said unsharp masking module for executing a sharpening subroutine, said sharpening subroutine further comprising:

(a) a FinalGrayScale(i, j) variable for storing a grayscale value for final display of each pixel being processed;

(b) a Grayscale(i, j) variable for storing a grayscale value of an individual pixel at coordinates i and j in said digital image;

(c) a SumEdgePixels variable for storing a count of visual edge pixels in said kernel of pixels;

(d) an EdgeGrayscaleSum variable for storing a sum of grayscale values of all visual edge pixels in said kernel of pixels; and

(e) a selectable λ

variable for storing an edge sharpening factor; and

(f) wherein said sharpening subroutine is defined as;

FinalGrayScale(i, j)=(1+λ

)*Grayscale(i, j)−

(1/SumEdgePixels)*λ

*

EdgeGrayscaleSum.

21. The filter system of claim 1, said edge sharpener further comprising a limiter for decreasing said edge sharpening to avoid saturation of visual edges.

22. The filter system of claim 1, said filter system sharing data and calculations between said edge mapper, said pixel sorter, and said adaptive filter to reduce calculations.

23. A method of filtering signals of a digital image composed of a plurality of pixels, said method comprising the steps of:

(a) mapping visual edges in said digital image to produce an edge map;

(b) sorting pixels of said edge map into edge pixels representing visual edges and non-edge pixels representing visual non-edges;

(c) edge sharpening said edge pixels;

(d) de-ringing said non-edge pixels; and

(e) displaying said edge pixels after edge sharpening and said non-edge pixels after de-ringing.

24. The method of claim 23, said step of mapping visual edges further comprising the step of mapping visual edges pixel by pixel using at least one edge gradient operator.

25. The method of claim 23, said step of sorting pixels of said edge map further comprising the step of sorting each non-edge pixel according to a number of edge pixels in a kernel of pixels surrounding said non-edge pixel.

26. The method of claim 23, said step of sorting pixels of said edge map further comprising the step of sorting a non-edge pixel for no filtering if a number of edge pixels in said kernel of pixels surrounding said non-edge pixel is greater than a selected threshold.

27. The method of claim 23, said step of de-ringing said non-edge pixels further comprising the steps of:

(a) averaging grayscale values of pixels in said kernel of pixels surrounding each non-edge pixel; and

(b) altering a grayscale value of each non-edge pixel in proportion to averaged grayscale values of said pixels in a kernel of pixels surrounding each non-edge pixel.

28. The method of claim 23, said step of de-ringing non-edge pixels further comprising the step of de-ringing using at least some data previously calculated in said steps of mapping and sorting.

29. The method of claim 23, said step of edge sharpening further comprising the step of unsharp masking each edge pixel by adding a high pass filtered image of said edge pixel to an original image of said edge pixel.

30. The method of claim 23, said step of edge sharpening further comprising the step of edge sharpening using at least some data previously calculated in said steps of mapping, sorting, and de-ringing.

31. A method for post-processing a digital image having a plurality of pixels, said method comprising the steps of:

(a) edge mapping edge pixels representing visual edges and non-edge pixels representing visual non-edges in said digital image to produce a binary map of edge mapped individual pixels;

(b) sorting said edge mapped individual pixels for different types of filtration;

(c) filtering sorted individual pixels adaptively, said step of filtering comprising the steps of;

(i) edge sharpening said edge pixels; and

(ii) de-ringing said non-edge pixels;

(d) wherein said steps of edge sharpening and de-ringing may be performed substantially simultaneously.

32. The method of claim 31, said step of sorting further comprising the steps of:

(a) designating a group of pixels surrounding and including each non-edge pixel being sorted;

(b) reading a grayscale value of each pixel in said group of pixels;

(c) omitting said de-ringing and said edge sharpening for said non-edge pixel if said group of pixels includes at least a selected minimum number of edge pixels; and

(d) de-ringing said non-edge pixel if said group of pixels does not include at least a selected minimum number of edge pixels.

33. The method of claim 32, said step of de-ringing further comprising the step of scaling for display said grayscale value of each non-edge pixel sorted for de-ringing in proportion to averaged grayscale values of non-edge pixels in said group of pixels.

34. The method of claim 31, said step of edge sharpening further comprising the step of unsharp masking an edge pixel by adding a high pass filtered image of said edge pixel to an original image of said edge pixel.

35. A filter system for post-processing a digital image, said digital image having a plurality of visual-edge pixels and a plurality of visual non-edge pixels, said filter comprising:

(a) edge mapping means for producing a binary map of said visual edge pixels and said visual non-edge pixels;

(b) pixel sorting means for assigning visual non-edge pixels to a de-ringing means and edge pixels to an edge sharpening means;

(c) said de-ringing means for post-processing said visual non-edge pixels; and

(d) said edge sharpening means for post-processing said edge pixels.

36. The filter of claim 35, said edge mapping means further comprising edge detecting means.

37. A filter system for post-processing a digital image, said digital image having a plurality of pixels, said filter system comprising:

(a) an edge mapper for producing a binary map of said plurality of pixels;

(b) a pixel sorter for sorting pixels of said digital image into categories for appropriate post-processing;

(c) a first post-processing module for post-processing a first category of said plurality of pixels; and

(d) a second post-processing module for post-processing a second category of said plurality of pixels.

38. The filter system of claim 37, wherein said first post-processing module is a de-ringing module and said first category is non-edge pixels.

39. The filter system of claim 37, wherein said second post-processing module is an edge sharpening module and said second category is edge pixels.

40. The filter system of claim 37 further comprising a third post-processing module for post-processing a third category of said plurality of pixels, wherein said third post-processing module is a non-filter.