Systems and methods for content-based document image enhancement

• US 20040037473A1
• Filed: 08/20/2002
• Published: 02/26/2004
• Est. Priority Date: 08/20/2002
• Status: Active Grant

• Current Assignee: Lexmark International Incorporated (Ninestar Corporation)
• Original Assignee: Lexmark International Incorporated (Ninestar Corporation)
• Inventors: Cooper, Brian E., Lhamon, Michael E., Ahmed, Mohamed N.
• Granted Patent: US 7,079,686 B2
• Time in Patent Office: Days
• US Class Current: 382/254
• CPC Class Codes: G06T 2207/10008 from scanner, fax or copier
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: H04N 1/40062 Discrimination between different image types, e.g. two-tone, continuous tone

Abstract

A system can include a system processor that is programmed or adapted to classify pixels in a document image, and in some instances, to enhance the document image based upon such classification. Image data including pixels is received. For each pixel in some subset of the image data, a feature vector is generated. Each pixel in the subset is classified as text or image based upon its associated feature vector. The classifications can be outputted for storage and/or further processing. Further processing can include pixel level enhancement processing. Such enhancement processing can include application of a sharpening filter to pixels classified as text and of a smoothing filter to pixels classified as image. In some instances, background suppression can be performed upon a subset of the image data prior to classification.

Claims

1. An image processing system, the system comprising a system processor that is programmed or adapted to:

a) receive image data comprising a plurality pixels each having a pixel value;

b) selectively mark a current pixel as a potential text element;

c) determine a distance from the current pixel to a previous pixel marked as a potential text element; and

d) selectively mark all pixels in between the current and previous pixels as text pixels or image pixels based upon the determined distance.

2. The system of claim 1, wherein the system processor selectively marks the current pixel by performing steps comprising of:

a) generating a feature vector for the current pixel, wherein the feature vector comprises one or more elements representing attributes of the current pixel;

b) calculating a magnitude for the generated feature vector;

c) determining whether the current pixel is a potential text element by comparing the calculated magnitude with a selected magnitude threshold; and

d) marking the current pixel as a potential text element if the current pixel is determined to be a potential text element.

3. The system of claim 2, wherein the feature vector generated by the system processor comprises a vertical edge parameter.

4. The system of claim 3, wherein the system processor'"'"'s generation of the feature vector comprises assigning a value equal to |I(x,y+1)−

I(x,y−

1)| as the vertical edge parameter, wherein I(x, y) corresponds to the pixel value of the pixel associated with coordinates x and y among the plurality of pixels in the received image data.

5. The system of claim 2, wherein the feature vector generated by the system processor comprises a horizontal edge parameter.

6. The system of claim 5, wherein the system processor'"'"'s generation of the feature vector comprises assigning a value equal to |I(x+1,y)−

I(x−

1,y)| as the horizontal edge parameter, wherein I(x, y) corresponds to the pixel value of the pixel associated with coordinates x and y among the plurality of pixels in the received image data.

7. The system of claim 2, wherein the system processor calculates the magnitude of each feature vector by calculating a norm η

of each generated feature vector χ

, wherein

8. The system of claim 1, wherein the system processor is further programmed or adapted to apply a filter to one or more marked pixels in the image data based upon each pixel'"'"'s marking as text or image.

9. The system of claim 7, wherein the system processor'"'"'s application of a filter comprises applying a sharpening filter to each pixel marked as text.

10. The system of claim 9, wherein the system processor is further programmed or adapted to dynamically determine coefficients of the sharpening filter based upon the received image data.

11. The system of claim 10, wherein the system processor'"'"'s dynamic determination of coefficients of the sharpening filter for a particular pixel based upon the spatial distance between the particular pixel and the closest other pixel marked as text.

12. The system of claim 9, wherein the sharpening filter applied by the system processor has fixed coefficients.

13. The system of claim 12, wherein the sharpening filter applied by the system processor is

14. The system of claim 7, wherein the system processor'"'"'s application of a filter comprises applying a smoothing filter to each pixel marked as image.

15. The system of claim 14, wherein the system processor is further programmed or adapted to dynamically determine coefficients of the smoothing filter based upon the received image data.

16. The system of claim 14, wherein the smoothing filter applied by the system processor has fixed coefficients.

17. The system of claim 16, wherein the smoothing filter applied by the system processor is

18. The system of claim 1, further comprising a data store, in communication with the system processor, that is capable of storing image data and wherein the system processor is further programmed or adapted to store the received image data in the data store.

19. The system of claim 18, wherein the system processor is further programmed or adapted to apply a filter to one or more marked pixels in the image data based upon each pixel'"'"'s marking as text or image and to replace pixel data of the received image data in the data store with pixel data it generates from applying filters to one or more pixels of the received image data.

20. The system of claim 1, wherein the system processor is further programmed or adapted to apply a filter to one or more marked pixels in the image data based upon each pixel'"'"'s marking as text or image and further comprising a data store, in communication with the system processor, that is capable of storing image data and wherein the system processor is further programmed or adapted to store pixel data of the received image data or pixel data generated from applying filters to one or more pixels of the received image data.

21. The system of claim 20, wherein the system processor is further programmed or adapted to store pixel data comprising pixel data of the received image data and pixel data it generates from applying filters to one or more pixels of the received image data.

22. The system of claim 1, further comprising an imaging element, in communication with the system processor, that generates the image data by scanning a source document.

23. The system of claim 1, wherein the system processor comprises a field programmable gate-array or an application-specific integrated circuit.

24. The system of claim 23, wherein the system processor comprises a field programmable gate-array according to FIG. 1.

25. The system of claim 1, wherein the system processor is further programmed or adapted to suppress background in the received image data prior to pixel classification.

26. The system of claim 25, wherein the system processor is further programmed or adapted to suppress background by:

i) determining whether each pixel is a background pixel by comparing an intensity value associated with each pixel with an intensity threshold;

ii) for each pixel not determined to be a background pixel, replacing its intensity value with a new intensity value by mapping its current intensity value into a range of intensity values corresponding to a full intensity spectrum; and

iii) for each pixel determined to be a background pixel, replacing its intensity value with a new intensity value corresponding to a terminal intensity value.

27. The system of claim 26, wherein the system processor is further programmed or adapted to determine the intensity threshold during background suppression by:

1) generating a histogram distribution of frequency of occurrence of intensity values from one or more scan lines of the image data;

2) determining a background peak intensity value for the image data based upon the generated histogram distribution; and

3) deriving the intensity threshold from the determined background peak intensity value.

28. The system of claim 27, wherein the system processor is further programmed or adapted to derive the intensity threshold during intensity threshold determination by:

(a) determining a standard deviation associated with the generated histogram; and

(b) subtracting a function of the determined standard deviation from the determined background peak intensity value to derive the intensity threshold.

29. The system of claim 27, wherein the system processor is further programmed or adapted to generate the histogram of frequency of occurrence of intensity values during intensity threshold determination using a range of scan lines of the image comprising scan lines one through ten.

30. The system of claim 26, wherein the system processor is further programmed or adapted to replace a pixel intensity value with a new pixel intensity value by mapping its current intensity into a range of intensity values corresponding to a full intensity spectrum.

31. The system of claim 30, wherein the system processor is further programmed or adapted to perform the mapping by multiplying the current intensity by a value corresponding to a maximum intensity value and dividing by the intensity threshold.

32. The system of claim 1, wherein the system processor further processes one or more pixels of the image data based upon respective markings of the one or more pixels.

33. The system of claim 1, wherein the system processor processes one or more pixels of the image data by performing at least a Hough transformation on one or more pixels marked as text.

34. The system of claim 1, wherein the system processor processes one or more pixels of the image data by performing at least a descreening transformation on one or more pixels marked as image.

35. The system of claim 1, wherein the system processor selectively marks all pixels in between the current and previous pixels as text pixels or image pixels by performing steps comprising of comparing the determined distance to a selected distance threshold and if the calculated distance is below the distance threshold, marking all pixels in between the current and previous pixels as text pixels;

and if not, marking all pixels between the current and previous pixels as image pixels.

36. An image processing method, the method comprising the steps of:

a) receiving image data comprising a plurality pixels each having a gray level value;

b) generating a feature vector for each pixel, wherein each feature vector comprises a vertical edge parameter equal to |I(x, y+1)−

I(x, y−

1)| and a horizontal edge parameter equal to |I(x+1, y)−

I(x−

1, y)|, wherein I(x, y) corresponds to the gray level value of the pixel addressed by coordinates x and y;

c) classifying each pixel as text or image by;

i) calculating a norm η

of each generated feature vector χ

, wherein

$\mathrm{\eta <br><br>}=\sqrt{\underset{i=1}{\overset{d}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>x_i^2}$

 

and wherein {x_i|i=1,2, . . . , d} are elements of χ

.

ii) for each feature vector norm η

greater than a selected threshold,

1) marking the pixel associated with the respective feature vector as a potential text element; and

2) if the spatial distance between two consecutive potential text element pixels is below a specified distance threshold, marking all pixels in between the two consecutive potential text element pixels as text pixels; and

if not, marking all pixels between the two consecutive potential text element pixels as image pixels; and

iii) for each feature vector norm η

not greater than the selected threshold, marking the pixel associated with the respective feature vector as an image pixel; and

d) applying a filter to one or more pixels based upon each pixel'"'"'s classification as text or image.

37. The method of claim 36, wherein the step of applying a filter comprises the step of applying a filter, K, to each pixel classified as text, wherein

38. The method of claim 36, wherein the step of applying a filter comprises the step of applying a filter, K, to each pixel classified as image, wherein

39. The method of claim 36, and further comprising the step of scanning a source document to produce the image data.

40. The method of claim 36, and further comprising the step of suppressing background in the image data by:

i) generating a histogram distribution of frequency of occurrence of gray values from the first ten scan lines of the image data;

ii) determining a background peak gray value for the image data based upon the generated histogram distribution;

iii) for each pixel value below the determined peak gray value less two standard deviations, replacing its pixel value with a new pixel value by mapping its current pixel value into a range of pixel values corresponding to a full gray scale spectrum; and

iv) for each pixel value not below the determined peak gray value less two standard deviations, replacing its pixel value with a new pixel value corresponding to white.

41. Computer readable media having computer-executable instructions for performing the steps recited in claim 36.

42. An image processing system, the system comprising:

a) imaging means for scanning a source document to produce image data comprising a plurality of pixel values each having a gray level value;

b) receiving means for receiving the image data from the imaging means;

c) background suppression means for suppressing background in the image data;

d) feature vector calculating means for generating a feature vector for each pixel, wherein each feature vector comprises a vertical edge parameter equal to |(x,y+1)−

I(x,y−

1)| and a horizontal edge parameter equal to |I(x+1, y)−

I(x−

1, y)|, wherein I(x, y) corresponds to the gray level value of the pixel addressed by coordinates x and y;

e) classification means for classifying each pixel of the image data as text or image by;

i) calculating a norm η

of each generated feature vector χ

, wherein

$\mathrm{\eta <br><br>}=\sqrt{\underset{i=1}{\overset{d}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>x_i^2}$

 

and wherein {x_i|i=1,2, . . . , d} are elements of χ

.

ii) for each feature vector norm 17 greater than a selected threshold,

1) marking the pixel associated with the respective feature vector as a potential text element; and

2) if the spatial distance between two consecutive potential text element pixels is below a specified distance threshold, marking all pixels in between the two consecutive potential text element pixels as text pixels; and

if not, marking all pixels between the two consecutive potential text element pixels as image pixels; and

iii) for each feature vector norm η

not greater than the selected threshold, marking the pixel associated with the respective feature vector as an image pixel; and

f) filtering means for applying a filter, K, to classified pixels, wherein

$K=\left[\begin{array}{ccc}-1& -1& -1\\ -1& 9& -1\\ -1& -1& -1\end{array}\right]$

for each pixel classified as text and wherein

$K=1/9<br><br>\left[\begin{array}{ccc}1& 1& 1\\ 1& 1& 1\\ 1& 1& 1\end{array}\right]$

for each pixel classified as image.

43. An image processing system for suppressing background in image data, the image processing system comprising a system processor that:

a) receives image data comprising a plurality pixels each having a gray level value;

b) generating a histogram distribution of frequency of occurrence of gray values from the first ten scan lines of the image data;

c) determines a background peak gray value for the image data based upon the generated histogram distribution;

d) for each pixel value below the determined peak gray value less two standard deviations, replaces its pixel value with a new pixel value by mapping its current pixel value into a range of pixel values corresponding to a full gray scale spectrum by multiplying the current pixel value by a pixel value corresponding to white and dividing by the determined peak gray value less two standard deviations; and

e) for each pixel value not below the determined peak gray value less two standard deviations, replacing its pixel value with a new pixel value corresponding to white.