Preprocessing device and method for recognizing image characters

• US 7,471,830 B2
• Filed: 03/12/2004
• Issued: 12/30/2008
• Est. Priority Date: 03/15/2003
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lim, Chae-Whan, Kim, Chong-Heun, Kim, Nam-Chul, Shin, Ki-Taeg, Jang, Ick-Hoon, Park, Jun-Hyo, Kim, Hwan, Seo, Jeong-Wook
• Primary Examiner(s): KIM, CHONG R
• Application Number:
  US10/798,860
• Publication Number:
  US 20040240737A1
• Time in Patent Office: 1,754 Days
• Field of Search: None
• US Class Current: 382/181
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/162 Quantising the image signal
• CPC Class Codes: G06V 30/164 Noise filtering

Abstract

Disclosed is a device and method for recognizing characters in an image. An input part receives the image. A blurring decision part classifies the received image into character blocks and background blocks, calculates an average energy ratio of the character blocks, and compares the average energy ratio with a predetermined threshold to determine whether the received image is blurred. If the received image is not blurred, an image binarization part classifies the received image into character blocks and background blocks, compares pixels in the character blocks with a pixel threshold, binarizes the pixels in the character blocks into a brightness value for a character pixel and a brightness value for a background pixel based on the comparison, and binarizes pixels in the background blocks into the brightness value for a background pixel. A character recognition part recognizes characters in the binarized image.

Claims

1. A device for recognizing characters in an image, comprising:

an input part for receiving the image;

a blurring decision part for classifying the received image into character blocks and background blocks, calculating an average energy ratio of the character blocks, and comparing the average energy ratio with a predetermined threshold to determine whether the received image is blurred;

an object skew correction part for classifying, if the received image is not blurred, stripes having or exceeding a predetermined length in the received image, calculating direction angles of the classified stripes to measure a skew of an object, determining a skew angle corresponding to the measured skew, and rotating the image by the determined skew angle to correct the skew of the object in the image;

an image binarization part for classifying the skew-corrected image into character blocks and background blocks, comparing pixels in the character blocks with a pixel threshold, binarizing the pixels in the character blocks into a brightness value for a character pixel and a brightness value for a background pixel based on the comparison, and binarizing pixels in the background blocks into the brightness value for a background pixel; and

a character recognition part for recognizing characters in the binarized image.

2. The device of claim 1, wherein the blurring decision part comprises:

a block classification part for dividing the received image into the blocks, and classifying the divided blocks into character blocks and background blocks;

an average character block energy calculation part for calculating an average energy ratio of the character blocks classified by the block classification part; and

a blurring detection part for comparing the average energy ratio with a predetermined threshold, and determining whether the image is blurred based on the comparison.

3. The device of claim 2, wherein the block classification part comprises:

a block division part for dividing the received image into blocks having a predetermined size;

a Discrete Cosine Transform (DCT) conversion part for DCT-converting the blocks output from the block division part;

an energy calculation part for calculating a sum of absolute values of dominant DCT coefficients in each of the DCT-converted blocks, and outputting the calculated sum as an energy value of a corresponding block;

a block threshold calculation part for summing energy values of the blocks output from the energy calculation part, calculating an average by dividing the summed energy value by the total number of the blocks, and outputting the calculated average as a block threshold; and

a block decision part for sequentially receiving the energy values of the blocks output from the energy calculation part, comparing the received energy values of the blocks with the threshold, and classifying the blocks as character blocks or background blocks based on the comparison.

4. The device of claim 3, wherein the average character block energy calculation part comprises:

an energy ratio calculation part for calculating an energy ratio of DCT coefficients in each of the character blocks; and

an average energy ratio calculation part for calculating an average energy ratio of the character blocks by averaging the energy ratios of the character blocks.

5. The device of claim 1, wherein the image binarization part comprises:

a block classification part for dividing the image into blocks, and classifying the divided blocks into character blocks and background blocks;

an edge enhancement part for enhancing edges of the character blocks classified by the block classification part using a relationship between neighboring pixels in the character blocks, and generating a pixel threshold for distinguishing between character pixels and background pixels in the character blocks; and

a binarization part for comparing pixels in the character blocks output from the edge enhancement part with the threshold, binarizing the pixels into a first brightness value for a character pixel and a second brightness value for a background pixel based on the comparison, and binarizing pixels in the background blocks output from the block classification part into the second brightness value.

6. The device of claim 5, wherein the edge enhancement part comprises:

a first threshold calculation part for calculating a first threshold for classifying pixels in the character block into character pixels and background pixels;

a mean computation part for classifying pixels in the character block into character pixels and background pixels on the basis of the first threshold, and calculating mean brightness values for the character pixels and the background pixels in the character block;

a normalization part for normalizing the pixels in the character block using a mean brightness value for a character pixel and a mean brightness value for a background pixel output from the mean computation part so that the character pixels have a value close to ‘

1’ and

the background pixels have a value close to ‘

0’

;

a quadratic operation part for performing a quadratic operation on the normalized character block to enhance edges of the character block and reduce noises of the character block;

a denormalization part for denormalizing an image of the quadratic-processed character block, and outputting the denormalized character block to the binarization part; and

a second threshold calculation part for calculating a second threshold for classifying pixels in the denormalized character block into character pixels and background pixels, and outputting the second threshold as a threshold for the binarization part.

7. The device of claim 5, wherein the block classification part comprises:

a block division part for dividing the received image into blocks having a predetermined size;

a DCT conversion part for DCT-converting the blocks output from the block division part;

an energy calculation part for calculating a sum of absolute values of dominant DCT coefficients in each of the DCT-converted blocks, and outputting the calculated sum as an energy value of a corresponding block;

a block threshold calculation part for summing energy values of the blocks output from the energy calculation part, calculating an average by dividing the summed energy value by the total number of the blocks, and outputting the calculated average as a block threshold; and

a block decision part for sequentially receiving the energy values of the blocks output from the energy calculation part, comparing the received energy values of the blocks with the threshold, and determining the blocks as character blocks or background blocks based on the comparison.

8. The device of claim 5, wherein the edge enhancement part comprises:

a first threshold calculation part for calculating a first threshold for classifying pixels in the character block into character pixels and background pixels;

a mean computation part for classifying pixels in the character block into character pixels and background pixels on the basis of the first threshold, and calculating mean brightness values for character pixels and background pixels in the character block;

a normalization part for normalizing the pixels in the character block using a mean brightness value for a character pixel and a mean brightness value for the background pixel output from the mean computation part so that the character pixels have a value close to ‘

1’ and

the background pixels have a value close to ‘

0’

;

a quadratic operation part for performing a quadratic operation on the normalized character block to enhance edges of the character block and reduce noises of the character block; and

a second threshold calculation part for normalizing the first threshold to calculate a second threshold for classifying the pixels into character pixels and background pixels, and outputting the second threshold as a threshold for the binarization part.

9. The device of claim 1, wherein the image binarization part comprises:

a block classification part for dividing the image into the blocks, and classifying the divided blocks into character blocks and background blocks;

a block growing part for growing the character blocks to restore a block including character pixels, incorrectly classified as a background block, to a character block;

an edge enhancement part for enhancing edges of the character block output from the block growing part using a relationship between neighboring pixels in the character block, and generating a threshold for distinguishing between character pixels and background pixels in the character block; and

a binarization part for comparing pixels in the character blocks output from the edge enhancement part with the threshold, binarizing the pixels into a first brightness value for a character pixel and a second brightness value for a background pixel according to the comparison result, and binarizing pixels in the background pixels output from the block growing part into the second brightness value.

10. The device of claim 9, wherein the block growing part comprises:

a dilation part for dilating a character block and converting a block containing character pixels, incorrectly classified as a background block, to a character block; and

a closing part for eroding the dilated character block to separate connected blocks.

11. The device of claim 1, wherein the image binarization part comprises:

a block classification part for dividing the image into the blocks, and classifying the divided blocks into character blocks and background blocks;

a block grouping part for grouping each of the character blocks classified by block classification part with neighboring blocks thereof to generate a grouped block;

an edge enhancement part for enhancing edges of the grouped block using a relationship between neighboring pixels in the grouped block, and generating a threshold for distinguishing between character pixels and background pixels in the character block;

a block splitting part for separating the character block from the grouped block output from the edge enhancement part; and

a binarization part for comparing pixels in the separated character block with the threshold, binarizing the pixels into a first brightness value for a character pixel and a second brightness value for a background pixel based on the comparison, and binarizing pixels in the background pixels output from the block classification part into the second brightness value.

12. The device of claim 1, wherein the image binarization part comprises:

a block classification part for dividing the image into the blocks, and classifying the divided blocks into character blocks and background blocks;

a block growing part for growing the character blocks to restore a block including character pixels, incorrectly classified as a background block, to a character block;

a block grouping part for grouping a character block output from the block growing part with neighboring blocks thereof to generate a grouped block;

an edge enhancement part for enhancing edges of the grouped block using a relationship between neighboring pixels in the grouped block, and generating a threshold for distinguishing between character pixels and background pixels in the character block;

a block splitting part for separating the character block from the grouped block output from the edge enhancement part; and

a binarization part for comparing pixels in the separated character block with the threshold, binarizing the pixels into a first brightness value for a character pixel and a second brightness value for a background pixel based on the comparison, and binarizing pixels in the background pixels output from the block growing part into the second brightness value.

13. The device of claim 1, further comprising a noise reduction part for reducing noises of the received image and outputting the noise-reduced image to the image binarization part.

14. The device of claim 13, wherein the noise reduction part comprises a directional Lee filter.

15. The device of claim 1, wherein the object skew correction part comprises:

a binarization part for binarizing pixels in the image into a brightness value for a character pixel and a brightness value for a background pixel;

a candidate stripe generation part for generating candidate stripes by performing dilation on character regions in the binarized image;

a stripe classification part for classifying, as stripes, candidate stripes having an eccentricity and a blob size having or exceeding a predetermined value among the candidate stripes;

a skew angle decision part for calculating direction angles of the classified stripes, and determining a direction angle having the largest count value as a skew angle; and

a skew correction part for correcting a skew of the image by rotating the image by the skew angle.

16. The device of claim 15, further comprising an image correction part for filling blank spaces at corners of the image in which a skew of an object is corrected by the skew correction part, with pixels close to the blank space in a horizontal direction.

17. The device of claim 15, wherein the binarization part comprises:

a block classification part for dividing the image into blocks having a predetermined size, calculating pixel energies of the divided blocks, and classifying the blocks into character blocks and background blocks according to the calculated pixel energies;

a pixel threshold calculation part for calculating a pixel threshold using Otsu'"'"'s method that calculates a brightness value having the maximum between-class variance between character pixels and background pixels in the character blocks output from the block classification part; and

a binarization part for comparing pixels in the character blocks output from the block classification part with the pixel threshold, binarizing the pixels into a brightness value for a character pixel and a brightness value for a background pixel based on the comparison, and binarizing pixels in the background blocks into the brightness value for a background pixel.

18. The device of claim 17, further comprising a block grouping part, interconnected between the block classification part and the pixel threshold calculation part, for grouping the character block classified by block classification part with neighboring blocks thereof to generate a grouped block, and outputting the grouped block to the pixel threshold calculation part.

19. The device of claim 15, wherein the candidate stripe generation part comprises:

a dilation part for dilating a region of the binarized character block and generating candidate stripes in which neighboring characters are connected; and

an erosion part for eroding the candidate stripes so that candidate stripes located up and down the dilated candidate stripes are separated.

20. The device of claim 15, wherein the stripe classification part calculates a length of a candidate stripe through calculation of a blob size and eccentricity based on a moment of the candidate stripe, and classifies the corresponding candidate stripe as a valid stripe when the eccentricity and the blob size are larger than or equal to their predetermined thresholds.

21. The device of claim 15, wherein the skew angle decision part calculates direction angles of the classified stripes, and determines a direction angle having the largest count value as a skew angle.

22. The device of claim 1, wherein the object skew correction part comprises:

a binarization part for binarizing pixels in the image into a brightness value for a character pixel and a brightness value for a background pixel;

a horizontal pixel subsampling part for performing horizontal subsampling on the binarized image at a predetermined subsampling rate;

a candidate stripe generation part for generating candidate stripes by dilating character blocks in the binarized image;

a vertical pixel subsampling part for performing vertical subsampling on an image having the candidate stripes at a predetermined subsampling rate;

a stripe classification part for classifying, as stripes, candidate stripes having an eccentricity and a blob size having or exceeding a predetermined value among the vertical-subsampled candidate stripes;

a skew angle decision part for calculating direction angles of the classified stripes, accumulating count values of the direction angles, and determining a direction angle having the largest count value as a skew angle; and

a skew correction part for correcting a skew of an object in the image by rotating the image by the skew angle.

23. The device of claim 22, further comprising an image correction part for filling blank spaces at corners of the image in which a skew of an object is corrected by the skew correction part, with pixels close to the blank space in a horizontal direction.

24. The device of claim 1, further comprising a Region Of Contents (ROC) extension part for classifying the image in which a skew of an object is corrected into character blocks and background blocks, extracting a character region by searching for a position of the character blocks in the image, and extending an image of the extracted character region to a size of the received image;

and

when the device comprises the Region of Contents (ROC) extension part, the image binarization part classifies the ROC-extended image into character blocks and background blocks, compares pixels in the character blocks with a pixel threshold, binarizes the pixels into a brightness value for a character pixel and a brightness value for a background pixel based on the comparison result, and binarized pixels in the background blocks into the brightness value for a background pixel.

25. The device of claim 24, wherein the ROC extension part comprises:

a block classification part for classifying the image into character blocks and background blocks, and converting the character blocks into pixels having a first brightness value and the background blocks into pixels having a second brightness value;

a position search part for searching for left, right, top and bottom positions of a character region by horizontally and vertically scanning the block-classified image, and determining a position of the character region according to the search result;

an ROC extraction part for extracting an image in the determined position of the character region from the received image; and

an image extension part for extending the extracted image of the character region to a size of the received image.

26. The device of claim 25, wherein the block classification part comprises:

a block division part for dividing the received image into blocks having a predetermined size;

a Discrete Cosine Transform (DCT) conversion part for DCT-converting the blocks output from the block division part;

an energy calculation part for calculating a sum of absolute values of dominant DCT coefficients in each of the DCT-converted blocks, and outputting the calculated sum as an energy value of a corresponding block;

a block threshold calculation part for summing energy values of the blocks output from the energy calculation part, calculating an average by dividing the summed energy value by the total number of the blocks, and outputting the calculated average as a block threshold; and

a block decision part for sequentially receiving the energy values of the blocks output from the energy calculation part, comparing the received energy values of the blocks with the threshold, and classifying the blocks as character blocks or background blocks based on the comparison.

27. The device of claim 25, wherein the position search part searches a position of a character region by horizontally and vertically scanning the block-classified image, and determines a position of the character region based on the search result so that the character region has an aspect ratio of the received image.

28. The device of claim 25, wherein the image extension part extends an image of the extracted character region by bilinear interpolation.

29. The device of claim 28, wherein the median filter determines isolated character blocks as incorrectly classified character blocks.

30. The device of claim 24, wherein the ROC extension part comprises:

a block classification part for classifying the image into character blocks and background blocks, and converting the character blocks into pixels having a first brightness value and the background blocks into pixels having a second brightness value;

a median filter for median-filtering an image output from the block classification part to remove blocks incorrectly classified as character blocks;

a position search part for searching for left, right, top and bottom positions of a character region by horizontally and vertically scanning the median-filtered image, and determining a position of the character region based on the search result;

an ROC extraction part for extracting an image in the determined position of the character region from the received image; and

an image extension part for extending an image of the extracted character region to a size of the received image.

31. The device of claim 24, wherein the ROC extension part comprises:

a mean filter for mean-filtering the received image to blur the received image;

a block classification part for classifying the mean-filtered image into character blocks and background blocks, and converting the character blocks into pixels having a first brightness value and the background blocks into pixels having a second brightness value;

a median filter for median-filtering an image output from the block classification part to remove blocks incorrectly classified as character blocks;

a position search part for searching for left, right, top and bottom positions of a character region by horizontally and vertically scanning the median-filtered image, and determining a position of the character region according to the search result;

an ROC extraction part for extracting an image in the determined position of the character region from the received image; and

an image extension part for extending an image of the extracted character region to a size of the received image.

32. The device of claim 24, wherein the ROC extension part comprises:

a mean filter for mean-filtering the received image to blur the received image;

a block classification part for classifying the mean-filtered image into character blocks and background blocks, and converting the character blocks into pixels having a first brightness value and the background blocks into pixels having a second brightness value;

a subsampling part for subsampling pixels in the image output from the block classification part to reduce the number of the pixels;

a median filter for median-filtering the subsampled image to remove blocks incorrectly classified as character blocks;

an interpolation part for interpolating pixels in the median-filtered image to extend the median-filtered image to a size of the received image;

a position search part for searching for left, right, top and bottom positions of a character region by horizontally and vertically scanning the block-classified image, and determining a position of the character region according to the search result;

an ROC extraction part for extracting an image in the determined position of the character region from the received image; and

an image extension part for extending an image of the extracted character region to a size of the received image.

33. A method for recognizing characters in an image, comprising the steps of:

receiving the image;

classifying the received image into character blocks and background blocks, calculating an average energy ratio of the character blocks, and comparing the average energy ratio with a predetermined threshold to determine whether the received image is blurred;

an object skew correction part for classifying, if the received image is not blurred, stripes having or exceeding a predetermined length in the received image, calculating direction angles of the classified stripes to measure a skew of an object, determining a skew angle corresponding to the measured skew, and rotating the image by the determined skew angle to correct the skew of the object in the image;

classifying the skew-corrected image into character blocks and background blocks, comparing pixels in the character blocks with a pixel threshold, binarizing the pixels in the character blocks into a brightness value for a character pixel and a brightness value for a background pixel based on the comparison, and binarizing pixels in the background blocks into the brightness value for a background pixel; and

recognizing characters in the binarized image.

34. The method of claim 33, further comprising a nose reduction part for reducing noises of the received image and outputting the noise-reduced image to the image binarization part.

35. The method of claim 33, further comprising the steps of:

classifying the skew-corrected image into character blocks and background blocks, extracting a character region by searching for a position of the character blocks in the image, and extending an image of the extracted character region to a size of the received image; and

when extending an image of the extracted character region to a size of the received image, classifying a Region Of Contents (ROC)-extended image into character blocks and background blocks, comparing pixels in the character blocks with a pixel threshold, binarizing the pixels into a brightness value for a character pixel and a brightness value for a background pixel based on the comparison, and binarizing pixels in the background blocks into the brightness value for a background pixel.

36. The method of claim 35, further comprising the step of reducing noises of the ROC-extended image and then proceeding to the image binarization step.