VIDEO TEXT PROCESSING APPARATUS

• US 20100220930A1
• Filed: 05/12/2010
• Published: 09/02/2010
• Est. Priority Date: 12/26/2002
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Sun, Jun, Naoi, Satoshi, Katsuyama, Yutaka
• Granted Patent: US 7,929,765 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

Video frames that contain text areas are selected from given video frames by removing redundant frames and non-text frames, the text areas in the selected frames are located by removing false strokes, and text lines in the text areas are extracted and binarized.

Claims

1. A text extraction apparatus that extracts at least one text line region from a given image, said apparatus comprising:

an edge image generation unit generating edge information of the given image;

a stroke image generation unit generating a binary image of candidate character strokes in the given image by using the edge information;

a stroke filtering unit removing a false stroke from the binary image by using the edge information;

a text line region formation unit combining a plurality of strokes into a text line region;

a text line verification unit removing a false character stroke from the text line region and reforming the text line region;

a text line binarization unit binarizing the text line region by using a height of the text line region; and

an output unit outputting a binary image of the text line region.

2. The text extraction apparatus according to claim 1, wherein the edge image generation unit includes:

an edge strength calculation unit calculating edge strength for every pixel in the given image by using a Sobel edge detector;

a first edge image generation unit generating a first edge image by comparing the edge strength of every pixel with a predefined edge threshold and setting a value of a corresponding pixel in the first edge image to one binary value if the edge strength is greater than the threshold and the other binary value if the edge strength is less than the threshold; and

a second edge image generation unit generating a second edge image by comparing the edge strength of every pixel in a window centered at a position of every pixel of the one binary value in the first edge image with mean edge strength of the pixels in the window and setting a value of a corresponding pixel in the second edge image to the one binary value if the edge strength of the pixel is greater than the mean edge strength and the other binary value if the edge strength of the pixel is less than the mean edge strength.

3. The text extraction apparatus according to claim 2, wherein the stroke image generation unit includes a local image binarization unit binarizing a gray scale image of the given image in a Niblack'"'"'s binarization method to obtain the binary image of the candidate character strokes by using a window centered at a position of every pixel of the one binary value in the second edge image.

4. The text extraction apparatus according to claim 2, wherein the stroke filtering unit includes:

a stroke edge coverage validation unit checking an overlap rate of a contour of a stroke in the binary image of the candidate character strokes by pixels of the one binary value in the second edge image, determining that the stroke is a valid stroke if the overlap rate is greater than a predefined threshold and an invalid stroke if the overlap rate is less than the predefined threshold, and removing the invalid stroke; and

a long straight line detection unit removing a large stroke by using a width and a height of the stroke.

5. The text extraction apparatus according to claim 2, wherein the text line binarization unit includes:

an automatic size calculation unit determining a size of a window for binarization; and

a block image binarization unit binarizing a gray scale image of the given image in a Niblack'"'"'s binarization method to obtain the binary image of the text line region by using the window centered at a position of every pixel of the one binary value in the second edge image.

6. The text extraction apparatus according to claim 1, wherein the text line region formation unit includes a stroke connection checking unit checking whether two adjacent strokes are connectable by using an overlap ratio of heights of the two strokes and a distance between the two strokes, and the text line region formation unit combines the plurality of strokes into a text line region by using a result of checking.

7. The text extraction apparatus according to claim 1, wherein the text line verification unit includes:

a vertical false stroke detection unit checking every stroke with a height higher than a mean height of strokes in the text line region, and marking the stroke as a false stroke if the stroke connects two horizontal text line regions into one big text line region;

a horizontal false stroke detection unit checking every stroke with a width larger than a threshold determined by a mean width of the strokes in the text line region, and marking the stroke as a false stroke if a number of strokes in a region that contains the stroke is less than a predefined threshold; and

a text line reformation unit reconnecting strokes except for a false stroke in the text line region if the false stroke is detected in the text line region.

8. A text extraction apparatus that extracts at least one text line region from a given image, said apparatus comprising:

an edge image generation unit generating an edge image of the given image;

a stroke image generation unit generating a binary image of candidate character strokes in the given image by using the edge image;

a stroke filtering unit checking an overlap rate of a contour of a stroke in the binary image of the candidate character strokes by pixels indicating an edge in the edge image, determining that the stroke is a valid stroke if the overlap rate is greater than a predefined threshold and an invalid stroke if the overlap rate is less than the predefined threshold, and removing the invalid stroke; and

an output unit outputting information of remaining strokes in the binary image of the candidate character strokes.

9. A computer-readable storage medium storing a program used to direct a computer, that extracts at least one text line region from a given image, to perform a process comprising:

generating edge information of the given image;

generating a binary image of candidate character strokes in the given image by using the edge information;

removing a false stroke from the binary image by using the edge information;

combining a plurality of strokes into a text line region;

removing a false character stroke from the text line region and reforming the text line region;

binarizing the text line region by using a height of the text line region; and

outputting a binary image of the text line region.

10. The storage medium according to claim 9, wherein the generating edge information includes:

calculating edge strength for every pixel in the given image by using a Sobel edge detector;

generating a first edge image by comparing the edge strength of every pixel with a predefined edge threshold and setting a value of a corresponding pixel in the first edge image to one binary value if the edge strength is greater than the threshold and the other binary value if the edge strength is less than the threshold; and

generating a second edge image by comparing the edge strength of every pixel in a window centered at a position of every pixel of the one binary value in the first edge image with mean edge strength of the pixels in the window and setting a value of a corresponding pixel in the second edge image to the one binary value if the edge strength of the pixel is greater than the mean edge strength and the other binary value if the edge strength of the pixel is less than the mean edge strength.

11. The storage medium according to claim 10, wherein the generating the binary image of the candidate character strokes includes binarizing a gray scale image of the given image in a Niblack'"'"'s binarization method to obtain the binary image of the candidate character strokes by using a window centered at a position of every pixel of the one binary value in the second edge image.

12. The storage medium according to claim 10, wherein the removing the false stroke from the binary image includes:

removing a large stroke by using a width and a height of the stroke.

checking an overlap rate of a contour of a stroke in the binary image of the candidate character strokes by pixels of the one binary value in the second edge image;

determining that the stroke is a valid stroke if the overlap rate is greater than a predefined threshold and an invalid stroke if the overlap rate is less than the predefined threshold; and

removing the invalid stroke.

13. The storage medium according to claim 10, wherein the binarizing the text line region includes:

determining a size of a window for binarization; and

binarizing a gray scale image of the given image in a Niblack'"'"'s binarization method to obtain the binary image of the text line region by using the window centered at a position of every pixel of the one binary value in the second edge image.

14. The storage medium according to claim 9, wherein the combining the plurality of strokes into the text line region includes checking whether two adjacent strokes are connectable by using an overlap ratio of heights of the two strokes and a distance between the two strokes, and the combining the plurality of strokes into the text line region combines the plurality of strokes into a text line region by using a result of checking.

15. The storage medium according to claim 9, wherein the removing the false character stroke from the text line region and reforming the text line region includes:

checking every stroke with a height higher than a mean height of strokes in the text line region;

marking the stroke as a false stroke if the stroke connects two horizontal text line regions into one big text line region;

checking every stroke with a width larger than a threshold determined by a mean width of the strokes in the text line region;

marking the stroke as a false stroke if a number of strokes in a region that contains the stroke is less than a predefined threshold; and

reconnecting strokes except for a false stroke in the text line region if the false stroke is detected in the text line region.

16. A computer-readable storage medium storing a program used to direct a computer, that extracts at least one text line region from a given image, to perform a process comprising:

generating an edge image of the given image;

generating a binary image of candidate character strokes in the given image by using the edge image;

checking an overlap rate of a contour of a stroke in the binary image of the candidate character strokes by pixels indicating an edge in the edge image;

determining that the stroke is a valid stroke if the overlap rate is greater than a predefined threshold and an invalid stroke if the overlap rate is less than the predefined threshold;

removing the invalid stroke; and

outputting information of remaining strokes in the binary image of the candidate character strokes.

17. A text extraction method for extracting at least one text line region from a given image, said method comprising:

generating edge information of the given image;

generating a binary image of candidate character strokes in the given image by using the edge information;

removing a false stroke from the binary image using the edge information;

combining a plurality of strokes into a text line region;

removing a false character stroke from the text line region and reforming the text line region;

binarizing the text line region by using a height of the text line region; and

presenting a binary image of the text line region.