COMPUTER-READABLE RECORD MEDIUM IN WHICH A TELOP CHARACTER EXTRACTION PROGRAM IS RECORDED, TELOP CHARACTER EXTRACTION METHOD AND TELOP CHARACTER EXTRACTION APPARATUS

• US 20080187225A1
• Filed: 01/29/2008
• Published: 08/07/2008
• Est. Priority Date: 02/05/2007
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Katsuyama, Yutaka
• Granted Patent: US 7,961,948 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/155 Removing patterns interfering with the pattern to be recognised, such as ruled lines or underlines
• CPC Class Codes: G06V 30/287 of Kanji, Hiragana or Katakana characters

Abstract

A telop character extraction apparatus generates a gray-scale image from a telop region image, and generates an edge region image where an edge is extracted from within the gray-scale image. The apparatus uses the edge region image as a mask image, and creates a luminance histogram of pixels of the gray-scale image. The apparatus divides a luminance of the luminance histogram into luminance classes. The apparatus generates binary images where pixels of luminances belonging to the individual classes are collected for the respective luminance classes, and selects the binary image of the class of maximum area, as a telop character candidate image. The apparatus decides on a character color used in the telop region image, in correspondence with the telop character candidate image, and extracts character pixels corresponding to the character color, from within the telop character candidate image, thereby to eliminate noise pixels and to generate telop character image.

Claims

1. A computer-readable record medium in which a telop character extraction program to be executed by a computer is stored, the telop character extraction program comprising:

an image input unit which receives a telop region image that contains telop characters cut out from color image data;

a gray-scale image generation unit which generates a gray-scale image from the telop region image;

an edge region extraction unit which generates an edge region image where an edge is extracted from within the gray-scale image and is inflated;

a histogram creation unit which uses the edge region image as a mask image, and which creates a luminance histogram of pixels of the gray-scale image as belong to the mask image;

a luminance class generation unit which divides a luminance of the luminance histogram into a plurality of luminance classes;

a character pattern candidate selection unit which generates binary images where pixels of luminances belonging to the individual classes are collected for the respective luminance classes, and which selects the binary image of the class of maximum area, as a telop character candidate image; and

a noise elimination unit which decides on a character color used in the telop region image, in correspondence with the telop character candidate image, and which extracts character pixels corresponding to the character color, from within the telop character candidate image, thereby to eliminate noise pixels and to generate a telop character image.

2. A computer-readable record medium as defined in claim 1, wherein:

said edge region extraction unit generates an edge image in which the edge has been extracted from within the gray-scale image; and

said edge region extraction unit binarizes the edge image, thereby to generate an edge binary image; and

said edge region extraction unit inflates edge pixels of the edge binary image, thereby to generate the edge region image.

3. A computer-readable record medium as defined in claim 1, wherein after having created the luminance histogram, said luminance histogram creation unit calculates a moving average value with a plurality of adjacent luminance histogram values, as each luminance histogram value, thereby to smooth the luminance histogram.

4. A computer-readable record medium as defined in claim 1, wherein:

said luminance class generation unit generates a plurality of sorts of luminance histograms which have luminance classes divided by setting one or more threshold values on a luminance axis of the first-mentioned luminance histogram; and

said luminance class generation unit compares average values of histogram values on the threshold values, as to the plurality of sorts of luminance histograms, so as to select the luminance histogram of minimum average value.

5. A computer-readable record medium as defined in claim 1, wherein:

said luminance class generation unit generates a first luminance histogram which has luminance classes bisected by setting one threshold value on a luminance axis of the luminance histogram, and a second luminance histogram which has luminance classes trisected by setting two threshold values on the luminance axis of the luminance histogram; and

said luminance class generation unit compares a histogram value on the threshold value, in the first luminance histogram of the bisected luminance classes, and an average value of histogram values on the two threshold values, in the second luminance histogram of the trisected luminance classes, so as to select the luminance histogram of smaller value.

6. A computer-readable record medium as defined in claim 1, wherein:

said character pattern candidate selection unit generates binary images in which pixels of luminances belonging to the individual classes are collected for the respective luminance classes;

said character pattern candidate selection unit generates binary label images in each of which character patterns are formed by affixing identical label Nos. to continuous pixels;

said character pattern candidate selection unit detects the label No. of the character pattern which is contiguous to an outer periphery of the telop region image, from the character patterns contained in each of the binary label images;

said character pattern candidate selection unit eliminates character pixels having the detected label No., as background noise; and

said character pattern candidate selection unit obtains areas of binary images from each of which the background noise has been removed, for the respective classes, so as to select the binary image of the class of maximum area as a telop character candidate image.

7. A computer-readable record medium as defined in claim 6, wherein said character pattern candidate selection unit calculates line densities of the respective binary level images of the individual classes in a case where the areas of the binary images obtained for the respective classes do not have a large difference, so as to select the binary image of the class of minimum line density as the telop character candidate image.

8. A computer-readable record medium as defined in claim 7, wherein each of the line densities is a value which is obtained in such a way that a summation obtained by adding numbers of change points from background pixels to character pixels, for respective lines of the corresponding binary image, is divided by the number of the lines of the corresponding binary image.

9. A computer-readable record medium as defined in claim 1, wherein:

said noise elimination unit generates a character color image by taking a logical product between the telop character image and the telop character region image being an original image, so as to extract a maximum frequency color, and extracts character pixels corresponding to the maximum frequency color and eliminates noise pixels from within the telop character candidate image, thereby to generate a telop character image;

wherein said extraction of the maximum frequency color is executed in such a way;

that a three-dimensional color histogram of an RGB color space is generated as to the character color image;

that each of coordinate axes of the RGB color space is divided by n, thereby to create quantized cubelets;

that the number of pixels of the character color image contained in each of the cubelets is counted;

that, in a case where each of the cubelets has a count value which is equal to or larger than count values of the plurality of cubelets surrounding the pertinent cubelet, a value obtained by totalizing all the count values of the surrounding cubelets is set as the count value of the pertinent cubelet; and

that a color belonging to the cubelet of maximum count value is extracted as the maximum frequency color.

10. A computer-readable record medium as defined in claim 1, wherein:

said noise elimination unit generates a character color image by taking a logical product between the telop character image and the telop region image being an original image, so as to obtain an entire average color from the character color image;

said noise elimination unit obtains a regional average color and a regional area for every character pattern of the binary label image, in which pixels of identical label No. are aggregated;

said noise elimination unit calculates a value which is obtained by multiplying a difference of the entire average color and the regional average color by the regional area, as a noise criterion value for every character pattern of the binary label image; and

said noise elimination unit identifies the character pattern whose noise criterion value is larger than a predetermined threshold value, as noise, so as to generate a telop character image in which the noise has been eliminated from the telop character candidate image.

11. A telop character extraction method comprising:

an image input step of receiving a telop region image that contains telop characters cut out from color image data;

a gray-scale image generation step of generating a gray-scale image from the telop region image;

an edge region extraction step of generating an edge region image where an edge is extracted from within the gray-scale image and is inflated;

a histogram creation step of using the edge region image as a mask image, and creating a luminance histogram of pixels of the gray-scale image from the mask image;

a luminance class generation step of dividing a luminance of the luminance histogram into luminance classes;

a character pattern candidate selection step of generating binary images where pixels of luminances belonging to the individual classes are collected for the respective luminance classes, and selecting the binary image of the class of maximum area, as a telop character candidate image; and

a noise elimination step of deciding a character color used in the telop region image, in correspondence with the telop character candidate image, and extracting character pixels corresponding to the character color, from within the telop character candidate image, thereby to eliminate noise pixels and to generate a telop character image.

12. A telop character extraction method as defined in claim 11, wherein:

said edge region extraction step generates an edge image in which the edge has been extracted from within the gray-scale image;

said edge region extraction step binarizes the edge image, thereby to generate an edge binary image; and

said edge region extraction step inflates edge pixels of the edge binary image, thereby to generate the edge region image.

13. A telop character extraction method as defined in claim 11, wherein after having created the luminance histogram, said luminance histogram creation step calculates a moving average value with a plurality of adjacent luminance histogram values, as each luminance histogram value, thereby to smooth the luminance histogram.

14. A telop character extraction method as defined in claim 11, wherein:

said luminance class generation step generates a first luminance histogram which has luminance classes bisected by setting one threshold value on a luminance axis of the luminance histogram, and a second luminance histogram which has luminance classes trisected by setting two threshold values on the luminance axis of the luminance histogram; and

said luminance class generation step compares a histogram value on the threshold value, in the first luminance histogram of the bisected luminance classes, and an average value of histogram values on the two threshold values, in the second luminance histogram of the trisected luminance classes, so as to select the luminance histogram of smaller value.

15. A telop character extraction method as defined in claim 11, wherein:

said character pattern candidate selection step generates binary images in which pixels of luminances belonging to the individual classes are collected for the respective luminance classes;

said character pattern candidate selection step generates binary label images in each of which character patterns are formed by affixing identical label Nos. to continuous pixels;

said character pattern candidate selection step detects the label No. of the character pattern which is contiguous to an outer periphery of the telop region image, from the character patterns contained in each of the binary label images;

said character pattern candidate selection step eliminates character pixels having the detected label No., as background noise;

said character pattern candidate selection step obtains areas of binary images from each of which the background noise has been removed, for the respective classes; and

said character pattern candidate selection step selects the binary image of the class of maximum area as a telop character candidate image.

16. A telop character extraction method as defined in claim 11, wherein:

said character pattern candidate selection step calculates line densities of the respective binary level images of the individual classes in a case where the areas of the binary images obtained for the respective classes do not have a large difference; and

said character pattern candidate selection step selects the binary image of the class of minimum line density as the telop character candidate image.

17. A telop character extraction method as defined in claim 11, wherein:

said noise elimination step generates a character color image by taking a logical product between the telop character image and the telop character region image being an original image, so as to extract a maximum frequency color; and

said noise elimination step extracts character pixels corresponding to the maximum frequency color and eliminates noise pixels from within the telop character candidate image, thereby to generate a telop character image;

wherein said extraction of the maximum frequency color is executed in such a way;

that a three-dimensional color histogram of an RGB color space is generated as to the character color image;

that each of the coordinate axes of the RGB color space is divided by n, thereby to create quantized cubelets;

that the number of pixels of the character color image contained in each of the cubelets is counted;

that, in a case where each of the cubelets has a count value which is equal to or larger than count values of the plurality of cubelets surrounding the pertinent cubelet, a value obtained by totalizing all the count values of the surrounding cubelets is set as the count value of the pertinent cubelet; and

that a color belonging to the cubelet of maximum count value is extracted as the maximum frequency color.

18. A telop character extraction method as defined in claim 11, wherein:

said noise elimination step generates a character color image by taking a logical product between the telop character image and the telop region image being an original image;

said noise elimination step obtains an entire average color from the character color image;

said noise elimination step obtains a regional average color and a regional area for every character pattern of the binary label image, in which pixels of identical label No. are aggregated;

said noise elimination step calculates a value which is obtained by multiplying a difference of the entire average color and the regional average color by the regional area, as a noise criterion value every character pattern of the binary label image; and

said noise elimination step identifies the character pattern whose noise criterion value is larger than a predetermined threshold value, as noise, so as to generate a telop character image in which the noise has been eliminated from the telop character candidate image.

19. A telop character extraction apparatus comprising:

image input unit for receiving a telop region image which contains telop characters cut out from color image data;

gray-scale image generation unit for generating a gray-scale image from the telop region image;

edge region extraction unit for generating an edge region image where an edge is extracted from within the gray-scale image and is inflated;

histogram creation unit for using the edge region image as a mask image, and for creating a luminance histogram of pixels of the gray-scale image as belonging to the mask image;

luminance class generation unit for dividing a luminance of the luminance histogram into a plurality of luminance classes;

character pattern candidate selection unit for generating binary images where pixels of luminances belonging to the individual classes are collected for the respective luminance classes, and for selecting the binary image of the class of maximum area, as a telop character candidate image; and

noise elimination unit for deciding on a character color used in the telop region image, in correspondence with the telop character candidate image, and for extracting character pixels corresponding to the character color, from within the telop character candidate image, thereby to eliminate noise pixels and to generate a telop character image.