TEXT REGION DETECTION SYSTEM AND METHOD

• US 20120224765A1
• Filed: 12/13/2011
• Published: 09/06/2012
• Est. Priority Date: 03/04/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Koo, Hyung-Il, Hwang, Kyu Woong, Kim, Chunghoon
• Granted Patent: US 8,867,828 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

A method for detecting a text region in an image is disclosed. The method includes detecting a candidate text region from an input image. A set of oriented gradient images is generated from the candidate text region, and one or more detection window images of the candidate text region are captured. A sum of oriented gradients is then calculated for a region in one of the oriented gradient images. It is classified whether each detection window image contains text by comparing the associated sum of oriented gradients and a threshold. Based on the classifications of the detection window images, it is determined whether the candidate text region is a true text region.

Claims

1. A method for detecting a text region in an image, the method comprising:

detecting a candidate text region from an input image;

generating a set of oriented gradient images of the candidate text region;

capturing one or more detection window images of the candidate text region;

calculating a sum of oriented gradients for a region in one of the oriented gradient images;

classifying whether each detection window image contains text by comparing the associated sum of oriented gradients and a threshold; and

determining whether the candidate text region is a true text region based on the classifications of the detection window images.

2. The method of claim 1, wherein the region is in a portion of the one of the oriented gradient images and the portion corresponds to each detection window image.

3. The method of claim 1, wherein a plurality of classifiers arranged in parallel calculates a plurality of sums of oriented gradients for one or more regions in one or more of the oriented gradient images.

4. The method of claim 3, wherein each of the classifiers classifies whether each detection window image contains text image by comparing the associated sum of oriented gradients and threshold.

5. The method of claim 4, wherein each detection window image is classified as containing text based on the classifications of the classifiers.

6. The method of claim 3, wherein the one or more oriented gradient images have at least one orientation selected from a vertical, horizontal and diagonal directions.

7. The method of claim 1, wherein each detection window image overlaps partially with one or more other detection window images.

8. The method of claim 1, further comprising:

converting the candidate text region into a binary image; and

verifying whether the candidate text region is a true text region based on one or more features of the binary image.

9. The method of claim 8, wherein verifying the candidate text region comprises:

splitting the binary image into a plurality of blocks;

calculating the one or more features from the blocks;

calculating a likelihood that each block contains text; and

determining whether the candidate text region is a true text region based on the likelihoods for the blocks.

10. The method of claim 9, wherein calculating the one or more features comprises:

dividing each of the blocks into a plurality of sub-blocks; and

calculating the one or more features from one or more of the sub-blocks.

11. The method of claim 10, wherein the one or more features include at least one of the number of white pixels, the number of black pixels, the number of white pixel to black pixel transitions, and the number of black pixel to white pixel transitions.

12. A method for detecting a text region in an image, the method comprising:

i) detecting a candidate text region from an input image;

ii) generating a set of oriented gradient images of the candidate text region;

iii) capturing one or more detection window images from the candidate text region;

iv) calculating a sum of oriented gradients from a region in an oriented gradient image obtained from each detection window image;

v) classifying whether each detection window image contains text by comparing the sum of oriented gradients and a threshold;

vi) determining whether each detection window image contains text based on each classification; and

vii) if each detection window image is determined to contain text, repeating iv) to vi) to determine whether the candidate text region is a true text region.

13. The method of claim 12, wherein the region is in a portion of the one of the oriented gradient images and the portion corresponds to each detection window image.

14. The method of claim 12, wherein a plurality of classifiers arranged in parallel calculates a plurality of sums of oriented gradients from one or more regions in one or more of the oriented gradient images.

15. The method of claim 14, wherein each of the classifiers classifies whether each detection window image contains text by comparing the associated sum of oriented gradients and threshold.

16. The method of claim 15, wherein each detection window image is determined to contain text based on the classifications of the classifiers.

17. The method of claim 14, wherein the one or more oriented gradient images have at least one orientation selected from a vertical, horizontal and diagonal directions.

18. The method of claim 12, wherein one or more of the detection window images overlap partially.

19. The method of claim 12, further comprising:

converting the candidate text region into a binary image; and

verifying whether the candidate text region is a true text region based on one or more features of the binary image.

20. The method of claim 19, wherein verifying the candidate text region comprises:

splitting the binary image into a plurality of blocks;

calculating the one or more features from the blocks;

calculating a likelihood that each block contains text; and

determining whether the candidate text region is a true text region based on the likelihoods for the blocks.

21. The method of claim 20, wherein calculating the one or more features comprises:

dividing each of the blocks into a plurality of sub-blocks; and

calculating the one or more features from one or more of the sub-blocks.

22. The method of claim 21, wherein the one or more features include at least one of the number of white pixels, the number of black pixels, the number of white pixel to black pixel transitions, and the number of black pixel to white pixel transitions.

23. A method for generating one or more classifiers for classifying an image into a text region, the method comprising:

providing a plurality of oriented gradient images generated from learning sample images, one or more oriented gradient images being generated from each learning sample image including a tag indicating whether the learning sample image is a text image or non-text image;

generating a plurality of candidate weak classifiers, each candidate weak classifier including parameters defining a region and a direction of an oriented gradient image and being configured to classify each learning sample image based on a sum of oriented gradients calculated for the defined region and direction;

verifying whether each candidate weak classifier correctly classifies each learning sample image based on the tag;

determining an error rate for each candidate weak classifier; and

selecting one or more weak classifiers from the candidate weak classifiers based on the error rate of each candidate weak classifier.

24. The method of claim 23, wherein each candidate weak classifier further includes a parameter defining a threshold for the sum of oriented gradients.

25. The method of claim 24, wherein each candidate weak classifier is further configured to classify each learning sample image based on the threshold.

26. The method of claim 23, further comprising adjusting weights of the learning sample images based on the verification of the selected one or more weak classifiers.

27. The method of claim 23, wherein the method is performed using an AdaBoost algorithm.

28. An apparatus for detecting a text region in an image, the apparatus comprising:

a candidate text region detector configured to receive an input image and detect a candidate text region from the input image;

a gradient image generator configured to receive the candidate text region and generate a set of oriented gradient images of the candidate text region;

a detection window locator configured to capture one or more detection window images of the candidate text region;

a classifier configured to classify whether each detection window image contains text based on a plurality of sums of oriented gradients, each of the sums of oriented gradients being calculated for a region in one of the oriented gradient images; and

a first determination unit configured to determine whether the candidate text region is a true text region based on the classifications of the detection window images.

29. The apparatus of claim 28, wherein the region is in a portion of the one of the oriented gradient images and the portion corresponds to each detection window image.

30. The apparatus of claim 28, wherein the classifier includes a plurality of sub-classifiers arranged in parallel, each sub-classifier being configured to calculate one of the sums of oriented gradients and classify whether each detection window image contains text based on the sum of oriented gradients.

31. The apparatus of claim 30, wherein each sub-classifier calculates a sum of oriented gradients based on an orientation for one of the oriented gradient images and the region that matches an area of the detection window images in size and location.

32. The apparatus of claim 31, wherein each sub-classifier classifies whether each detection window image contains text by comparing the sum of oriented gradients with a threshold.

33. The apparatus of claim 32, wherein the classifier further includes a determinator configured to determine whether each detection window image contains text based on the classifications of the sub-classifiers.

34. The apparatus of claim 30, wherein at least one of the orientation, the size and location of the region, and the threshold are determined based on an error rate of classification by each sub-classifier.

35. The apparatus of claim 29, wherein the sub-classifiers are configured based on an AdaBoost algorithm.

36. The apparatus of claim 28, wherein one or more of the detection window images overlap partially.

37. The apparatus of claim 28, further comprising

a binary conversion unit configured to convert the candidate text region into a binary image;

and

a binary image classifier configured to verify whether the candidate text region is a true text region based on one or more features of the binary image.

38. The apparatus of claim 37, wherein the binary image classifier is further configured to split the binary image into a plurality of blocks and calculate the one or more features from the blocks.

39. The apparatus of claim 38, wherein the binary image classifier is further configured to divide each of the blocks into a plurality of sub-blocks and calculate the one or more features from one or more of the sub-blocks.

40. The apparatus of claim 37, wherein the one or more features include at least one of the number of white pixels, the number of black pixels, the number of white pixel to black pixel transitions, and the number of black pixel to white pixel transitions.

41. An apparatus for detecting a text region in an image, the apparatus comprising:

a candidate text region detector configured to receive an input image and detect a candidate text region from the input image;

a gradient image generator configured to receive the candidate text region and generate a set of oriented gradient images of the candidate text region;

a detection window locator configured to capture one or more detection window images of the candidate text region;

a plurality of classifiers arranged in series, each classifier being configured to classify whether each detection window image contains text based on a plurality of sums of oriented gradients when the classification from the previous classifier in the series indicates that the detection window image contains text, each of the sums of oriented gradients being calculated for a region in one of the oriented gradient images; and

a first determination unit configured to determine whether the candidate text region is a true text region based on the classifications of the detection window images.

42. The apparatus of claim 41, wherein the region is in a portion of the one of the oriented gradient images and the portion corresponds to each detection window image.

43. The apparatus of claim 41, wherein the classifier includes a plurality of sub-classifiers arranged in parallel, each sub-classifier being configured to calculate one of the sums of oriented gradients and classify whether each detection window image contains text based on the sum of oriented gradients.

44. The apparatus of claim 43, wherein each sub-classifier calculates a sum of oriented gradients based on an orientation for one of the oriented gradient images and the region that matches an area of the detection window images in size and location.

45. The apparatus of claim 44, wherein each sub-classifier classifies whether each detection window image contains text by comparing the sum of oriented gradients with a threshold.

46. The apparatus of claim 43, wherein at least one of the orientation, the size and location of the region, and the threshold are determined based on an error rate of classification by each sub-classifier.

47. The apparatus of claim 42, wherein the sub-classifiers are configured based on an AdaBoost algorithm.

48. A non-transitory computer-readable storage medium comprising instructions for detecting a text region in an image, the instructions causing a processor to perform the operations of:

detecting a candidate text region from an input image;

generating a set of oriented gradient images of the candidate text region;

capturing one or more detection window images of the candidate text region;

calculating a sum of oriented gradients for a region in one of the oriented gradient images;

classifying whether each detection window image contains text by comparing the associated sum of oriented gradients and a threshold; and

determining whether the candidate text region is a true text region based on the classifications of the detection window images.

49. The medium of claim 48, wherein the region is in a portion of the one of the oriented gradient images and the portion corresponds to each detection window image.

50. The medium of claim 48, wherein calculating further comprises calculating a plurality of sums of oriented gradients for one or more regions in one or more of the oriented gradient images.

51. The medium of claim 50, wherein classifying further comprises classifying whether each detection window image contains text image by comparing each sum of oriented gradients and associated threshold.

52. The medium of claim 51, wherein each detection window image is classified as containing text based on the classifications of the classifiers.

53. The medium of claim 50, wherein the one or more oriented gradient images have at least one orientation selected from a vertical, horizontal and diagonal directions.

54. The medium of claim 48, wherein each detection window image overlaps partially with one or more other detection window images.

55. The medium of claim 48, wherein the operations further comprise converting the candidate text region into a binary image, and verifying whether the candidate text region is a true text region based on one or more features of the binary image.

56. The medium of claim 55, wherein verifying the candidate text region comprises:

splitting the binary image into a plurality of blocks;

calculating the one or more features from the blocks;

calculating a likelihood that each block contains text; and

determining whether the candidate text region is a true text region based on the likelihoods for the blocks.

57. The medium of claim 56, wherein calculating the one or more features comprises dividing each of the blocks into a plurality of sub-blocks, and calculating the one or more features from one or more of the sub-blocks.

58. The medium of claim 57, wherein the one or more features include at least one of the number of white pixels, the number of black pixels, the number of white pixel to black pixel transitions, and the number of black pixel to white pixel transitions.

59. A non-transitory computer-readable storage medium comprising instructions for detecting a text region in an image, the instructions causing a processor to perform the operations of:

i) detecting a candidate text region from an input image;

ii) generating a set of oriented gradient images of the candidate text region;

iii) capturing one or more detection window images from the candidate text region;

iv) calculating a sum of oriented gradients from a region in an oriented gradient image obtained from each detection window image;

v) classifying whether each detection window image contains text by comparing the sum of oriented gradients and a threshold;

vi) determining whether each detection window image contains text based on each classification; and

vii) if each detection window image is determined to contain text, repeating iv) to vi) to determine whether the candidate text region is a true text region.

60. A non-transitory computer-readable storage medium comprising instructions for generating one or more classifiers for classifying an image into a text region, the instructions causing a processor to perform the operations of:

providing a plurality of oriented gradient images generated from learning sample images, one or more oriented gradient images being generated from each learning sample image including a tag indicating whether the learning sample image is a text image or non-text image;

generating a plurality of candidate weak classifiers, each candidate weak classifier including parameters defining a region and a direction of an oriented gradient image and being configured to classify each learning sample image based on a sum of oriented gradients calculated for the defined region and direction;

verifying whether each candidate weak classifier correctly classifies each learning sample image based on the tag;

determining an error rate for each candidate weak classifier; and

selecting one or more weak classifiers from the candidate weak classifiers based on the error rate of each candidate weak classifier.

61. The medium of claim 60, wherein each candidate weak classifier further includes a parameter defining a threshold for the sum of oriented gradients.

62. The medium of claim 61, wherein each candidate weak classifier is further configured to classify each learning sample image based on the threshold.

63. The medium of claim 60, wherein the operations further comprise adjusting weights of the learning sample images based on the verification of the selected one or more weak classifiers.

64. The medium of claim 60, wherein the instructions are configured to generate the classifiers based on an AdaBoost algorithm.

65. An apparatus for detecting a text region in an image, the apparatus comprising:

means for receiving an input image and detecting a candidate text region from the input image;

means for receiving the candidate text region and generating a set of oriented gradient images of the candidate text region;

means for capturing one or more detection window images of the candidate text region;

means for classifying whether each detection window image contains text based on a plurality of sums of oriented gradients, each of the sums of oriented gradients being calculated for a region in one of the oriented gradient images; and

means for determining whether the candidate text region is a true text region based on the classifications of the detection window images.

66. The apparatus of claim 65, wherein the region is in a portion of the one of the oriented gradient images and the portion corresponds to each detection window image.

67. The apparatus of claim 65, wherein the means for classifying includes a plurality of means for calculating one of the sums of oriented gradients and classifying whether each detection window image contains text based on the sum of oriented gradients, wherein the plurality of means for calculating and classifying are arranged in parallel.

68. The apparatus of claim 67, wherein each means for calculating and classifying calculates a sum of oriented gradients based on an orientation for one of the oriented gradient images and the region that matches an area of the detection window images in size and location.

69. The apparatus of claim 68, wherein each means for calculating and classifying classifies whether each detection window image contains text by comparing the sum of oriented gradients with a threshold.

70. The apparatus of claim 69, wherein the means for classifying further includes a means for determining whether each detection window image contains text based on the classifications of the plurality of means for calculating and classifying.

71. The apparatus of claim 67, wherein at least one of the orientation, the size and location of the region, and the threshold are determined based on an error rate of classification by each of the plurality of means for calculating and classifying.

72. The apparatus of claim 66, wherein the means for calculating and classifying are configured based on an AdaBoost algorithm.

73. The apparatus of claim 65, wherein one or more of the detection window images overlap partially.

74. The apparatus of claim 65, further comprising

means for converting the candidate text region into a binary image;

and

means for verifying whether the candidate text region is a true text region based on one or more features of the binary image.

75. The apparatus of claim 74, wherein the means for verifying is further configured to split the binary image into a plurality of blocks and calculate the one or more features from the blocks.

76. The apparatus of claim 75, wherein the means for verifying is further configured to divide each of the blocks into a plurality of sub-blocks and calculate the one or more features from one or more of the sub-blocks.

77. The apparatus of claim 74, wherein the one or more features include at least one of the number of white pixels, the number of black pixels, the number of white pixel to black pixel transitions, and the number of black pixel to white pixel transitions.

78. An apparatus for detecting a text region in an image, the apparatus comprising:

means for receiving an input image and detecting a candidate text region from the input image;

means for receiving the candidate text region and generating a set of oriented gradient images of the candidate text region;

means for capturing one or more detection window images of the candidate text region;

a plurality of means for classifying whether each detection window image contains text based on a plurality of sums of oriented gradients when the classification from the previous classifier in the series indicates that the detection window image contains text, each of the sums of oriented gradients being calculated for a region in one of the oriented gradient images, wherein the plurality of means for classifying is arranged in series; and

means for determining whether the candidate text region is a true text region based on the classifications of the detection window images.