Method and apparatus for recognizing text in an image sequence of scene imagery

• US 7,333,676 B2
• Filed: 01/11/2006
• Issued: 02/19/2008
• Est. Priority Date: 09/22/2000
• Status: Expired due to Fees

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Myers, Gregory K., Bolles, Robert C., Luong, Quang-Tuan, Herson, James A.
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US11/330,494
• Publication Number:
  US 20060120629A1
• Time in Patent Office: 769 Days
• Field of Search: 382/173, 382/174, 382/176, 382/177, 382/202, 382/229, 382/289, 382/290, 382/181, 382/295, 382/296, 382/105, 382/143, 382/103, 382/106, 382/107, 382/154, 382/192, 382/193, 382/194, 382/286, 382/291, 382/292, 348/147, 348/144, 348/143, 348/42, 348/47, 348/48, 348/153, 348/159, 345/863
• US Class Current: 382/289
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1478 of characters or characters lines

Abstract

An apparatus and a concomitant method for detecting and recognizing text information in a captured imagery. The present method transforms the image of the text to a normalized coordinate system before performing OCR, thereby yielding more robust recognition performance. The present invention also combines OCR results from multiple frames, in a manner that takes the best recognition results from each frame and forms a single result that can be more accurate than the results from any of the individual frames.

Claims

1. Method for recognizing text in a captured imagery, where said captured imagery is of a three-dimensional scene, said method comprising the steps of:

(a) detecting a text region in the captured imagery;

(b) adjusting along three axes said detected text region to produce a rectified image; and

(c) applying optical character recognition (OCR) processing to said rectified image to recognize the text in the captured imagery.

2. The method of claim 1, wherein said adjusting step (b) comprises the step of (b1) computing a base line and a top line for a line of detected text within said detected text region.

3. The method of claim 2, wherein said base line and said top line correlate substantially to horizontal parallel lines of a rectangular bounding box that is fitted to said line of detected text.

4. The method of claim 2, wherein said base line and said top line are estimated by rotating said line of detected text at various angles and then computing a plurality of horizontal projections over a plurality of vertical edge projections.

5. The method of claim 2, wherein said base line is selected comprising the steps of:

locating a plurality of bottom edge pixels, where each bottom edge pixel is located for each column in said rectangular bounding box;

rotating said plurality of bottom edge pixels through a series of angles around an initial estimated text angle for said line of detected text;

summing horizontally along each row; and

determining a baseline angle from a maximum sum of squared projections and determining a baseline position from a maximum projection.

6. The method of claim 2, wherein said top line is selected comprising the steps of:

locating a plurality of top edge pixels, where each top edge pixel is located for each column in said rectangular bounding box;

rotating said plurality of top edge pixels through a series of angles around an initial estimated text angle for said line of detected text;

summing horizontally along each row; and

determining a top line angle from a maximum sum of squared projections and determining a top line position from a maximum projection.

7. The method of claim 2, wherein said adjusting step (b) further comprises the step of (b2) computing a dominant vertical direction of character strokes for a line of detected text within said detected text region.

8. The method of claim 1, further comprising the step of:

(b1) binarizing said detected text region prior to applying said OCR processing step (c).

9. The method of claim 8, further comprising the step of:

(d) applying agglomeration processing subsequent to said OCR processing to produce the text in the captured imagery.

10. The method of claim 9, further comprising the step of:

(e) applying lexicon processing subsequent to said agglomeration processing to produce the text in the captured imagery.

11. The method of claim 10, further comprising the step of:

(f) applying false text elimination processing subsequent to said lexicon processing to produce the text in the captured imagery.

12. Apparatus for recognizing text in a captured imagery, where said captured imagery is of a three-dimensional scene, said apparatus comprising:

means for detecting a text region in the captured imagery;

means for adjusting along three axes said detected text region to produce a rectified image; and

means for applying optical character recognition (OCR) processing to said rectified image to recognize the text in the captured imagery.

13. The apparatus of claim 12, wherein said adjusting means computes a base line and a top line for a line of detected text within said detected text region.

14. The apparatus of claim 13, wherein said base line and said top line correlate substantially to horizontal parallel lines of a rectangular bounding box that is fitted to said line of detected text.

15. The apparatus of claim 13, wherein said base line and said top line are estimated by rotating said line of detected text at various angles and then computing a plurality of horizontal projections over a plurality of vertical edge projections.

16. The apparatus of claim 15, wherein said base line is selected that corresponds to a rotation angle that yields a steepest slope on a bottom side of one of said plurality of horizontal projections.

17. The apparatus of claim 13, wherein said adjusting means further computes a dominant vertical direction of character strokes for a line of detected text within said detected text region.