Low resolution OCR for camera acquired documents

• US 7,499,588 B2
• Filed: 05/20/2004
• Issued: 03/03/2009
• Est. Priority Date: 05/20/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rinker, James R., Jacobs, Charles E., Simard, Patrice Y., Viola, Paul A.
• Primary Examiner(s): Mariam; Daniel G
• Application Number:
  US10/850,335
• Publication Number:
  US 20050259866A1
• Time in Patent Office: 1,748 Days
• Field of Search: 382/159, 382/173, 382/181, 382/182, 382/190, 382/224, 382/229, 382/156, 382/298, 382/312, 235/435, 715/243
• US Class Current: 382/182
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/153 using recognition of characters or words
• CPC Class Codes: G06V 30/18029 filtering with Haar-like subimages, e.g. computation thereof with the integral image technique
• CPC Class Codes: G06V 30/18057 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 30/268 Lexical context
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

A global optimization framework for optical character recognition (OCR) of low-resolution photographed documents that combines a binarization-type process, segmentation, and recognition into a single process. The framework includes a machine learning approach trained on a large amount of data. A convolutional neural network can be employed to compute a classification function at multiple positions and take grey-level input which eliminates binarization. The framework utilizes preprocessing, layout analysis, character recognition, and word recognition to output high recognition rates. The framework also employs dynamic programming and language models to arrive at the desired output.

Claims

1. An optical character recognition system that facilitates text recognition on a low resolution image, comprising:

at least one processor that executes;

a layout analysis component that determines a set of text lines in the low resolution image, the layout analysis component further segments each text line in the set of lines into individual text words, wherein the layout analysis employs at least two linear filters at each location of the low resolution image;

a character recognition component that segments the individual text words into one or more character portions and provides an observation on the most probable character for each of the one or more character portions; and

a word recognizer that employs dynamic programming mechanisms to ascertain words based upon a series of observations from the character recognition component.

2. The system of claim 1, further comprising a preprocessor component that receives the low resolution image of a text document.

3. The system of claim 2, the preprocessor component employs an adaptive whitebalancing algorithm to maximize contrast locally over regions of the low resolution image.

4. The system of claim 1, the layout analysis component utilizes connected component analysis mechanisms to facilitate determination of the set of text lines.

5. The system of claim 1, the layout analysis component employs at least two linear filters at each location of the low resolution image.

6. The system of claim 1, the at least two linear filters includes at least a first filter configured to identify gaps between text and a second filter designed to identify lines of text.

7. The system of claim 1, the at least two linear filters include thresholds such that locations of the low resolution image that evaluate above the threshold are identified as text.

8. The system of claim 1, the layout analysis component performs a connected components analysis on gaps identified by the at least two linear filters to generate gap connected components, the layout analysis component verifies geometric alignment between gap connected components to ascertain a text line in the low resolution image.

9. The system of claim 1, the layout analysis component identifies columns of whitespace within each text line to determine individual text words within the text line.

10. The system of claim 9, the layout analysis component employs a threshold to determine whitespace, the threshold is established as a function of a minimum and a maximum brightness value along a text line.

11. The system of claim 1, the character recognition component employs a convolutional neural network to provide the observation.

12. The system of claim 11, the convolutional neural network extracts simple features of a character portion at a higher resolution and converts the extracted features into more complex features at a coarse resolution.

13. The system of claim 1, further comprising a language model that facilitates word recognition.

14. The system of claim 13, the language model is a dictionary model that includes an entire lexicon, the word recognition component scans through the lexicon, evaluates a probability for each word of the lexicon based upon correspondence to the series of observations and outputs a word with a highest probability.

15. The system of claim 13, the language model is language neutral and provides an interpretation of the series of observations from the character recognition component with the highest probability of correctness.

16. A computer-implemented method for performing optical character recognition on a low resolution image, comprising:

receiving a low resolution image of a text document;

a processor for implementing the following steps;

identifying lines of text in the low resolution image to generate a set of text lines;

partitioning each text lines in the set of text lines into a plurality of text words;

segmenting each text word into character portions;

determining an observation that includes a probable character imaged in the character portion of the low resolution image, wherein determining the observation comprises employing a convolutional neural network;

and employing dynamic programming techniques to evaluate a series of observations related to probable characters to provide a word determination.

17. The method of claim 16, determining the observation comprises employing a convolutional neural network.

18. The method of claim 16, further comprising training the convolutional neural network to accurately recognize text characters.

19. The method of claim 16, further comprising utilizing a language model to facilitate the word determination.

20. An optical character recognition system, comprising:

at least one processor coupled to memory, the at least one processor configured to act as;

means for receiving a low resolution image of a text document;

means for identifying lines of text in the low resolution image to generate a set of text lines;

means for partitioning each text lines in the set of text lines into a plurality of text words;

means for segmenting each text word into character portions;

means for determining an observation that includes a probable character imaged in the character portion of the low resolution image, wherein the means for determining the observation comprises employing a convolutional neural network;

and means for employing dynamic programming techniques to evaluate a series of observations related to probable characters to provide a word determination.