Automatic document reading system for technical drawings

• US 6,996,295 B2
• Filed: 04/18/2002
• Issued: 02/07/2006
• Est. Priority Date: 01/10/2002
• Status: Expired due to Fees

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Fang, Ming, Tyan, Jenn-Kwei
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US10/125,200
• Publication Number:
  US 20030130992A1
• Time in Patent Office: 1,391 Days
• Field of Search: 382/113, 382/155, 382/156, 382/176, 382/218, 382/305, 345/441, 345/470, 707/3, 707/204
• US Class Current: 382/305
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1444 Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

A system and method for electronic archival of paper-based technical drawings includes a system having a processor, a form learning unit in signal communication with the processor for learning the form of a model drawing image, a form localization unit in signal communication with the processor for localizing the form of an input drawing image, an optical character recognition unit in signal communication with the processor for optically recognizing identification text of the input drawing image, and a result verification unit in signal communication with the processor for verifying the results of the recognized identification text of the input drawing image; and further includes a method for providing a model drawing image, learning the form of the provided model drawing image, receiving an input drawing image, localizing the form of the input drawing image, optically recognizing identification text of the input drawing image, verifying the results of the recognized identification text of the input drawing image, and storing the input drawing image and the corresponding verified identification text into a drawing database.

Claims

1. A method for electronic archival of paper-based technical drawings, the method comprising:

providing a model drawing image;

learning the form of the provided model drawing image;

wherein learning the form further comprises;

extracting form lines from the model drawing image, selecting fields of interest relative to the form lines and programming contexts for the fields of interest into a form template;

receiving an input drawing image;

localizing the form of the input drawing image;

wherein localizing the form further comprises;

preprocessing data of the input drawing image, extracting form lines of the input drawing image, matching the extracted form lines to those of the learned model drawing image and localizing fields of interest relative to the extracted form lines;

optically recognizing identification text of the input drawing image;

verifying the results of the recognized identification text of the input drawing image; and

storing the input drawing image and the corresponding verified identification text into a drawing database.

2. A method as defined in claim 1 wherein extracting form lines is based on a connected-components analysis algorithm, the algorithm comprising:

finding lines;

labelling long connected lines with run-length encoding; and

forming a box from a group of labelled lines.

3. A method as defined in claim 1 wherein selecting fields of interest comprises:

receiving a user selection for a field of interest;

determining whether the selected field of interest is regularly located in the form area or randomly located in the graphic area; and

analyzing the field of interest in accordance with a set of pre-defined field styles.

4. A method as defined in claim 1 wherein programming contexts comprises:

defining the length of a text string and the positions of its characters by indicating where at least one of alphabetic, numeric and special symbols may be expected; and

verifying confusingly similar characters in accordance with potential text fonts and ranges of text lines within a selected field.

5. A method as defined in claim 1 wherein a context relates filled-in text to preprinted text.

6. A method as defined in claim 1 wherein a context relates to a text formatting style.

7. A method as defined in claim 1 wherein preprocessing comprises:

reducing data by resizing the image; and

detecting skew by horizontal frame line projection.

8. A method as defined in claim 1 wherein extracting form lines comprises:

choosing long horizontal and vertical lines for matching;

extracting matched lines from both the template form and the input form;

linking broken lines in accordance with a distance threshold;

adding a virtual line if any side of a borderline is missing; and

parameterizing each line with a box that details the line length, width and position.

9. A method as defined in claim 1 wherein matching the extracted form lines comprises:

iteratively matching lines to estimate the form offset and scaling;

defining a cost function based on distance;

iteratively reducing the average distance between lines in the two sets in accordance with a least squares function;

locating the optimal solution at the function minima; and

using the estimated pose to align the input document with the corresponding template if the minima cost meets a predefined threshold.

10. A method as defined in claim 1 wherein localizing fields of interest comprises:

refining the pose of a field of interest; and

masking out a preprinted region of a field of interest to leave a filled-in data region for subsequent processing.

11. A method as defined in claim 10, further comprising detecting a field of interest randomly located in a graphic area.

12. A method as defined in claim 1 wherein optically recognizing identification text comprises:

analyzing connected components of a field of interest;

recognition-based character segmentation of the connected components;

extracting features from the field of interest; and

hybrid classifier-based character recognition of the features.

13. A method as defined in claim 12 wherein analyzing connected components comprises:

run-length encoding to label and represent connected components;

removing lines and reconstructing character strokes where they may be touching or overlapping with lines;

building components connected to horizontal and vertical lines;

merging neighboring connected components in accordance with geometrical constrains to isolate a segmented character.

14. A method as defined in claim 12 wherein segmentation comprises:

performing character segmentation in accordance with the feedback of character recognition;

recognizing as suspicious a character that touches another character;

defining a discrimination function based on contour projection;

applying the discrimination function to identify split points for the suspicious character.

15. A method as defined in claim 14, further comprising:

constructing a decision tree for merging characters caused by an incorrect splitting process;

recognizing a node of the tree having a significant confidence measure;

selecting the correct segmentation in accordance with the most likely path through the nodes of the tree; and

immediately providing the corresponding character recognition.

16. A method as defined in claim 12 wherein extracting features comprises:

extracting a set of feature vectors from a component;

marking holes with a gray value other than those used for background and object pixels in accordance with a topology analysis that detects the set of background pixels in a hole of a character;

normalizing the size of a character by sampling the character size to a given pixel height and width to provide feature vectors invariant to character size;

padding the background pixels for extreme characters in one dimension to adjust the aspect ratio for the extreme character;

computing a stroke width in a character component to give a dedicated feature for template-matching.

17. A method as defined in claim 12 wherein the hybrid classifier-based character recognition comprises:

neural network classification using a three-layered perceptron and error back-propagation;

hybridizing the neural network classification with a template-matching classification by applying the template-matching classifier only when the neural network classifier gives a low confidence measure; and

determining an output by a weighted sum of their respective results if neither classifier individually produces a result having a sufficient confidence measure.

18. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for electronic archival of paper-based technical drawings, the method steps comprising:

providing a model drawing image;

learning the form of the provided model drawing image;

wherein the method step of learning the form further comprises steps for;

extracting form lines from the model drawing image, selecting fields of interest relative to the form lines and programming contexts for the fields of interest into a form template;

receiving an input drawing image;

localizing the form of the input drawing image;

wherein the method step of localizing the form further comprises steps for;

preprocessing data of the input drawing image, extracting form lines of the input drawing image, matching the extracted form lines to those of the learned model drawing image and localizing fields of interest relative to the extracted form lines;

optically recognizing identification text of the input drawing image;

verifying the results of the recognized identification text of the input drawing image; and

storing the input drawing image and the corresponding verified identification text into a drawing database.

19. A program storage device as defined in claim 18 wherein the method step of optically recognizing identification text comprises steps for:

analyzing connected components of a field of interest;

recognition-based character segmentation of the connected components;

extracting features from the field of interest; and

hybrid classifier-based character recognition of the features.

20. A system for electronic archival of paper-based technical drawings, the system comprising:

providing means for providing a model drawing image;

learning means for learning the form of the provided model drawing image;

wherein the learning means for learning the form further comprises;

extracting means for extracting form lines from the model drawing image, selecting means for selecting fields of interest relative to the form lines and programming means for programming contexts for the fields of interest into a form template;

receiving means for receiving an input drawing image;

localizing means for localizing the form of the input drawing image;

wherein the localizing means for localizing the form further comprises;

preprocessing means for preprocessing data of the input drawing image, extracting means for extracting form lines of the input drawing image, matching means for matching the extracted form lines to those of the learned model drawing image and localizing means for localizing fields of interest relative to the extracted form lines;

recognizing means for optically recognizing identification text of the input drawing image;

verifying means for verifying the results of the recognized identification text of the input drawing image; and

storing means for storing the input drawing image and the corresponding verified identification text into a drawing database.

21. A system as defined in claim 20, further comprising:

analyzing means for analyzing connected components of a field of interest;

segmentating means for recognition-based character segmentation of the connected components;

extracting means for extracting features from the field of interest; and

recognizing means for hybrid classifier-based character recognition of the features.

22. A system (100) for electronic archival of paper-based technical drawings, the system comprising:

a processor (102);

a form learning unit (160) in signal communication with the processor for learning the form of a model drawing image;

wherein the form learning unit (160) for learning the form further comprises;

form line extraction unit (362) for extracting form lines from the model drawing image, fields of interest selection unit (364) for selecting fields of interest relative to the form lines and context programming unit (366) for programming contexts for the fields of interest into a form template;

a form localization unit (170) in signal communication with the processor for localizing the form of an input drawing image;

wherein the form localization unit (170) further comprises;

data processing unit (672) for preprocessing data of the input drawing image, form line extraction unit (674) for extracting form lines of the input drawing image, form matching unit (676) for matching the extracted form lines to those of the learned model drawing image and fields of interest localization unit (678) for localizing fields of interest relative to the extracted form lines;

an optical character recognition unit (180) in signal communication with the processor for optically recognizing identification text of the input drawing image; and

a result verification unit (190) in signal communication with the processor for verifying the results of the recognized identification text of the input drawing image.