Hierarchical information extraction using document segmentation and optical character recognition correction

• US 10,755,093 B2
• Filed: 06/12/2017
• Issued: 08/25/2020
• Est. Priority Date: 01/27/2012
• Status: Active Grant

• Current Assignee: Open Text Holdings, Inc. (Open Text Corporation)
• Original Assignee: Open Text Holdings, Inc. (Open Text Corporation)
• Inventors: Stadermann, Jan, Jager, Denis, Zernik, Uri
• Primary Examiner(s): Shmatov, Alexey
• Assistant Examiner(s): Bejcek, II, Robert
• Application Number:
  US15/620,733
• Publication Number:
  US 20170277946A1
• Time in Patent Office: 1,170 Days
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/274 Syntactic or semantic context, e.g. balancing
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

Systems, methods, and media for extracting and processing entity data included in an electronic document are provided herein. Methods may include executing one or more extractors to extract entity data within an electronic document based upon an extraction model for the document, selecting extracted entity data via one or more experts, each of the experts applying at least one business rule to organize at least a portion of the selected entity data into a desired format, and providing the organized entity data for use by an end user.

Claims

1. A method for providing extracted entity data from electronic documents, the method comprising:

receiving entity data extracted from an electronic document, the electronic document comprising a scanned version of a hardcopy document;

selecting extracted entity data via two or more experts, each of the experts applying at least one unique business rule to organize at least a portion of the selected entity data into a desired format, wherein the at least one unique business rule comprises a set of slots that comprise properties that define conditions for filling the set of slots with table cell data that includes the extracted entity data;

preventing extraction of entity data from a section of the electronic document having distorted content by;

generating a first-order hidden markov model for each section of the electronic document, based upon a layout of the document;

applying the first-order hidden markov model to a section of the electronic document that includes distorted text to determine the most likely hidden states for the section;

aligning the section with characters extracted from the section of the electronic document; and

configuring one or more extractors and the two or more experts to ignore at least a portion of the electronic document determined to include distorted content, based upon the alignment;

assembling the selected entity data into desired formats;

filling a portion of the set of slots with the portions of the selected entity data; and

outputting a marked phrase from the organized entity data.

2. The method according to claim 1, wherein the organized entity data are arranged into an extensible markup language file.

3. The method according to claim 1, further comprising generating a user interface that includes the organized entity data and a view of the electronic document that includes an annotation for each of the extracted entities.

4. The method according to claim 1, wherein the layout defines a target section and one or more target entity data included in the target section that are to be extracted by the two or more extractors.

5. The method according to claim 1, further comprising filling a slot with extracted entity data when the extracted entity data matches the property for the slot.

6. The method according to claim 5, further comprising validating the slot when the slots of the set are filled with extracted entity data.

7. A system for providing extracted entity data from electronic documents, the system comprising:

two or more experts that each;

receives entity data extracted from an electronic document, the electronic document comprising a scanned version of a hardcopy document;

selects extracted entity data, each of the experts applying at least one unique business rule to organize at least a portion of the selected entity data into a desired format, wherein the at least one unique business rule comprises a set of slots that comprise properties that define conditions for filling the set of slots with table cell data that includes the extracted entity data;

assembles the selected entity data into desired formats; and

fills a portion of the set of slots with the portions of the selected entity data;

a disambiguation module that prevents extraction of entity data from a section of the electronic document having distorted content by;

generating a first-order hidden markov model for each section of the electronic document, based upon a layout of the document;

applying the first-order hidden markov model to a section of the electronic document that includes distorted text to determine the most likely hidden states for the section;

aligning the section with characters extracted from the section of the electronic document; and

configuring one or more extractors and the two or more experts to ignore at least a portion of the electronic document determined to include distorted content, based upon the alignment; and

an output generator that outputs a marked phrase from the organized entity data.

8. The system according to claim 7, wherein the output generator organizes the entity data into an extensible markup language file.

9. The system according to claim 7, wherein the output generator generates a user interface that includes the organized entity data and a view of the electronic document that includes an annotation for each of the extracted entity data.

10. The system according to claim 7, wherein the layout defines a target section and one or more target entity data included in the target section that are to be extracted by the two or more extractors.

11. The system according to claim 7, wherein an expert of the two or more experts fills a slot with extracted entity data when the extracted entity data matches the property for the slot.

12. The system according to claim 11, wherein the expert validates the slot when the slots of the set are filled with extracted entity data.

13. The system according to claim 12, wherein the expert generates a combined set that includes a validated set and one or more additional slots which are to be filled.

14. A non-transitory computer readable storage media having a program embodied thereon, the program being executable by a processor to perform a method for extracting entity data from electronic documents, the method comprising:

receiving entity data extracted from an electronic document, the electronic document comprising a scanned version of a hardcopy document;

normalizing the extracted entity data by applying a normalization scheme to the extracted entity data, the normalization scheme converting the extracted entity data, the normalization scheme converting the extracted entity data into a standardized format;

selecting extracted entity data via two or more experts, each of the experts applying at least one unique business rule to organize at least a portion of the selected entity data into a desired format, wherein the at least one unique business rule comprises a set of slots that comprise properties that define conditions for filling the set of slots with table cell data that includes the extracted entity data;

preventing extraction of entity data from a section of the electronic document having distorted content by;

generating a first-order hidden markov model for each section of the electronic document, based upon a layout of the document;

applying the first-order hidden markov model to a section of the electronic document that includes distorted text to determine the most likely hidden states for the section;

aligning the section with characters extracted from the section of the electronic document; and

configuring one or more extractors and the two or more experts to ignore at least a portion of the electronic document determined to include distorted content, based upon the alignment;

executing table experts that produce special annotations that identify table cells for the electronic document which include the extracted and normalized entity data;

assembling the selected entity data into desired formats;

filling a portion of the set of slots with the portions of the selected entity data; and

outputting a marked phrase from the organized entity data.

15. A method for disambiguation that prevents extraction of entity data from a section of an electronic document having distorted content, the method comprising:

generating a first-order hidden markov model for each section of an electronic document, based upon a layout of the document;

applying the first-order hidden markov model to a section of the electronic document that includes distorted text to determine the most likely hidden states for the section;

aligning the section with characters extracted from the section of the electronic document;

configuring one or more extractors and two or more experts to ignore at least a portion of the electronic document determined to include distorted content, based upon the alignment;

receiving entity data extracted from the electronic document, the electronic document comprising a scanned version of a hardcopy document;

selecting extracted entity data via two or more experts, each of the experts applying at least one unique business rule to organize at least a portion of the selected entity data into a desired format, wherein the at least one unique business rule comprises a set of slots that comprise properties that define conditions for filling the set of slots with table cell data that includes the extracted entity data;

assembling the selected entity data into desired formats;

filling a portion of the set of slots with the portions of the selected entity data; and

outputting a marked phrase from the organized entity data.