INGESTION PLANNING FOR COMPLEX TABLES

• US 20170116190A1
• Filed: 10/17/2016
• Published: 04/27/2017
• Est. Priority Date: 10/23/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bastide, Paul R., Broomhall, Matthew E., Byron, Donna K., Loredo, Robert E.
• Granted Patent: US 9,928,240 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/243 Natural language query formulation
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/3326 using relevance feedback from the user, e.g. relevance feedback on documents, documents sets, document terms or passages
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 40/151 Transformation
• CPC Class Codes: G06F 40/177 of tables; using ruled lines
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: G06F 40/56 Natural language generation

Abstract

Embodiments of the present invention disclose a method, computer program product, and system for generating a plan for document processing. A plurality of electronic documents are received from a data store. The plurality of electronic documents are analyzed. Textual data within the identified tabular data are identified, by performing a first natural language search of the analyzed plurality of electronic documents. Textual hints are generated, where the generated textual hints are mapped to a lookup set. References are identified, and a count of identified references are determined. A priority score is calculating based on the count of identified references. In response to receiving a priority score modifying value, a modified priority score is calculated. Ingestion plans are generated based on the modified priority score. Generated ingestion plans are communicated by the computer using the network.

Claims

1. A computer program product for generating a plan for document processing, the computer program product comprising:

one or more computer-readable storage media and program instructions stored on the one or more computer-readable storage media, the program instructions comprising;

program instructions to receive a plurality of electronic documents from a data store, by a computer using a network;

program instructions to analyze the plurality of electronic documents, using the computer to identify a plurality of tabular data by performing a search for one or more table markers, based on the analyzed plurality of electronic documents;

program instructions to identify textual data within the identified tabular data, by performing a first natural language search of the analyzed plurality of electronic documents;

program instructions to generate textual hints, based on the identified textual data within the identified tabular data by associating identified textual data into a set using a second natural language search;

program instructions to map the generated textual hints to a lookup set;

program instructions to identify references, wherein references are based on mapped textual hints with associated identified textual data in the received plurality of electronic documents;

program instructions to determine a count of identified references;

program instructions to calculate a priority score based on the count of identified references, wherein the program instructions to calculate further comprises program instructions to multiply the count of identified references by a predetermined scale value;

in response to program instructions to receive a priority score modifying value, wherein the priority score modifying value is a numerical value, program instructions to calculate a modified priority score, wherein the program instructions to calculate further comprises program instructions to multiply the priority score by the received priority score modifying value;

program instructions to generate one or more ingestion plans based on the modified priority score; and

program instructions to communicate the one or more generated ingestion plans by the computer using the network.