BLOCKWISE EXTRACTION OF DOCUMENT METADATA

• US 20190171875A1
• Filed: 12/01/2017
• Published: 06/06/2019
• Est. Priority Date: 12/01/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: NORTHRUP, Kevin, TRIM, Craig M., HICKEY, Terry, JAVU, Thozamile
• Granted Patent: US 10,452,904 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/367 Ontology
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 40/109 Font handling; Temporal or kinetic typography
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/154 Tree transformation for tree-structured or markup documents, e.g. XSLT, XSL-FO or stylesheets
• CPC Class Codes: G06F 40/163 Handling of whitespace
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/412 Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

Methods, computer program products, and systems are presented. The methods include, for instance: obtaining a document image, wherein the document image includes a plurality of objects; identifying a plurality of macroblocks within the document image; performing microblock processing within macroblocks of the plurality of macroblocks, wherein the microblock processing includes examining content of microblocks within a macroblock for extraction of key-value pairs, the examining content including performing an ontological analysis of microblocks, wherein the microblock processing includes associating confidence levels to the extracted key-value pairs; and outputting metadata based on the performing microblock processing within macroblocks of the plurality of macroblocks.

Claims

1. A method comprising:

obtaining a document image, wherein the document image includes a plurality of objects;

identifying a plurality of macroblocks within the document image;

performing microblock processing within macroblocks of the plurality of macroblocks, wherein the microblock processing includes examining content of microblocks within a macroblock for extraction of key-value pairs, the examining content including performing an ontological analysis of microblocks, wherein the microblock processing includes associating confidence levels to the extracted key-value pairs; and

outputting metadata based on the performing microblock processing within macroblocks of the plurality of macroblocks.

2. The method of claim 1, wherein the outputting metadata includes outputting metadata to a process interface.

3. The method of claim 1, wherein the method includes processing the document image to identify a baseline styling parameter value, the baseline styling parameter value specifying a baseline font height;

identifying for each word of a line of text of the document image a relative styling parameter, the relative styling parameter being defined in reference to the baseline styling parameter value, wherein the relative styling parameter specifies a font height of a word of text of the text line as a percentage of the baseline styling parameter value; and

wherein the outputting metadata includes providing the relative styling parameter as output metadata for output.

4. The method of claim 1, wherein the metadata presents a hierarchy including indications of macroblocks, and identified key-value pairs identified and subheaded within designators for each macroblock.

5. The method of claim 1, wherein the examining content of microblocks within an area of a the macroblock includes iteratively performing the examining using different machine logic microblock delineation rules so that for a first iteration a first set of microblocks within the area of the macroblock is identified and for a second iteration a second set of microblocks within the area of the macroblock is identified, wherein a count of microblocks of the second set of microblocks is different from the count of microblocks of the second set of microblocks.

6. The method of claim 1, wherein the outputting includes discarding key-value pairs so that key-value pairs having confidence levels below a threshold are not subject to outputting.

7. The method of claim 1, wherein the examining content includes performing an ontological analysis of unaligned microblocks, and determining that unaligned microblocks are collinear based on the ontological analysis.

8. The method of claim 1, wherein the method includes determining a white space styling parameter value for an area of the document image larger than a microblock and applying a machine logic microblock delineation rule based on the white space styling parameter value.

9. A method comprising:

obtaining a document image, wherein the document image includes a plurality of objects;

identifying a macroblock within the document image, wherein the macroblock includes objects of the plurality of objects;

examining content of microblocks within an area of the macroblock of the document image for extraction of one or more key-value pair, wherein the examining includes examining content of unaligned microblocks within the area of the microblock, and wherein the examining content of unaligned microblocks within the area of the microblock includes applying an ontological analysis;

associating a confidence level to a key-value pair of the one or more key-value pair; and

outputting the one or more key-value pair.

10. The method of claim 9, wherein the examining includes analyzing respective semantic content of both a first microblock and a second microblock within the area of the macroblock, ascertaining that first semantic content of the first microblock is associated with a key name, discovering, from key ontology data corresponding to the key name, that a second semantic content of the second microblock is of a data type corresponding to the key name.

11. The method of claim 9, wherein the examining content of unaligned microblocks is commenced prior to the identifying a macroblock, and wherein the identifying a macroblock within the document image is performed based on the examining content of unaligned microblocks.

12. The method of claim 9, wherein the examining content of unaligned microblocks within an area of a the macroblock includes iteratively performing the examining using different machine logic microblock delineation rules so that for a first iteration a first set of microblocks within the area of the macroblock is identified and for a second iteration a second set of microblocks within the area of the macroblock is identified, wherein a count of microblocks of the second set of microblocks is different from the count of microblocks of the second set of microblocks.

13. The method of claim 9, wherein the method includes processing the document image to identify a baseline styling parameter value, the baseline styling parameter value specifying a baseline font height;

identifying for each word of a line of text of the document image a relative styling parameter, the relative styling parameter being defined in reference to the baseline styling parameter value, wherein the relative styling parameter specifies a font height of a word of text of the text line as a percentage of the baseline styling parameter value; and

wherein the method includes providing the relative styling parameter as output metadata for output.

14. The method of claim 9, wherein the examining content of unaligned microblocks includes performing the examining in a first iteration and in a second iteration, wherein the first iteration results in the identifying the macroblock, and wherein the second iteration is performed subsequent to the identifying the macroblock.

15. The method of claim 9, wherein the examining content of unaligned microblocks includes performing the examining in a first iteration and in a second iteration, wherein the first iteration results in the identifying the macroblock, and wherein the second iteration is performed subsequent to the identifying the macroblock, wherein the performing the examining in in a first iteration includes applying a first machine logic microblock delineation rule to identify relatively fewer microblocks within the area of the macroblock, wherein performing the examining in a second iteration rule includes applying a second machine logic microblock delineation rule to identify relatively more microblocks within the area of the macroblock, wherein the outputting includes outputting the one or more key-value pair as metadata to a process interface selected from the group consisting of a process interface for document processing, a process interface for search engine searching, and a process interface for form regeneration.

16. A method comprising:

obtaining a document image, wherein the document image includes a plurality of objects;

processing the document image to identify a baseline styling parameter value, the baseline styling parameter value specifying a baseline font height;

identifying for each word of a line of text of the document image a relative styling parameter, the relative styling parameter being defined in reference to the baseline styling parameter value, wherein the relative styling parameter specifies a font height of a word of text of the text line as a percentage value the baseline styling parameter value; and

providing the relative styling parameter as output metadata for output.

17. The method of claim 16, wherein the method includes determining the baseline styling parameter value by providing a histogram of font height values in an area of the document that is larger than the line of text, and selecting the baseline styling parameter value based on a central dispersion of the histogram.

18. The method of claim 16, wherein the process interface is a form regenerator.

19. The method of claim 16, wherein the process interface is a form regenerator that is configured to examine the relative styling parameter that specifies a font height of a word of text of the text line as a percentage of the baseline styling parameter value and based on the relative styling parameter indicating a change in height, changing an attribute of the word other than font height in an output document output by the form regenerator.

20. A computer program product comprising:

a computer readable storage medium readable by one or more processing circuit and storing instructions for execution by one or more processor for performing a method comprising;

obtaining a document image, wherein the document image includes a plurality of objects;

identifying a plurality of macroblocks within the document image;

performing microblock processing within macroblocks of the plurality of macroblocks, wherein the microblock processing includes examining content of microblocks within a macroblock for extraction of key-value pairs, the examining content including performing an ontological analysis of microblocks, wherein the microblock processing includes associating confidence levels to the extracted key-value pairs; and

outputting metadata based on the performing microblock processing within macroblocks of the plurality of macroblocks.

21. The computer program product of claim 20, wherein the examining content of microblocks within an area of a the macroblock includes iteratively performing the examining using different machine logic microblock delineation rules so that for a first iteration a first set of microblocks within the area of the macroblock is identified and for a second iteration a second set of microblocks within the area of the macroblock is identified, wherein a count of microblocks of the second set of microblocks is different from the count of microblocks of the second set of microblocks.

22. The computer program product of claim 20, wherein the outputting includes discarding key-value pairs so that key-value pairs having confidence levels below a threshold are not subject to outputting.

23. The computer program product of claim 20, wherein the examining content includes performing an ontological analysis of unaligned microblocks, and determining that unaligned microblocks are collinear based on the ontological analysis.

24. The computer program product of claim 20, wherein the method includes determining a white space baseline styling parameter value for an area of the document image larger than a microblock and applying a machine logic microblock delineation rule based on the white space baseline styling parameter rule.

25. A system comprising:

a memory;

at least one processor in communication with memory; and

program instructions executable by one or more processor via the memory to perform a method comprising;

obtaining a document image, wherein the document image includes a plurality of objects;

identifying a plurality of macroblocks within the document image;

performing microblock processing within macroblocks of the plurality of macroblocks, wherein the microblock processing includes examining content of microblocks within a macroblock for extraction of key-value pairs, the examining content including performing an ontological analysis of microblocks, wherein the microblock processing includes associating confidence levels to the extracted key-value pairs; and

outputting metadata based on the performing microblock processing within macroblocks of the plurality of macroblocks.