DISTRIBUTED COMPUTING ON DOCUMENT FORMATS

• US 20180032478A1
• Filed: 07/29/2016
• Published: 02/01/2018
• Est. Priority Date: 07/29/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Felderman, Gregory S., Hoyt, Brian K.
• Granted Patent: US 11,146,613 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 40/131 Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

According to one embodiment of the present invention, a system includes at least one processor. The system partitions a document into a plurality of data blocks, wherein each data block comprises one or more complete logical units of the document. A plurality of sub-documents is produced from the plurality of data blocks. The sub-documents are processed in parallel by a plurality of processing elements. Embodiments of the present invention further include a method and computer program product for processing a document in parallel in substantially the same manner described above.

Claims

1. A method comprising:

partitioning a document into a plurality of data blocks, wherein each data block comprises one or more complete logical units of the document;

producing a plurality of sub-documents from the plurality of data blocks; and

processing the sub-documents in parallel by a plurality of processing elements.

2. The method of claim 1, wherein each of the plurality of data blocks is of a predetermined size.

3. The method of claim 1, wherein the document includes a format unable to be partitioned based on a size boundary.

4. The method of claim 1, wherein the one or more complete logical units include pages.

5. The method of claim 1, wherein the processing elements include map/reduce jobs of a map/reduce platform.

6. The method of claim 1, wherein the document includes a non-sequential data type, and the method further comprises:

extracting resource information from the document for partitioning of the document into the plurality of data blocks.

7. The method of claim 1, wherein the partitioning includes adjusting content of the plurality of data blocks to enable each data block to comprise the one or more complete logical units.

8. The method of claim 1, wherein the producing a plurality of sub-documents includes formatting the plurality of sub-documents in a format of the document.

9. A system comprising:

at least one processor configured to;

partition a document into a plurality of data blocks, wherein each data block comprises one or more complete logical units of the document;

produce a plurality of sub-documents from the plurality of data blocks; and

process the sub-documents in parallel by a plurality of processing elements.

10. The system of claim 9, wherein the partitioning includes adjusting content of the plurality of data blocks to enable each data block to comprise the one or more complete logical units.

11. The system of claim 9, wherein the document includes a format unable to be partitioned based on a size boundary.

12. The system of claim 9, wherein the one or more complete logical units include pages.

13. The system of claim 9, wherein producing a plurality of sub-documents includes formatting the plurality of sub-documents in a format of the document.

14. The system of claim 9, wherein the document includes a non-sequential data type, and the at least one processor is further configured to:

extract resource information from the document for partitioning of the document into the plurality of data blocks.

15. A computer program product comprising a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code executable by at least one processor to cause the at least one processor to:

partition a document into a plurality of data blocks, wherein each data block comprises one or more complete logical units of the document;

produce a plurality of sub-documents from the plurality of data blocks; and

process the sub-documents in parallel by a plurality of processing elements.

16. The computer program product of claim 15, wherein the partitioning includes adjusting content of the plurality of data blocks to enable each data block to comprise the one or more complete logical units.

17. The computer program product of claim 15, wherein the document includes a format unable to be partitioned based on a size boundary.

18. The computer program product of claim 15, wherein the one or more complete logical units include pages.

19. The computer program product of claim 15, wherein the producing a plurality of sub-documents includes formatting the plurality of sub-documents in a format of the document.

20. The computer program product of claim 15, wherein the document includes a non-sequential data type, and the computer readable program code is further configured to cause the at least one processor to:

extract resource information from the document for partitioning of the document into the plurality of data blocks.