Method and server cluster for map reducing flow services and large documents

• US 8,682,998 B2
• Filed: 12/12/2008
• Issued: 03/25/2014
• Est. Priority Date: 10/31/2008
• Status: Active Grant

• Current Assignee: Software AG
• Original Assignee: Software AG
• Inventors: Reddy Byreddy, Bhaskar, Ramaiah, Ramu, Punnoose, Vinay
• Primary Examiner(s): Feild, Lynn
• Assistant Examiner(s): EDWARDS, JAMES A
• Application Number:
  US12/314,607
• Publication Number:
  US 20100115046A1
• Time in Patent Office: 1,929 Days
• Field of Search: 709/208
• US Class Current: 709/208
• CPC Class Codes: G06F 2209/5017 Task decomposition
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

The present invention refers to a method for MapReducing the processing of an Electronic Data Interchange (EDI) document (1, the method comprising the following steps:

• • a. mapping the EDI document (1) into a plurality of intermediate documents (10, 11);
  • b. processing the intermediate documents (10, 11) to produce a plurality of intermediate results (20-23);
  • c. reducing the plurality of intermediate results (20-23) to produce a plurality of reduced intermediate results (30, 31); and
  • d. reducing the reduced intermediate results (30, 31) to produce a final result (2) representing the processed EDI document (1).

Claims

1. A method for adapting execution of a pre-existing service that includes a predefined internal processing logic that, when executed on a processing system, is configured to process an Electronic Data Inter-change (EDI) document, the pre-existing service also including a predetermined input signature and a predetermined output signature, the method comprising:

accepting the EDI document as input through a mapper service, the input to the mapper service conforming to the predetermined input signature of the pre-existing service;

(a) mapping the EDI document, via the mapper service, into a plurality of intermediate documents;

(b) processing the intermediate documents on a plurality of instances of the pre-existing service, to produce a plurality of intermediate results, each one of the plurality of intermediate results conforming to the predetermined output signature of the pre-existing service, each one of the plurality of instances executing processing that is based on the predefined internal processing logic of the pre-existing service;

(c) reducing the plurality of intermediate results, via a first reducer service, to produce a plurality of reduced intermediate results; and

(d) reducing the reduced intermediate results, via a second reducer service, to produce a final result representing a result of processing the EDI document, the result conforming to the predetermined output signature of the pre-existing service,

wherein the predefined internal processing logic of the pre-existing service is different from processing logic in the mapper service, the first reducer service, and the second reducer service,

wherein the pre-existing service is defined independently of the mapper service, the first reducer service, and the second reducer service, and

wherein the executed processing of the plurality of instances is performed without having to rewrite the internal processing logic of the pre-existing service upon which the executed processing is based.

2. The method of claim 1, wherein the EDI document is mapped such that each of the intermediate documents comprises at least one of a plurality of interchange envelopes, at least one of a plurality of functional group envelopes and/or at least one of a plurality of transaction set envelopes of the EDI document.

3. The method of claim 1, wherein (a) and (d) are performed by a master server of a server cluster and wherein (b) and (c) are performed by a plurality of slave servers of the server cluster, each slave server processing one or more intermediate documents and reducing one or more intermediate results.

4. The method of claim 3, further comprising sending the intermediate documents to the slave servers from the master server by an asynchronous invocation.

5. The method of claim 3, wherein the EDI document is stored in a distributed file system accessible to the slave servers and wherein the method further comprises sending a reference to the intermediate documents to the slave servers from the master server by an asynchronous invocation.

6. The method of claim 1, wherein each of the intermediate results comprises an identifier relating the respective intermediate result to the EDI document.

7. The method of claim 1, wherein a processing logic for performing the processing of the slave servers in (b) is distributed to the slave servers during runtime.

8. A server cluster comprising a master server and a plurality of slave servers configured to perform the method of claim 1.

13. A non-transitory computer readable-storable medium storing computer readable instructions the stored instructions comprising instructions configured to perform the method of claim 1.

9. A method for adapting execution of a pre-existing FLOW service that includes to a predetermined input signature, a predetermined output signature, and a predetermined internal processing logic, the method comprising:

accepting input through a mapper service, the input to the mapper service conforming to the predetermined input signature of the pre-existing FLOW service;

mapping the input, via the mapper service, to a plurality of intermediate inputs;

dynamically parallelizing the pre-existing FLOW service by instantiating a plurality of instances of the FLOW service, each one of the plurality of instances having an input signature and output signature that respectively conform to the predetermined input and output signature of the pre-existing FLOW service;

sending the plurality of intermediate inputs to the plurality of instances;

executing the plurality of instances with the intermediate inputs, the processing in the respective instances performed via processing logic that is substantially the same as the predetermined internal processing logic, to thereby produce a plurality of intermediate results for each one of the plurality of instances;

reducing the plurality of intermediate results for each instance in the plurality of instances into a plurality of reduced intermediate results via a first reducer service;

reducing the plurality of reduced intermediate results to produce a final output via a second reducer service, the final output conforming to the predetermined output signature; and

outputting the final output from the second reducer service,

wherein the predetermined internal processing logic of the pre-existing FLOW service is different from processing logic in the mapper service, the first reducer service, and the second reducer service,

wherein the pre-existing FLOW service is defined independently of said mapping and said reducing, and

wherein the processing logic in the respective instances is executed without having to rewrite the predetermined internal processing logic upon which said processing logic is based.

10. The method of claim 9, wherein the mapper service and the second reducer service are executed on a master server of a server cluster and wherein the plurality of instances of the FLOW service and the plurality of first reducer services are executed on a plurality of slave servers of the server cluster.

11. The method of claim 9, wherein the first predetermine input signature includes an Electronic Data Interchange (EDI) document.

12. A server cluster comprising a master server and a plurality of slave servers configured to perform the method of claim 9.

14. A non-transitory computer readable-storable medium storing computer readable instructions for performing a MapReduce method in association with a pre-existing FLOW service that includes a predetermined input signature, a predetermined output signature, and a predefined internal processing structure that is configured to process input that is in conformance with the predetermined input signature and provide output in conformance with the predetermined output signature, the stored instructions comprising instructions configured to:

accept input through a mapper service, the input to the mapper service conforming to the predetermined input signature of the pre-existing FLOW service;

map the input, via the mapper service, to a plurality of intermediate inputs;

dynamically parallelize the pre-existing FLOW service by instantiating a plurality of instances of the FLOW service, each one of the plurality of instances having an input signature and output signature that respectively conform to the predetermined input and output signature of the pre-existing FLOW service;

send the plurality of intermediate inputs to the plurality of instances;

execute the plurality of instances to process the intermediate inputs to thereby produce a plurality of intermediate results for each one of the plurality of instances, the execution of the plurality of instances including performing processing that is based on the predefined internal processing logic of the pre-existing FLOW service;

reduce the plurality of intermediate results for each instance in the plurality of instances into a plurality of reduced intermediate results via a first reducer service;

reduce the plurality of reduced intermediate results to produce a final output via a second reducer service, the final output conforming to the predetermined output signature; and

output the final output,

wherein the predetermined internal processing logic of the pre-existing FLOW service is different from processing logic in the mapper service, the first reducer service, and the second reducer service,

wherein the pre-existing FLOW service is defined independently of said mapping, said reducing, the mapper service, the first reducer service, and the second reducer service, and

wherein the performed processing in the plurality of instances is done without having to rewrite the predefined internal processing logic of the pre-existing FLOW service.