PROCESSING TRANSACTIONS IN GRAPH-BASED APPLICATIONS

• US 20110078500A1
• Filed: 12/15/2009
• Published: 03/31/2011
• Est. Priority Date: 09/25/2009
• Status: Active Grant

• Current Assignee: AB Initio Orginal Works LLC
• Original Assignee: Ab Initio Software Corporation
• Inventors: Wholey, Joseph Skeffington III, Bromley, H. Mark, Atterbury, Matthew Darcy, Douros, Bryan Phil, Stanfill, Craig W.
• Granted Patent: US 8,667,329 B2
• Time in Patent Office: Days
• US Class Current: 714/15
• CPC Class Codes: G06F 11/1474 in transactions G06F16/20 takes precedence
• CPC Class Codes: G06F 9/4494 data driven
• CPC Class Codes: G06F 9/466 Transaction processing

Abstract

A graph-based computation has a plurality of nodes representing graph components connected by one or more links representing data flows between the components. Preparing the computation includes: identifying at least a first set of components to process a plurality of transactions in the computation; and associating with a first transaction a first group of one or more data records and data operations corresponding to the data records, and associating with a second transaction a second group of one or more data records and data operations corresponding to the data records. Executing the computation to process at least the first and second transactions includes delaying execution by the first set of components of a data operation corresponding to an initial data record of the second group until after execution by the first set of components of a data operation corresponding to a final data record of the first group.

Claims

1. A computer-implemented method, including:

preparing a graph-based computation having a plurality of nodes representing graph components connected by one or more links representing data flows between the graph components, including;

identifying at least a first set of graph components to process a plurality of transactions in the computation; and

associating with a first transaction a first group of one or more data records and data operations corresponding to the data records, and associating with a second transaction a second group of one or more data records and data operations corresponding to the data records; and

executing the graph-based computation to process at least the first and second transactions, including delaying execution by the first set of graph components of a data operation corresponding to an initial data record of the second group until after execution by the first set of graph components of a data operation corresponding to a final data record of the first group.

2. The method of claim 1 wherein executing the graph-based computation further includes:

executing the first and second transactions as a first batch of transactions by;

processing the data records and data operations corresponding to the first and second transactions; and

upon detecting a failure, rolling back the first batch of transactions by rolling back all processed data records and data operations corresponding to the processed transactions.

3. The method of claim 1 further including:

specifying a first graph component having at least an output port as a beginning of a data flow for the first t of graph components; and

specifying a second graph component having at least an input port as an end of the data flow for the first set of graph components.

4. The method of claim 1 further including dynamically invoking the first set of graph components to process the one or more transactions.

5. The method of claim 1 further including, upon successfully processing substantially all transactions in the first batch of transactions, committing the first batch of transactions.

6. The method of claim 5 in which committing the first batch of transactions includes committing all the data operations corresponding to each one of the transactions in the first batch of transactions at substantially the same time.

7. The method of claim 2 in which detecting a failure includes detecting a failure of any of the data operations corresponding to each one of the data records in a transaction of the first batch of transactions.

8. The method of claim 2 further including after detecting the failure, re-processing, as a second different batch of transactions, unprocessed transactions of the first batch of transactions and the processed transactions that were not committed during processing of the first batch of transactions.

9. The method of claim 2 further including, after detecting the failure, re-processing, as a series of discrete batches, each one of the processed and unprocessed transactions associated with the first batch of transactions.

10. The method of claim 2 further including:

upon detecting the failure, identifying one or more transactions that failed as failed transactions,

re-processing, as a second different batch, the first batch of transactions excluding the failed transactions, and

re-processing, as third batch, the failed transactions.

11. The method of claim 2 further including, in the course of executing the first transaction, performing one or more data operations in a first database by a set of graph components belonging to the graph-based computation.

12. The method of claim 11 further including, in the course of executing the first transaction, establishing a first session with the first database, the first session being shared by one or more of a first set of graph components processing the first transaction.

13. The method of claim 2 in which a size of the first transactions is specified by a user.

14. A computer-readable medium storing a computer program, the computer program including instructions for causing a computer to:

prepare a graph-based computation having a plurality of nodes representing graph components connected by one or more links representing data flows between the graph components, including;

identifying at least a first set of graph components to process a plurality of transactions in the computation; and

associating with a first transaction a first group of one or more data records and data operations corresponding to the data records, and associating with a second transaction a second group of one or more data records and data operations corresponding to the data records; and

execute the graph-based computation to process at least the first and second transactions, including delaying execution by the first set of graph components of a data operation corresponding to an initial data record of the second group until after execution by the first set of graph components of a data operation corresponding to a final data record of the first group.

15. A computer system, including:

means for preparing a graph-based computation having a plurality of nodes representing graph components connected by one or more links representing data flows between the graph components, including;

identifying at least a first set of graph components to process a plurality of transactions in the computation; and

associating with a first transaction a first group of one or more data records and data operations corresponding to the data records, and associating with a second transaction a second group of one or more data records and data operations corresponding to the data records; and

means for executing the graph-based computation to process at least the first and second transactions, including delaying execution by the first set of graph components of a data operation corresponding to an initial data record of the second group until after execution by the first set of graph components of a data operation corresponding to a final data record of the first group.