MANAGING DEPENDENCIES BETWEEN OPERATIONS IN A DISTRIBUTED SYSTEM

• US 20150172412A1
• Filed: 07/06/2013
• Published: 06/18/2015
• Est. Priority Date: 07/06/2012
• Status: Abandoned Application

• Current Assignee: Cornell University
• Original Assignee: Cornell University
• Inventors: Sirer, Emin Gun, Escriva, Robert, Wong, Bernard
• Application Number:
  US14/412,105
• Publication Number:
  US 20150172412A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 9/466 Transaction processing
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/62 Establishing a time schedule for servicing the requests

Abstract

An efficient fault-tolerant event ordering service as well as a simplified approach to transaction processing based on global event ordering determines the order of interdependent operations in a distributed system. The fault-tolerant event ordering service externalizes the task of tracking dependencies to capture a global view of dependencies between a set of distributed operations in a distributed system. A novel protocol referred to as linear transactions coordinates distributed transactions with Atomicity, Consistency, Isolation, Durability (ACID) semantics on top of a sharded data store. The linear transactions protocol achieves scalability by distributing the coordination task to only those servers that hold relevant data for each transaction and achieves high performance by serializing only those transactions whose concurrent execution could potentially yield a violation of ACID semantics.

Claims

1. A method of operation of a computer for managing time dependencies in a distributed system including two or more subsystems with each subsystem including at least one event, wherein the computer comprises a central control unit, a storage system, and a network interface device, comprising the steps of:

receiving by the central control unit through the network interface device two or more events from the two or more subsystems;

building by the central control unit an event dependency graph, wherein the event dependency graph includes a plurality of vertices with each vertex representing an event and a plurality of edges with each edge representing a happens-before relationship;

storing the event dependency graph in the storage system;

tracking by the central control unit dependencies between the two or more events that traverse the two or more subsystems;

selecting by the central control unit an order of the two or more events as late as possible; and

executing in each subsystem the two or more events according to the order selected by the central control unit.

2. The method according to claim 1, wherein each edge of the plurality of edges is added to the event dependency graph when dependencies are added between the two or more events.

3. The method according to claim 1, wherein the plurality of edges includes specially marked edges representing explicitly created happens-before dependencies.

4. The method according to claim 1, wherein the plurality of edges includes automatically deduced edges representing transitively-computed dependencies not explicitly instantiated.

5. The method according to claim 1 further comprising the step of using the event dependency graph to answer queries regarding the ordering between two or more new events.

6. The method according to claim 1 further comprising the step of adding a new event to the event dependency graph by creating a vertex with a globally unique identifier.

7. The method according to claim 6 further comprising the step of using the globally unique identifier to query the event dependency graph to establish happens-before relationships between vertices.

8. The method according to claim 1, wherein the order is a hard constraint that the two or more events must be ordered in a requested manner.

9. The method according to claim 8, wherein the order is aborted when the two or more events cannot be ordered in the requested manner.

10. The method according to claim 8, wherein the order is a soft preference that the two events be ordered in a requested sequence if permitted by the previously established happens-before relationships.

11. The method according to claim 8, wherein the events that have been executed to completion are excised from the event dependency graph, thereby maintaining a size for the event dependency graph that is proportional to the quantity of active events.

12. The method according to claim 1 further comprising the steps of:

replicating by the central control unit the event dependency graph to obtain a replicated event dependency graph; and

providing by the central control unit to each subsystem the replicated event dependency graph.

13. A method of operation for coordinating distributed transactions on top of a sharded, distributed data store in a network, wherein the network comprises a plurality of servers and a plurality of clients, comprising the steps of:

selecting by a client one or more keys to obtain selected keys, wherein the selected keys deterministically determine a chain for each transaction of a plurality of transactions;

mapping by the client each selected key using a key-value store;

processing by the client each transaction through its corresponding chain through a forward pass and a backward pass;

checking each transaction of the plurality with one or more concurrent transactions;

applying by each server of the plurality of servers write keys for which the server is mapped to the key-value store;

assigning an order to each transaction of the plurality of transactions; and

executing each transaction of the plurality of transactions.

14. The method according to claim 13, wherein the forward pass includes the steps of:

determining overlapping transactions;

establishing happens-before relationships; and

validating previous reads.

15. The method according to claim 13, wherein the backward pass includes one step selected from the group of:

aborting the transaction; and

committing the transaction.

16. The method according to claim 13, wherein the one or more concurrent transactions operate on one or more keys separate from the plurality of keys of the transaction and require no consideration.

17. The method according to claim 13, wherein the one or more concurrent transactions operate on one or more keys that are the same as the plurality of keys of the transaction.

18. The method according to claim 17, wherein the one or more concurrent transactions are compatible transactions and are prepared concurrently with each transaction of the plurality of transactions and forwarded to a server in the chain.

19. The method according to claim 17, wherein the one or more concurrent transactions are conflicting transactions and are aborted.

20. The method according to claim 13, wherein the processing step further comprises the step of capturing all dependency information as each transaction of the plurality of transactions traverses the chain.