Method and apparatus for sequencing transactions globally in a distributed database cluster

• US 20090106323A1
• Filed: 05/09/2008
• Published: 04/23/2009
• Est. Priority Date: 09/09/2005
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Open Invention Network LLC
• Inventors: Yu, Xiong, Wong, Frankie, Wang, Elaine
• Granted Patent: US 9,785,691 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

A system and method for receiving and tracking a plurality of transactions and distributing the transactions to at least two replication queues over a network. The system and method comprise a global queue for storing a number of the received transactions in a first predetermined order. The system and method also comprise a sequencer coupled to the global queue for creating a copy of each of the transactions for each of said at least two replication queues and for distributing in a second predetermined order each said copy to each of said at least two replication queues respectively, said copy containing one or more of the received transactions.

Claims

1. A system for receiving and tracking a plurality of transactions and distributing the transactions to at least two replication queues over a network, the system comprising:

a global queue for storing a number of the received transactions in a first predetermined order; and

a sequencer coupled to the global queue for creating a copy of each of the transactions for each of said at least two replication queues and for distributing in a second predetermined order each said copy to each of said at least two replication queues respectively, said copy containing one or more of the received transactions.

2. The system according to claim 1, wherein the predetermined orders are selected from the group comprising:

the first predetermined order is the same as the second predetermined order; and

the first predetermined order is different from the second predetermined order.

3. The system according to claim 2 in which the sequencer distributes each said copy at a predetermined time interval.

4. The system according to claim 2 in which the sequencer distributes each said copy when the number of the transactions within the global queue exceeds a predetermined value.

5. The system according to claim 2 in which the sequencer distributes each said copy upon the earlier of:

a predetermined time interval; and

the number of the transactions within the global queue exceeds a predetermined value.

6. The system according to claim 5 in which each of the transactions comprises an update transaction and a unique transaction id assigned by the sequencer.

7. The system according to claim 6 further comprising a global disk queue in communication with the global queue for receiving and storing the transactions when the global queue is above a global threshold.

8. The system according to claim 7 wherein each of said at least two replication queues have a corresponding replication disk queue for receiving and storing the transactions from the global queue when the corresponding replication queue is above a replication threshold.

9. The system according to claim 8 in which the global queue receives the transactions from the global disk queue and other than receives the transactions from said at least one application server when the global disk queue is other than empty.

10. The system according to claim 5 further comprising an indoubt transaction queue in communication with the sequencer for storing the transactions identified as having unknown status by a database server during system failures.

11. The system according to claim 6 wherein the update transaction comprises at least one of a read, insert, update or delete request for at least one database in communication with at least one of said at least two replication queues.

12. The system according to claim 6 further comprising a resent transaction queue for storing the transactions when the transactions repeated the request for the transaction id.

13. The system according to claim 2, wherein the global queue is configured for receipt of the received transactions from a network entity selected from the group comprising:

an application; and

an application server.

14. The system according to claim 2, wherein the global queue is a searchable first-in first-out pipe.

15. The system according to claim 14 further comprising the sequencer configured for assuring the order of transactions in the global queue remain consistent with their execution order at a database server coupled to at least one of the replication queues.

16. The system according to claim 14, wherein the global disk queue is configured for storing an indexed and randomly accessible data set.

17. The system according to claim 2, wherein the global queue and sequencer are hosted on a network entity selected from the group comprising:

a central control server and a peer-to-peer node.

18. A system for receiving a plurality of transactions from at least one application server, distributing the transactions to at least two replication queues and applying the transactions to a plurality of databases comprising:

a director coupled to each of said at least one application server for capturing a plurality of database calls therefrom as the plurality of transactions; and

a controller for receiving each of the plurality of transactions, the controller configured for storing the transactions within a global queue in a predetermined order, for generating a copy of each said transaction for each of said at least two replication queues, and for transmitting in the predetermined order each said copy to each of said at least two replication queues respectively.

19. The system according to claim 18 further comprising at least two replication servers including said at least two replication queues wherein each of said at least two replication servers is coupled to each of the databases;

wherein the director routes each of the transactions to one or more of the databases relative to the workload and transaction throughput.

20. The system according to claim 19 further comprising a backup controller for receiving the transactions from said at least one application server upon failure of the controller, the backup controller including a backup global queue wherein the backup global queue is substantially synchronized with the controller and the backup global queue is a copy of the global queue.

21. A method for receiving and tracking a plurality of transactions and distributing the transactions to at least two replication queues over a network, the method comprising:

storing a number of the received transactions in a first predetermined order in a global queue;

creating a copy of each of the transactions for each of said at least two replication queues; and

distributing in a second predetermined order each said copy to each of said at least two replication queues respectively, said copy containing one or more of the received transactions.

22. The method according to claim 21 wherein the step of distributing each said copy occurs at a predetermined time interval.

23. The method according to claim 21 wherein the step of distributing each said copy occurs when the number of the transactions within the global queue exceeds a predetermined number.

24. The method according to claim 21 wherein the step of distributing each said copy occurs upon the earlier of:

a predetermined time interval; and

the number of the transactions within the global queue exceeds a predetermined number.

25. The method according to claim 24, wherein each of the transactions comprises an update transaction and a unique transaction id assigned by the sequencer.

26. The method according to claim 24 further comprising the step of receiving and storing the transactions within a global disk queue when the global queue storage capacity reaches a global threshold.

27. The method according to claim 21 further comprising the steps of:

determining whether the global disk queue is other than empty; and

receiving the transaction from the global disk queue rather than receiving the transactions from said at least one application server when the global disk queue is other than empty.

28. The method according to claim 21 further comprising the step of storing the transactions within an indoubt transaction queue during system failures.

29. The method according to claim 25 wherein the update transaction comprises at least one of a read, insert, update or delete request for at least one database in communication with at least one of said at least two replication queues.

30. The method according to claim 24 further comprising the steps of:

determining when at least one of said at least two replication queues are above a replication threshold, each of said at least two replication queues having a corresponding replication disk queue;

storing a number of the transactions within said corresponding replication disk queue based upon the determination; and

sending an alert to notify when said at least two replication queues and said corresponding replication disk queue capacity reach a preselected threshold.

31. The method according to claim 30 further comprising the step of:

redirecting the transactions to at least one of said at least two replication queues being below said preselected threshold, based on receiving the alert.

32. A system for receiving and tracking a plurality of transactions and distributing the transactions to at least two replication queues over a network, the system comprising:

means for storing a number of the received transactions in a first predetermined order; and

means for creating a copy of each of the transactions for each of said at least two replication queues and for distributing in a second predetermined order each said copy to each of said at least two replication queues respectively, said copy containing one or more of the received transactions.