Dynamic membership management in a distributed system

• US 20060235889A1
• Filed: 04/13/2006
• Published: 10/19/2006
• Est. Priority Date: 04/13/2005
• Status: Active Grant

• Current Assignee: Progress Software Corporation
• Original Assignee: Progress Software Corporation
• Inventors: Hu, Carl, Rousseau, Benjamin A., Tompkins, Frederick A. Jr.
• Granted Patent: US 7,739,235 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1425 by reconfiguration of node membership
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 9/466 Transaction processing

Abstract

Transactional database replication techniques are disclosed that do not require altering of the database management system implementation. A replicator module includes a dynamic membership manager, which is configured to manage the persistent membership of a coordination group. The persistent membership can be, for example, a durable set of sites that can replicate changes amongst themselves. Changes from sites not in the persistent membership are discarded. The set of recovered members is a subset of the persistent membership. The persistent membership changes incrementally by either removing or adding members. Failed members may still be part of the persistent membership. The dynamic membership manager module manages the modification of the persistent membership, initialization of replicas, and propagation of membership information.

Claims

1. A method for dynamically managing persistent membership of a coordination group in a distributed environment, the coordination group including a number of sites included in a persistent membership, with one or more of the sites including a database and a registrar, the method comprising:

sending, by an entrant seeking to become a member of the persisted membership, a Membership Request;

receiving, by a registrar that is a member of the persistent membership, the Membership Request;

modifying, by the registrar, the persistent membership for the entrant;

disconnecting, by the registrar, from the coordination group; and

initializing a database for the entrant.

2. The method of claim 1 wherein modifying the persistent membership for the entrant comprises:

sending, by the registrar, a Membership Change message to other sites in the coordination group;

receiving, by the registrar, the Membership Change message in total persisted order; and

proceeding, by the registrar, with initializing the entrant database in response to the persistent membership being changed.

3. The method of claim 1 further comprising:

reconnecting, by the registrar, to the coordination group after initialization of the entrant database; and

connecting, by the entrant, to the coordination group after initialization of the entrant database.

4. The method of claim 1 wherein should connection between the registrar and the entrant be severed, the entrant can reconnect to any available registrar via a group communication system providing an extended virtual synchrony service to the coordination group, and recover.

5. The method of claim 1 wherein initializing the entrant database is carried out using backup/historical initialization, and the Membership Request includes an action ID of a last action that was applied to the entrant'"'"'s database, the action ID uniquely identifying the persistent membership.

6. The method of claim 5 wherein in response to the registrar not containing historical update information for this action ID, the method further comprises refusing, by the registrar, to proceed.

7. The method of claim 1 further comprising:

acquiring, by the registrar, a distributed lock on the coordination group and entrant name;

examining, by the registrar upon receiving the distributed lock, the persistent membership to ensure that no other site utilizes the entrant'"'"'s name; and

refusing, by the registrar, to proceed with initializing the database in response to the entrant'"'"'s name being already in use.

8. The method of claim 7 wherein acquiring the distributed lock is carried out by a fault tolerant distributed lock manager associated with the registrar, the method further comprising:

aborting, by the fault tolerant distributed lock manager, any transactions in process;

releasing, by the fault tolerant distributed lock manager, any locks held by the registrar; and

disconnecting, by the registrar, from a group communication system providing an extended virtual synchrony service to the coordination group thereby allowing the registrar to provide a consistent snapshot to the entrant.

9. The method of claim 1 wherein initializing the entrant database is carried out using snapshot initialization, and the entrant'"'"'s database is initialized with contents of the registrar'"'"'s database.

10. The method of claim 1 wherein initializing the entrant database is carried out using verify/update initialization, the initializing further comprising:

comparing entrant and registrar database contents;

resolving discrepancies; and

in response to the entrant including new data, creating, by the entrant, actions corresponding to the new data and indicating total persisted order of those actions is not yet known.

11. The method of claim 1 wherein initializing the entrant database is carried out using backup/historical initialization, the initializing further comprising:

disconnecting, by the registrar, from a group communication system servicing the coordination group thereby inhibiting further update actions;

sending, by the registrar, all Green actions after last applied action at the entrant to the entrant, wherein Green actions are those whose total persisted order is known and confirmed; and

applying, by the entrant, those sent Green actions.

12. The method of claim 1 further comprising:

inhibiting, by a distributed lock manager, transactions during database initialization.

13. The method of claim 1 wherein the registrar can only service an entrant if the registrar has attained quorum based on a quorum requirement, and the quorum requirement can be loosened to allow more quorums by not counting previous quorum members that have left or have been evicted.

14. The method of claim 1 wherein members are allowed to become registrars only if they have participated in at least one quorum.

15. The method of claim 1 further comprising:

using a flow control protocol so as not to exceed any network data flow limits.

16. A method for dynamically managing persistent membership of a coordination group in a distributed environment, the coordination group including a number of sites included in a persistent membership, with one or more of the sites including a database and a registrar, the method comprising:

sending, by an entrant seeking to become a member of the persisted membership, a Membership Request;

receiving, by a registrar that is a member of the persistent membership, the Membership Request;

sending, by the registrar, a Membership Change message to other sites in the coordination group so that the Membership Change message can be persisted and totally ordered;

receiving, by the registrar, the Membership Change message in total persisted order from the coordination group;

disconnecting, by the registrar, from the coordination group;

initializing a database for the entrant;

reconnecting, by the registrar, to the coordination group after initialization of the entrant database; and

connecting, by the entrant, to the coordination group after initialization of the entrant database.

17. A machine-readable medium encoded with instructions, that when executed by one or more processors, cause the processor to carry out a process for dynamically managing persistent membership of a coordination group in a distributed environment, the coordination group including a number of sites included in a persistent membership, with one or more of the sites including a database and a registrar, the process comprising:

sending, by an entrant seeking to become a member of the persisted membership, a Membership Request;

receiving, by a registrar that is a member of the persistent membership, the Membership Request;

modifying, by the registrar, the persistent membership for the entrant;

disconnecting, by the registrar, from the coordination group;

initializing a database for the entrant;

reconnecting, by the registrar, to the coordination group after initialization of the entrant database; and

connecting, by the entrant, to the coordination group after initialization of the entrant database.

18. The machine-readable medium of claim 17 wherein modifying the persistent membership for the entrant comprises:

sending, by the registrar, a Membership Change message to other sites in the coordination group;

receiving, by the registrar, the Membership Change message in total persisted order; and

proceeding, by the registrar, with initializing the entrant database in response to the persistent membership being changed.

19. The machine-readable medium of claim 17, the process further comprising:

inhibiting, by a distributed lock manager, transactions during database initialization.

20. The machine-readable medium of claim 17 wherein the registrar can only service an entrant if the registrar has attained quorum based on a quorum requirement, and the quorum requirement can be loosened to allow more quorums by not counting previous quorum members that have left or have been evicted.