MANAGING CLUSTER SPLIT-BRAIN IN DATACENTER SERVICE SITE FAILOVER

• US 20090276657A1
• Filed: 05/05/2008
• Published: 11/05/2009
• Est. Priority Date: 05/05/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Benfield, Rebecca, Thiel, Gregory, Wetmore, Alexander R., Kol, Ayla
• Granted Patent: US 8,001,413 B2
• Time in Patent Office: Days
• US Class Current: 714/4
• CPC Class Codes: G06F 11/1482 by means of middleware or OS functionality
• CPC Class Codes: G06F 11/2028 eliminating a faulty processor or activating a spare
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component

Abstract

A central controlling service for datacenter activation/deactivation control in a cluster deployment to assist in preventing a split-brain scenario. The central controlling service provides a central point of control in the datacenter for application servers to periodically query as to whether to go offline, online, or normal. Redundancy of the central service facilitates detection of datacenter failure by the redundant services interacting to resolve the state of control information. This control information is then used to answer the server queries. On startup from a datacenter failure, a single instance of the central service queries other redundant instance(s) to determine if the single instance is starting up from a datacenter-wide failure or from operations other than total datacenter failure. If the failure is datacenter-wide, a central service protocol assists in resolving to the single service keeping the associated datacenter servers offline; otherwise, the server queries are answered to go online.

Claims

1. A computer-implemented system for managing activation and deactivation of servers or services in a datacenter, comprising:

a request component for receiving requests from servers of a datacenter for server control information; and

a response component for sending answers to the servers in response to the requests to control the servers to prevent a split-brain scenario between the datacenter and one or more other datacenters.

2. The system of claim 1, wherein the server control information relates to desired state for the servers being one of normal, offline, or online.

3. The system of claim 1, wherein the server control information applies to each of the datacenter servers or a subset of the datacenter servers.

4. The system of claim 1, wherein the servers of the datacenter are mailbox servers managed according to an active/passive redundancy model.

5. The system of claim 1, wherein the request component, the response component, and the control information are embodied as part of a coordination component of the active datacenter to process an instruction to maintain the servers in an offline state to prevent the split-brain scenario.

6. The system of claim 5, wherein the coordination component includes redundant instances of datacenter activation coordinators, the coordinators query each other on startup to determine desired state of the servers in the datacenter, and based on the state, block activation of the servers in the datacenter to prevent the split-brain scenario.

7. A computer-implemented system for managing redundant datacenters, comprising:

a passive coordination component for controlling passive message services of a passive datacenter; and

an active coordination component for controlling active message services of an active datacenter, the active coordination component configured to maintain the active message services in an offline state in response to a datacenter-wide failure of the active datacenter to prevent a split-brain situation with the passive datacenter.

8. The system of claim 7, wherein the active coordination component blocks activation of message services operating in an active/passive clustered model after a complete active datacenter failure.

9. The system of claim 7, wherein status of a directed transition associated with the service state transition is posted to the coordination component, the status includes resulting state of at least one of a service, a failure code, or a failure message.

10. The system of claim 7, wherein the message services of the active datacenter poll the coordination component at regular time intervals for updated desired state that indicates the message services are to go online.

11. The system of claim 7, wherein the passive coordination component is administered manually to bring the passive datacenter online.

12. The system of claim 7, wherein the passive coordination component and the active coordination component each include a set of redundant activation coordinators, and the redundant coordinators of a set intercommunicate to resolve state of control information for controlling associated messaging servers to an offline or online state.

13. A computer-implemented method of managing activation and deactivation of servers or services in a datacenter, comprising:

resolving control information at redundant activation coordinators of a datacenter due to failure of the datacenter;

receiving queries from application servers of the datacenter for service status based on the control information; and

sending answers to the queries from the activation coordinators that control the application servers to prevent a split-brain scenario between the datacenter and one or more passive datacenters.

14. The method of claim 13, further comprising changing state of the control information in control tables of the activation coordinators to activate or deactivate the application servers.

15. The method of claim 13, wherein the control information is resolved by an activation coordinator in startup mode, by querying one or more other activation coordinators of the datacenter for state information and determining the control information based on the state information.

16. The method of claim 15, wherein the control information is manipulated manually to activate or deactivate the application servers.

17. The method of claim 13, wherein the redundant activation coordinators share state upon startup, each coordinator receives the queries, processes the queries, and sends the answers to the application servers associated with the queries.

18. The method of claim 13, further comprising posting completion status information of the application servers to the redundant activation coordinators.

19. The method of claim 18, wherein the completion status information includes resulting state of the application servers, a failure code, and a failure message.

20. The method of claim 13, further comprising maintaining a passive datacenter in a passive mode by sending answers to queries from application servers of the passive datacenter that keep the application servers of the passive datacenter deactivated.