Cluster configuration backup and recovery

• US 8,484,163 B1
• Filed: 12/16/2010
• Issued: 07/09/2013
• Est. Priority Date: 12/16/2010
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Yucel, Sakir, Kownacki, Ronald William, Vijayarangan, Srinivasan
• Primary Examiner(s): Coby, Frantz
• Application Number:
  US12/970,523
• Time in Patent Office: 936 Days
• Field of Search: 707640-641, 707/650, 707/652, 707/653, 707674-679, 707685-686, 709/201, 709/220, 370/216
• US Class Current: 707/640
• CPC Class Codes: G06F 11/1415 at system level
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases

Abstract

A cluster network environment may comprise a plurality of nodes configured to perform various tasks, such as data I/O, storage, backup, etc. It may be advantageous to improve disaster recovery and reliability by creating backups that may be used to restore the cluster network environment and/or nodes therein. Accordingly, one or more techniques and/or systems are disclosed for restoring a cluster network environment and/or nodes therein. In particular, cluster configuration data, which may be stored in a healthy node or at a remote source, may be used to restore a node affected by an integrity loss. If the integrity loss affects the cluster network environment (e.g., a majority of healthy nodes are lost), then a new cluster network environment may be created from a healthy node or a restored healthy node. The creation of the new cluster network environment may be based at least in part on cluster configuration data.

Claims

1. A method for restoring a node of a cluster network environment, comprising:

receiving notification of an integrity loss associated with a node within a cluster network environment;

restoring a local configuration to the node to establish a restored node;

restoring a cluster configuration to the node to establish the restored node, the cluster configuration corresponding to at least one of cluster services, data volume locations, data block locations, user IDs, file access configuration, virtual servers, database configurations, cluster rules, or cluster management data, the restoring a cluster configuration comprising;

synchronizing the node with cluster operating data of the cluster network environment to restore the cluster configuration to the node, the synchronizing comprising;

removing one or more data replication service directories on the node; and

repopulating the one or more data replication service directories from a data replication service within the cluster network environment, the one or more data replication service directories corresponding to cluster configuration data; and

introducing the restored node with the cluster network environment based upon the cluster configuration of the restored node.

2. The method of claim 1, the restoring a local configuration comprising specifying one or more environmental variables on the node comprising at least one of a name, IP address, operating system parameter, local data, state information, or local identity for the node.

3. The method of claim 1, the restoring a local configuration comprising:

restoring the local configuration using node backup data associated with the node.

4. The method of claim 1, the restoring a cluster configuration comprising:

restoring the cluster configuration using cluster backup data from a healthy node within the cluster network environment.

5. The method of claim 1, the restoring a cluster configuration comprising:

restoring the cluster configuration using cluster backup data from a remote source using a URL.

6. The method of claim 1, the integrity loss corresponding to a loss of a root volume.

7. The method of claim 1, the integrity loss corresponding to a loss of data associated with the data replication service.

8. The method of claim 1, the restoring a cluster configuration to the node comprising:

specifying cluster specific metadata within the node, the cluster specific metadata specifying cluster environment variables that a cluster application consumes when operating within the cluster network environment.

9. The method of claim 1, comprising:

isolating the node from the cluster network environment before restoring the local configuration and restoring the cluster configuration.

10. A method for restoring a cluster network environment, comprising:

determining an integrity loss of a cluster network environment;

creating a new cluster network environment based upon a healthy node within the cluster network environment; and

for respective nodes affected by the integrity loss;

restoring a local configuration to an affected node to establish a restored node;

restoring a cluster configuration to the affected node to establish the restored node, the restoring a cluster configuration comprising;

synchronizing the affected node with cluster operating data of the cluster network environment to restore the cluster configuration to the affected node, the synchronizing comprising;

removing one or more data replication service directories on the affected node; and

repopulating the one or more data replication service directories from a data replication service within the cluster network environment, the one or more data replication service directories corresponding to cluster configuration data; and

introducing the restored node to the new cluster network environment.

11. The method of claim 10, the cluster configuration corresponding to at least one of:

cluster services;

data volume locations;

data block locations;

user IDs;

file access configuration;

virtual servers;

database configurations;

cluster rules;

or

cluster management data.

12. The method of claim 10, the determining an integrity loss comprising:

determining a quorum of healthy nodes does not exist within the cluster network environment.

13. A method for restoring a cluster network environment, comprising:

determining an integrity loss of a cluster network environment;

restoring a node affected by the integrity loss to generate a healthy node, the restoring comprising;

restoring a local configuration to the node to establish the healthy node; and

restoring a cluster configuration to the node, to establish the healthy node, using cluster backup data, the restoring a cluster configuration comprising invoking a data replication service to perform a sync command to repopulate one or more data replication service directories on the node; and

creating a new cluster network environment based upon the healthy node; and

for respective nodes affected by the integrity loss;

restoring a local configuration to an affected node to establish a restored node;

restoring a cluster configuration to the affected node, to establish the restored node, using the cluster configuration from the healthy node; and

introducing the restored node to the new cluster network environment.

14. The method of claim 13, comprising:

introducing nodes unaffected by the integrity loss to the new cluster network environment using the cluster configuration from the healthy node.

15. A system for restoring nodes of a cluster network environment, comprising:

an assessment component configured to;

determine whether an integrity loss affected a node within a cluster network environment; and

a recovery component configured to;

restore a local configuration to the node to establish a restored node;

restore a cluster configuration to the node to establish the restored node, comprising invoking a data replication service to perform a sync command to repopulate one or more data replication service directories on the node; and

introduce the restored node to the cluster network environment based upon the cluster configuration of the restored node.

16. The system of claim 15, comprising:

a backup component configured to;

generate node backup data of a node; and

generate cluster backup data of the cluster network environment.

17. The system of claim 16, the backup component configured to:

store at least one of the node backup data or the cluster backup data at a remote source outside of the cluster network environment.

18. The system of claim 16, the backup component configured to:

store at least one of the node backup data or the cluster backup data at one or more nodes within the cluster network environment.

19. The system of claim 16, the backup component configured to:

aggregate a plurality of node backup data into a tarball file using a file replication service.

20. The system of claim 19, the backup component configured to:

aggregate the cluster backup data into the tarball file to generate a mega tarball file.

21. The system of claim 15, the assessment component configured to:

determine whether the integrity loss affected the cluster network environment.

22. The system of claim 21, the recovery component configured to:

if the integrity loss affected the cluster network environment, then;

create a new cluster network environment based upon a healthy node.

23. The system of claim 21, the recovery component configured to:

if the integrity loss affected the cluster network environment, then;

restore a node affected by the integrity loss to generate a healthy node; and

create a new cluster network environment based upon the healthy node.