Cluster families for cluster selection and cooperative replication

• US 8,812,799 B2
• Filed: 12/11/2009
• Issued: 08/19/2014
• Est. Priority Date: 12/11/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bish, Thomas William, Nohta, Takeshi, Swingler, Joseph M., Twito, Rufus-John Yeisuke
• Primary Examiner(s): NGUYEN, THAN VINH
• Application Number:
  US12/635,702
• Publication Number:
  US 20110145497A1
• Time in Patent Office: 1,712 Days
• Field of Search: 711/141, 711/161, 711/162
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 12/0802 Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
• CPC Class Codes: G06F 12/0815 Cache consistency protocols
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 2212/1016 Performance improvement
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 3/0686 Libraries, e.g. tape libraries, jukebox
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

An apparatus, system, and method are disclosed to create cluster families for cluster selection and cooperative replication. The clusters are grouped into family members of a cluster family base on their relationships and roles. Members of the cluster family determine which family member is in the best position to obtain replicated information and become cumulatively consistent within their cluster family. Once the cluster family becomes cumulatively consistent, the data is shared within the cluster family so that all copies within the cluster family are consistent.

Claims

1. A method for cooperative replication of multiple clusters, the method comprising:

arranging at least one subset of the multiple clusters into family members of a cluster family;

negotiating between cluster family members to determine which cluster family member is in the best position to obtain at least one outside data object from at least one cluster outside of the family;

selecting one family member of the cluster family to obtain the outside data object;

sharing the outside data objects among the cluster family so that each cluster within the cluster family is consistent with respect to outside data objects; and

achieving cumulative consistency within the cluster family of a plurality of outside data objects prior to sharing outside data objects among clusters within the cluster family.

2. The method of claim 1, further comprises creating relationship among cluster families based on at least one of a cluster relationship and role.

3. The method of claim 1, wherein the family member with a consistent source maintains a volume in cache in order to make the volume readily available to other family members for peer replication.

4. The method of claim 3, further comprising causing the family member maintaining the volume in cache for the other family members to relieve an outside cluster from maintaining a copy in an outside cluster cache.

5. The method of claim 1, wherein each family member in the family replicates 1/Nth volumes, wherein N is equal to the number of family members in the cluster family.

6. The method of claim 1, further comprises serializing all replication of any one volume brought into the cluster family.

7. The method of claim 1, wherein a first family member to replicate 1/Nth volumes informs the outside cluster that the first family member will maintain the volumes for the cluster family members and relieves the outside cluster from maintaining the volumes in an outside cluster cache, wherein N is equal to the number of family members in the cluster family.

8. The method of claim 1, further comprises a plurality of cluster families wherein at least one family member from each family replicates a volume.

9. The method of claim 1, further comprises a domain comprising a plurality of clusters wherein the clusters cooperate to make sure at least one family member from each family replicates a volume and the remaining clusters surrender replication requirements.

10. A method for cooperative replication of multiple clusters, the method comprising:

arranging at least one subset of the multiple clusters into family members of a cluster family;

negotiating between cluster family members to determine which cluster family member is in the best position to obtain at least one outside data object from at least one cluster outside of the family;

selecting one family member of the cluster family to obtain the outside data object;

sharing the outside data objects among the cluster family so that each cluster within the cluster family is consistent with respect to outside data objects; and

favoring one of the family members in the cluster family for mount processing over clusters outside of the cluster family.

11. The method of claim 10, further comprises creating a relationship among cluster families based on at least one of a cluster relationship and role.

12. The method of claim 10, wherein the family member with a consistent source maintains a volume in cache in order to make the volume readily available to other family members for peer replication.

13. The method of claim 10, wherein each family members in the family replicates 1/Nth volumes, wherein N is equal to the number of family members in the cluster family.

14. The method of claim 10, further comprises serializing all replication of any one volume brought into the cluster family.

15. The method of claim 10, wherein a first family member to replicate 1/Nth volumes informs the outside cluster that the first family member will maintain the volumes for the cluster family members and relieves the outside cluster from maintaining the volumes in the outside cluster cache, wherein N is equal to the number of family members in the cluster family.

16. The method of claim 10, further comprises a domain comprising a plurality of clusters wherein the clusters cooperate to make sure at least one family member from each family replicates a volume and the remaining clusters surrender replication requirements.

17. A system for cooperative replication of multiple clusters, the system comprising:

a network;

a plurality of sites in communication over the network, each site comprising at least one host and a storage system comprising a plurality of clusters, each cluster comprising at least one tape drive configured to access volumes stored on magnetic tape, at least one tape volume cache, and a cluster manager configured to execute computer readable programs using a processor and a memory, wherein the computer readable programs comprising;

a creation module configured to setup and arrange a group of clusters into family members of a cluster family; and

a cooperative replication module configured to select a family member to cooperatively replicate data into the cluster family and achieve cumulative family consistency within the cluster family of a plurality of outside data objects prior to sharing outside data objects among clusters within the cluster family.

18. The system of claim 17, wherein the cooperative replication module is further configured to share replication among each family member.

19. The system of claim 18, wherein each family member in the family replicates 1/Nth volumes, wherein N is equal to the number of family members in the cluster family.

20. The system of claim 17, wherein the creation module is further configured to create cluster families based on at least one of a relationship and role between clusters.

21. An apparatus for cooperative replication of multiple clusters, the apparatus comprising:

a creation module comprising a computer readable program executing on a processor and configured to create a cluster family from a plurality of clusters, wherein the clusters are in communication over a network and each cluster comprises a cache; and

a cooperative replication module comprising a computer readable program executing on a processor and configured to cooperatively replicate data into the cluster family, wherein the cooperative replication module is further configured to handle persistent replication source awareness by deferring replication, wherein persistent replication source awareness includes instructing a cluster family of the one or more cluster families having a consistent source to maintain the consistent source in cache in order to make the consistent source available to members of the cluster family for peer replication.

22. The apparatus of claim 21, wherein the creation module is further configured to setup and arrange a group of clusters into family members according to at least one of a cluster relationship and role.

23. The apparatus of claim 21, wherein each family members in the cluster family replicates 1/Nth volumes, wherein N is equal to the number of family members in the cluster family.

24. The apparatus of claim 21, wherein the cooperative replication module is further configured to select a family member of the cluster family to obtain outside data objects and be the source cluster for all family members within the cluster family and have the source cluster responsible for achieving cumulative consistency within the cluster family prior to sharing the outside data objects.

25. The apparatus of claim 21, further comprising:

a relationship module comprising a computer readable program executing on a processor and configured to maintain factors defining roles, rules, and relationships between cluster families and family members.

26. An apparatus for cooperative replication of multiple clusters, the apparatus comprising:

a creation module comprising a computer readable program executing on a processor and configured to create a cluster family from a plurality of clusters, wherein the clusters are in communication over a network and each cluster comprises a cache;

a cooperative replication module comprising a computer readable program executing on a processor and configured to cooperatively replicate data into the cluster family; and

a mount processing module comprising a computer readable program executing on a processor and configured to favor family members within the cluster family over other cluster families for production purposes.

27. The apparatus of claim 26, wherein the creation module is further configured to setup and arrange a group of clusters into family members according to at least one of a cluster relationship and role.

28. The apparatus of claim 26, wherein each family members in the cluster family replicates 1/Nth volumes, wherein N is equal to the number of family members in the cluster family.

29. The apparatus of claim 26, wherein the cooperative replication module is further configured to select a family member of the cluster family to obtain outside data objects and be the source cluster for all family members within the cluster family and have the source cluster responsible for achieving cumulative consistency within the cluster family prior to sharing the outside data objects.

30. The apparatus of claim 26, further comprising:

a relationship module comprising a computer readable program executing on a processor and configured to maintain factors defining roles, rules, and relationships between cluster families and family members.

31. A computer program product comprising a computer useable medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:

group a plurality of clusters into family members of a cluster family;

determine which family member is in the best position to obtain outside data objects from a source;

select one or more family members of the cluster family to obtain the outside data objects;

cooperatively replicate the outside data objects into the cluster family; and

share the outside data objects within the cluster family so that all copies of the outside data objects within the cluster family are consistent,

wherein the computer readable program further causes the computer to achieve cumulative consistency within the cluster family of a plurality of outside data objects prior to sharing outside data objects among clusters within the cluster family.

32. The computer program product of claim 31, wherein the computer readable program further causes the computer to serialize replication of all volumes brought into the cluster family.

33. The computer program product of claim 31, wherein the computer readable program further causes the computer to select a family member with a consistent source, and to maintain a volume in cache in order to make the volume readily available to other family members for peer replication.

34. The computer program product of claim 31, wherein the computer readable program further causes the computer to instruct each family members in the family to replicate 1/Nth volumes, wherein N is equal to the number of family members in the cluster family.