Determining active membership among storage systems synchronously replicating a dataset

• US 10,585,733 B1
• Filed: 11/01/2017
• Issued: 03/10/2020
• Est. Priority Date: 03/10/2017
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Colgrove, John, Grunwald, David, Karr, Ronald, Ramanarayanan, Ganesh
• Primary Examiner(s): Bonzo, Bryce P
• Assistant Examiner(s): Arcos, Jeison C
• Application Number:
  US15/800,760
• Time in Patent Office: 860 Days
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/2053 where persistent mass storage functionality or persistent mass storage control functionality is redundant error detection or correction in information storage based on relative movement between record carrier and transducer G11B20/18
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2076 Synchronous techniques
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 12/0684 with feedback, e.g. presence or absence of unit detected by addressing, overflow detection
• CPC Class Codes: G06F 12/1072 Decentralised address translation, e.g. in distributed shared memory systems
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0616 in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
• CPC Class Codes: G06F 3/0619 : in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0632 : by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/065 : Replication mechanisms
• CPC Class Codes: G06F 3/0659 : Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 : Plurality of storage devices
• CPC Class Codes: G06F 3/0688 : Non-volatile semiconductor memory arrays
• CPC Class Codes: G06F 3/0689 : Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G06F 9/44505 : Configuring for program initiating, e.g. using registry, configuration files
• CPC Class Codes: H04L 45/12 : Shortest path evaluation
• CPC Class Codes: H04L 45/38 : Flow based routing
• CPC Class Codes: H04L 47/125 : by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 67/1095 : Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Determining active membership among storage systems synchronously replicating a dataset, where determining active membership includes: detecting, by a particular storage system of the storage systems, a communication fault with one or more of the storage systems, where the particular storage system is configured to request mediation from a mediation target; determining that at least one of the one or more storage systems is configured to request mediation from the mediation target responsive to the communication fault; and responsive to determining that one or more of the storage systems is configured to request mediation from the mediation target, determining whether to request mediation from the mediation target.

Claims

1. A method of determining active membership among storage systems synchronously replicating a dataset, the method comprising:

detecting, by a particular storage system of the storage systems, a communication fault with one or more of the storage systems, wherein the particular storage system is configured to request mediation from a mediation target;

determining that at least one of the one or more storage systems is configured to request mediation from the mediation target responsive to the communication fault; and

responsive to determining that one or more of the storage systems is configured to request mediation from the mediation target, determining whether to request mediation from the mediation target.

2. The method of claim 1, wherein determining whether to request mediation from the mediation target comprises determining to request mediation.

3. The method of claim 1, wherein determining whether to request mediation from the mediation target comprises determining to not request mediation.

4. The method of claim 1, wherein each storage system of the storage systems corresponds to zero or more votes within a quorum protocol that determines whether a first set of one or more storage systems may detach a second set of one or more storage systems.

5. The method of claim 4, wherein a respective quantity of votes corresponding to a respective storage system is dependent upon storage system characteristics.

6. The method of claim 4, further comprising:

determining, by the first set of one or more storage systems, that there is consistent communication among storage systems of the first set of one or more storage systems;

determining, by the first set of one or more storage systems, lack of communication with storage systems among the second set of one or more storage systems, wherein the first set of one or more storage systems is unable to form a quorum; and

determining, by the first set of one or more storage systems, that the second set of one or more storage systems is unable to form a quorum.

7. The method of claim 6, further comprising:

determining that the first set of one or more storage systems is unable to form a quorum in dependence upon the first set of one or more storage systems including exactly half of the votes for the storage systems synchronously replicating the dataset.

8. The method of claim 6, further comprising:

determining that the second set of one or more storage systems is unable to form a quorum in dependence upon the second set of one or more storage systems including half of the votes for the storage systems synchronously replicating the dataset.

9. The method of claim 6, wherein determining whether to request mediation from the mediation target is dependent upon the first set of one or more storage systems being unable to obtain a quorum and no other set of the one or more storage systems being able to obtain a quorum.

10. A method for determining active membership among storage systems synchronously replicating a dataset, the method comprising:

tracking one or more performance metrics of the storage systems;

detecting, by a first storage system of the storage systems, a communication fault with a second storage system of the storage systems;

generating a mediation analysis indicating that winning mediation would enable the storage systems in communication with the first storage system to continue servicing I/O requests directed to the dataset such that one or more performance criteria for the one or more performance metrics would be better satisfied in comparison to the second storage system continuing to service the I/O requests; and

requesting, in dependence upon the mediation analysis, mediation from a mediation target.

11. The method of claim 10, wherein each storage system of the storage systems corresponds to zero or more votes within a quorum protocol that determines whether a first set of one or more storage systems may detach a second set of one or more storage systems.

12. An apparatus for determining active membership among storage systems synchronously replicating a dataset, the apparatus comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:

detecting, by a particular storage system of the storage systems, a communication fault with one or more of the storage systems, wherein the particular storage system is configured to request mediation from a mediation target;

determining that at least one of the one or more storage systems is configured to request mediation from the mediation target responsive to the communication fault; and

responsive to determining that one or more of the storage systems is configured to request mediation from the mediation target, determining whether to request mediation from the mediation target.

13. The apparatus of claim 12, wherein determining whether to request mediation from the mediation target comprises determining to request mediation.

14. The apparatus of claim 12, wherein determining whether to request mediation from the mediation target comprises determining to not request mediation.

15. The apparatus of claim 12, wherein each storage system of the storage systems corresponds to zero or more votes within a quorum protocol that determines whether a first set of one or more storage systems may detach a second set of one or more storage systems.

16. The apparatus of claim 15, wherein a respective quantity of votes corresponding to a respective storage system is dependent upon storage system characteristics.

17. The apparatus of claim 15, further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:

determining, by the first set of one or more storage systems, that there is consistent communication among storage systems of the first set of one or more storage systems;

determining, by the first set of one or more storage systems, lack of communication with storage systems among the second set of one or more storage systems, wherein the first set of one or more storage systems is unable to form a quorum; and

determining, by the first set of one or more storage systems, that the second set of one or more storage systems is unable to form a quorum.

18. The apparatus of claim 17, further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:

determining that the first set of one or more storage systems is unable to form a quorum in dependence upon the first set of one or more storage systems including exactly half of the votes for the storage systems synchronously replicating the dataset.

19. The apparatus of claim 17, further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:

determining that the second set of one or more storage systems is unable to form a quorum in dependence upon the second set of one or more storage systems including half of the votes for the storage systems synchronously replicating the dataset.

20. The apparatus of claim 17, wherein determining whether to request mediation from the mediation target is dependent upon the first set of one or more storage systems being unable to obtain a quorum and no other set of the one or more storage systems being able to obtain a quorum.