Methods and systems for implementing shared disk array management functions

• US 8,028,191 B2
• Filed: 05/07/2010
• Issued: 09/27/2011
• Est. Priority Date: 03/03/1999
• Status: Expired due to Fees

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Matthews, Michael B., Hagglund, Dale, Brown, William P., Unrau, Ron
• Primary Examiner(s): Riad; Amine
• Application Number:
  US12/775,743
• Publication Number:
  US 20100218039A1
• Time in Patent Office: 508 Days
• Field of Search: 714/7
• US Class Current: 714/6.12
• CPC Class Codes: G06F 11/2089 Redundant storage control functionality
• CPC Class Codes: G06F 11/2092 Techniques of failing over between control units
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/52 Program synchronisation; Mutual exclusion, e.g. by means of semaphores

Abstract

Multiple Array Management Functions (AMFs) are connected to multiple redundancy groups over a storage area network (SAN), such as a fiber-channel based SAN. The multiple AMFs share management responsibility of the redundancy groups, each of which typically includes multiple resources spread over multiple disks. The AMFs provide concurrent access to the redundancy groups for associated host systems. When a host requests an AMF to perform an operation on a resource, the AMF synchronizes with the other AMFs sharing control of the redundancy group that includes the resource to be operated on, so as to obtain access to the resource. While performing the operation, the AMF send replication data and state information associated with the resource such that if the AMF fails, any of the other AMFs are able to complete the operation and maintain data reliability and coherency.

Claims

1. A controller device for use as one of a set of caching controller devices connected by a network and sharing access to a redundancy group, comprising:

an array management function (AMF) providing access to the redundancy group; and

a communications port for coupling to the network,

wherein the AMF is a first AMF operative to communicate over the network with the redundancy group and with one or more second AMFs distributed among the caching controller devices and sharing access to the redundancy group, the second AMFs including one or more sharing partners and one or more replication partners, the sharing partners maintaining caches for satisfying read and write requests for data of the redundancy group, the replication partners capable of receiving write data from the first AMF and writing the replicated data to a physical storage device if the controller device fails, at least one of the replication partners also being one of the sharing partners,

and wherein the first AMF is further operative, in response to a write request from a host to write data to the redundancy group, to;

(a) send an invalidate request to one or more of the sharing partners, the invalidate request indicating that any copies of the data cached at the sharing partners are stale and are to be invalidated;

(b) safely store the data by a selected one of (i) in the event that write-through caching is employed, writing the data to a physical storage device, and (ii) in the event that write-back caching is employed, replicating the data to one or more of the replication partners; and

(c) return a write acknowledgment to the host only upon safely storing the data.

2. A controller device according to claim 1, wherein the invalidate request is broadcast to all the sharing partners.

3. A controller device according to claim 1, wherein the first AMF is further operative to maintain a record of whether the data is shared, and wherein the invalidate request is sent only if the record indicates that the data is shared.

4. A controller device according to claim 3, wherein the record further indicates which of the sharing partners is sharing the data, and wherein the invalidate request is sent to only those sharing partners indicated by the record to be sharing the data.

5. A controller device according to claim 3, wherein sending the invalidate request includes broadcasting the invalidate request to all the sharing partners.

6. A controller device according to claim 1, wherein at least one of the sharing partners is not included among the replication partners.

7. A controller device according to claim 1, wherein the first AMF is further operative to (i) receive replicated data of another write request from one of the replication partners, and (ii) complete the other write request by writing the accompanying data to the physical storage device upon failure of a caching controller device on which the one replication partner resides.

8. A controller device according to claim 1, wherein the first AMF is further operative, in response to a read request from the host to obtain read data from the redundancy group, to:

(a) send a share request to one or more of the sharing partners requesting a copy of the read data;

(b) obtain the read data by (1) receiving it from one of the sharing partners when the read data resides in a cache of the sharing partner, and (2) retrieving it from the physical storage device when the read data does not reside in any of the caches of the sharing partners; and

(c) upon obtaining the read data, transfer the read data to the host in satisfaction of the read request.

9. A controller device according to claim 8, wherein the first AMF is further operative, in response to an incoming share request sent by one of the sharing partners, to (a) check whether a copy of data of the incoming share request resides in a local cache of the first AMF, and (b) if so, supply the copy of the data from the local cache to the one sharing partner.

10. A method of maintaining data coherency across a set of caching controller devices connected by a network and sharing access to a redundancy group, comprising:

receiving, at a first array management function (AMF) of one of the caching controller devices, a write request from a host to write data to the redundancy group, wherein the first AMF communicates over the network with the redundancy group and with one or more second AMFs distributed among the caching controller devices and sharing access to the redundancy group, the second AMFs including one or more sharing partners and one or more replication partners, the sharing partners maintaining caches for satisfying read and write requests for data of the redundancy group, the replication partners capable of receiving write data from the first AMF and writing the replicated data to a physical storage device if the one caching controller device fails, at least one of the replication partners also being one of the sharing partners; and

in response to the write request;

(a) sending an invalidate request to one or more of the sharing partners, the invalidate request indicating that any copies of the data cached at the sharing partners are stale and are to be invalidated;

(b) safely storing the data by a selected one of (i) in the event that write-through caching is employed, writing the data to a physical storage device, and (ii) in the event that write-back caching is employed, replicating the data to one or more of the replication partners; and

(c) returning a write acknowledgment to the host only upon safely storing the data.

11. A method according to claim 10, wherein the invalidate request is broadcast to all the sharing partners.

12. A method according to claim 10, further comprising maintaining a record of whether the data is shared, and wherein the invalidate request is sent only if the record indicates that the data is shared.

13. A method according to claim 12, wherein the record further indicates which of the sharing partners is sharing the data, and wherein the invalidate request is sent to only those sharing partners indicated by the record to be sharing the data.

14. A method according to claim 12, wherein sending the invalidate request includes broadcasting the invalidate request to all the sharing partners.

15. A method according to claim 10, wherein at least one of the sharing partners is not included among the replication partners.

16. A method according to claim 10, further comprising (i) receiving replicated data of another write request from one of the replication partners, and (ii) completing the other write request by writing the accompanying data to the physical storage device upon failure of a caching controller device on which the one replication partner resides.

17. A method according to claim 10, further comprising:

receiving a read request from the host to obtain read data from the redundancy group; and

in response to the read request;

(a) sending a share request to one or more of the sharing partners requesting a copy of the read data;

(b) obtaining the read data by (1) receiving it from one of the sharing partners in response to the share request when the read data resides in a cache of the sharing partner, and (2) reading the read data from the physical storage device when the read data does not reside in any of the caches of the sharing partners; and

(c) upon obtaining the read data, transferring the read data to the host in satisfaction of the read request.

18. A method according to claim 17, further comprising:

receiving an incoming share request sent by one of the sharing partners; and

in response to the incoming read request;

(a) checking whether a copy of data of the incoming share request resides in a local cache of the first AMF; and

(b) if so, supplying the copy of the data from the local cache to the one sharing partner.

19. A computer program product comprising a computer-readable storage medium on which computer instructions are stored, the computer instructions being operative when executed by a computerized device to cause the computerized device to perform the method of claim 10.