Method and apparatus for a global cache directory in a storage cluster

• US 6,996,674 B2
• Filed: 05/07/2001
• Issued: 02/07/2006
• Est. Priority Date: 05/07/2001
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hsu, Windsor Wee Sun, Young, Honesty Cheng, Chiu, Lawrence Yium-chee
• Primary Examiner(s): Sparks, Donald
• Assistant Examiner(s): Farrokh, Hashem
• Application Number:
  US09/851,452
• Publication Number:
  US 20030028819A1
• Time in Patent Office: 1,737 Days
• Field of Search: 711/100, 711/114, 711/118, 711/152, 711/154, 711/162, 709/203, 709216-232, 709/9, 709/201, 709236-238
• US Class Current: 711/118
• CPC Class Codes: G06F 11/2005 using redundant communication controllers
• CPC Class Codes: G06F 11/2007 using redundant communication media
• CPC Class Codes: G06F 11/2017 where memory access, memory control or I/O control functionality is redundant redundant communication control functionality G06F11/2005; redundant storage control functionality G06F11/2089
• CPC Class Codes: G06F 11/2089 Redundant storage control functionality
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 12/0815 Cache consistency protocols
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: H04L 67/563 Data redirection of data network streams
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data
• CPC Class Codes: H04L 67/59 Providing operational support to end devices by off-loading in the network or by emulation, e.g. when they are unavailable
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A method, apparatus, and article of manufacture provide the ability to maintain cache in a clustered environment. The cache is maintained in both a primary and secondary node. When data is requested, a symbolic list in a cache directory is examined to determine which node'"'"'s cache contains the requested data. If the symbolic list indicates data is not currently in cache of any node, any node may be used as the secondary node. However, if an original primary node maintains the data in cache, the original primary node is selected as the secondary node. Once a new write I/O operation is performed, the symbolic list is updated to provide. To install a new node, after applying for cluster admission, symbolic information and a modified track list is requested. The modified track list is merged with new symbolic entries and the new node then broadcasts its availability to the cluster.

Claims

1. A method of maintaining cache in a clustered environment comprising:

(a) receiving an I/O request, for modifying data, in a primary node of a storage cluster of two or more nodes;

(b) storing die modified data in cache of the primary node;

(c) selecting a secondary node for storing the modified data in the secondary node'"'"'s cache based on a historic point of access list maintained in a cache directory, wherein;

(i) the historic point of access list identifies which node'"'"'s cache contains which data;

(ii) the historic point of access indicates that the data is not currently in cache of any node of the storage cluster; and

(iii) die secondary node selected is any node in the storage cluster;

(d) forwarding the modified data and symbolic information to one or more relevant nodes in the storage cluster, wherein the symbolic information identifies the primary and secondary nodes as containing the modified data; and

(e) updating the historic point of access list based on the symbolic information.

2. The method of claim 1 wherein the request is to write data.

3. The method of claim 1 wherein:

the symbolic information includes information relating to the first node; and

the historic point of access list is updated by;

listing the first node as the primary node; and

listing the secondary node as the secondary node.

4. The method of claim 1 further comprising selecting a remote node that is an original secondary node in the historic point of access list maintained in the cache directory.

5. The method of claim 1 wherein:

a copy of the modified data is maintained in the first node and the secondary node; and

the symbolic information is maintained in remaining nodes of the storage cluster.

6. The method of claim 1 further comprising acquiring a lock on associated tracks on nodes in the storage cluster wherein the locking protocol provides for multiple readers and a single writer.

7. The method of claim 1 further comprising:

detecting a failure of a node in the storage cluster;

broadcasting a failover recovery message to all nodes in the storage cluster; and

replicating the data from the primary node or the secondary node to another node in the storage cluster.

8. The method of claim 1 further comprising:

detecting a failure of a node in the storage cluster;

broadcasting a failover recovery message to all nodes in the storage cluster; and

destaging the data from the primary node or the secondary node to disk.

9. The method of claim 1 further comprising:

applying for cluster admission;

requesting the symbolic information far new write requests;

requesting a modified track list comprising an identifier of modified data and an associated symbolic entry;

merging the modified tack list with any new symbolic entries; and

broadcasting availability to remaining nodes in the storage cluster.

10. An apparatus for maintaining cache in a clustered environment comprising;

(a) a cache;

(b) a cache directory comprising a historic point of access list for the cache, wherein the historic point of access list identifies which node'"'"'s cache contains which data;

(c) a storage node organized in a storage cluster and having an interface for connecting to a host, a storage disk, and one or more additional storage nodes, wherein the storage node maintains cache and the cache directory, and wherein the storage node is configured to;

(i) receive an I/O request for modifying data;

(ii) store the modified data in the cache of the storage node;

(iii) select a secondary node for storing data in the secondary node'"'"'s cache based on the historic point of access list;

(iv) forward the modified data and symbolic information to one or more additional storage nodes in the storage cluster, wherein the symbolic information identifies a primary node and the secondary node as containing the modified data; and

(v) update the historic point of access list based on the symbolic information

(d) a new node configured to;

(i) apply for cluster admission;

(ii) request the symbolic information for new write requests;

(iii) request a modified track list comprising an identifier of modified data and an associated symbolic entry;

(iv) merge the modified track list with any new symbolic entries; and

(v) broadcast availability to remaining nodes in the storage cluster.

11. The apparatus of claim 10 wherein the I/O request is to write data.

12. The apparatus of claim 10 wherein:

the historic point of access indicates that the data is not currently in cache of the nodes in the storage cluster; and

the secondary node selected is any node in the storage cluster.

13. The apparatus of claim 10 wherein:

the historic point of access indicates that an original primary node maintains the data in cache; and

the secondary node selected is the original primary node.

14. The apparatus of claim 10 wherein:

the symbolic information includes information relating to a first node that receives the request; and

the historic point of access list is updated by;

listing the first node as the primary node; and

listing the secondary node as the secondary node.

15. The apparatus of claim 10, wherein the storage node is further configured to select a remote node that is an original secondary node in the historic point of access list maintained in rite cache directory.

16. The apparatus of claim 10 wherein:

a copy of the modified data is maintained in two nodes in the storage cluster; and

the symbolic information is maintained in remaining nodes of the storage cluster.

17. The apparatus of claim 10, wherein the storage node is further configured to acquire a lock on associated tracks on relevant nodes in the storage cluster wherein the locking protocol provides for multiple readers and a single writer.

18. The apparatus of claim 10, wherein the storage node is further configured to:

detect a failure of a node in the storage cluster;

broadcast a failover recovery message to an additional storage node in the storage cluster; and

replicate the data from one node in the storage cluster to another node in the storage cluster.

19. The apparatus of claim 10, wherein the storage node is further configured to:

detect a failure of a node in the storage cluster;

broadcast a failover recovery message to an additional node in the storage cluster; and

desrage the data front a node in the storage cluster to disk.

20. An article of manufacture, including a computer readable storage medium that tangibly embodies logic to perform a method of maintaining cache in a clustered environment, the method comprising:

(a) receiving an I/O request, for modifying data, in a primary node of a storage cluster of two or more nodes;

(b) storing the modified data in cache of the primary node;

(c) selecting a secondary node for storing the modified data in the secondary node'"'"'s cache based on a historic point of access list maintained in a cache directory, wherein;

(i) the historic point of access list identifies which node'"'"'s cache contains which data;

(ii) the historic point of access indicates that an original primary node maintains the data in cache; and

(iii) the secondary node selected is the original primary node.

(d) forwarding the modified data and symbolic information to one or more relevant nodes in the storage cluster, wherein the symbolic information identifies the primary and secondary nodes as containing the modified data; and

(e) updating the historic point of access list based on the symbolic information.

21. The article of manufacture of claim 20 wherein the request is to write data.

22. The article of manufacture of claim 20 wherein:

the symbolic information includes information relating to the first node; and

the historic point of access list is updated by;

listing the first node as the primary node; and

listing the secondary node as the secondary node.

23. The article of manufacture of claim 20, the method further comprising selecting a remote node that is an original secondary node in the historic point of access list maintained in the cache directory.

24. The article of manufacture of claim 20 wherein:

a copy of the modified data is maintained in the first node and the secondary node; and

the symbolic information is maintained in remaining nodes of the storage cluster.

25. The article of manufacture of claim 20, the method further comprising acquiring a lock on associated tracks on nodes in the storage cluster wherein the locking protocol provides for multiple readers and a single writer.

26. The article of manufacture of claim 20, the method further comprising:

detecting a failure of a node in the storage cluster;

broadcasting a failover recovery message to all nodes in the storage cluster; and

replicating the data from the primary node or the secondary node to another node in the storage cluster.

27. The article of manufacture of claim 20, the method further comprising:

detecting a failure of a node in the storage cluster;

broadcasting a failover recovery message to all nodes in the storage cluster; and

desraging the data from the primary node or the secondary node to disk.

28. The article of manufacture of claim 20, the method further comprising:

applying for cluster admission;

requesting the symbolic information for new write requests;

requesting a modified track list comprising an identifier of modified data and an associated symbolic entry;

merging the modified crack list with any new symbolic entries; and

broadcasting availability to remaining nodes in the storage cluster.