Method and apparatus for a global cache directory in a storage cluster

• US 20030028819A1
• Filed: 05/07/2001
• Published: 02/06/2003
• Est. Priority Date: 05/07/2001
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Sun Hsu, Windsor Wee, Young, Honesty Cheng, Chiu, Lawrence Yium-chee
• Granted Patent: US 6,996,674 B2
• Time in Patent Office: Days
• US Class Current: 714/5
• CPC Class Codes: G06F 11/2005 using redundant communication controllers
• CPC Class Codes: G06F 11/2007 using redundant communication media
• CPC Class Codes: G06F 11/2017 where memory access, memory control or I/O control functionality is redundant redundant communication control functionality G06F11/2005; redundant storage control functionality G06F11/2089
• CPC Class Codes: G06F 11/2089 Redundant storage control functionality
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 12/0815 Cache consistency protocols
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: H04L 67/563 Data redirection of data network streams
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data
• CPC Class Codes: H04L 67/59 Providing operational support to end devices by off-loading in the network or by emulation, e.g. when they are unavailable
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A method, apparatus, article of manufacture, and a memory structure for an internode network. Nodes are active-active using commodity hardware so that the system can perform I/O together between any number of nodes, and data can be located on any given node. A single modified image is configured to maintain recent and updated data. At least one failure can occur (and be corrected) in the nodes before data is written to disk. A history of access points is kept in a cache directory, and it is assumed that the nodes most frequently accessed in the past are likely to be the most frequently accessed in the near future. One or more embodiments of the invention move this data to where it will likely be needed. This means that data is delivered to hosts quickly, as is required in high volume enterprise web environments. The symbolic list in the cache directory provides a history of the nodes that have previously performed I/O operations. When data is requested, embodiments of the invention look at the symbolic list to determine which node'"'"'s cache contains the requested data. Once a new write I/O operation is performed, the symbolic list is updated to provide for the new I/O operation. Additionally, embodiments of the invention provide Fast write support, fault tolerance support, and concurrent node installation support.

Claims

1. A method of maintaining cache in a clustered environment comprising:

receiving a request in a primary node of a storage cluster for accessing data;

selecting a secondary node for storing data in cache based on a historic point of access list maintained in a cache directory;

forwarding modified data and symbolic information to one or more relevant nodes in the storage cluster; and

updating the historic point of access list based on the symbolic information.

2. The method of claim 1 wherein the request is to write data.

3. The method of claim 1 wherein:

the historic point of access indicates that the data is not currently in cache of any node of the storage cluster; and

the secondary node selected is any node in the storage cluster.

4. The method of claim 1 wherein:

the historic point of access indicates that an original primary node maintains the data in cache; and

the secondary node selected is the original primary node.

5. The method of claim 1 wherein:

the symbolic information includes information relating to the first node; and

the historic point of access list is updated by;

listing the first node as the primary node; and

listing the secondary node as the secondary node.

6. The method of claim 1 further comprising selecting a remote node that is an original secondary node in the historic point of access list maintained in the cache directory.

7. The method of claim 1 wherein:

a copy of the modified data is maintained in the first node and the secondary node; and

the symbolic information is maintained in remaining nodes of the storage cluster.

8. The method of claim 1 further comprising acquiring a lock on associated tracks on nodes in the storage cluster wherein the locking protocol provides for multiple readers and a single writer.

9. The method of claim 1 further comprising:

detecting a failure of a node in the storage cluster;

broadcasting a failover recovery message to all nodes in the storage cluster; and

replicating the data from the primary node or the secondary node to another node in the storage cluster.

10. The method of claim 1 further comprising:

detecting a failure of a node in the storage cluster;

broadcasting a failover recovery message to all nodes in the storage cluster; and

destaging the data from the primary node or the secondary node to disk.

11. The method of claim 1 further comprising:

applying for cluster admission;

requesting the symbolic information for new write requests;

requesting a modified track list comprising an identifier of modified data and an associated symbolic entry;

merging the modified track list with any new symbolic entries; and

broadcasting availability to remaining nodes in the storage cluster.

12. An apparatus for maintaining cache in a clustered environment comprising:

(a) a cache;

(b) a cache directory comprising a historic point of access list for the cache;

(a) a storage node organized in a storage cluster and having an interface for connecting to a host, a storage disk, and one or more additional storage nodes, wherein the storage node maintains cache and the cache directory, and wherein the storage node is configured to;

(i) receive a request for accessing data;

(ii) select a secondary node for storing data in cache based on the historic point of access list;

(iii) forward modified data and symbolic information to one or more additional storage nodes in the storage cluster; and

(iv) update the historic point of access list based on the symbolic information.

13. The apparatus of claim 12 wherein the request is to write data.

14. The apparatus of claim 12 wherein:

the historic point of access indicates that the data is not currently in cache of the nodes in the storage cluster; and

the secondary node selected is any node in the storage cluster.

15. The apparatus of claim 12 wherein:

the historic point of access indicates that an original primary node maintains the data in cache; and

the secondary node selected is the original primary node.

16. The apparatus of claim 12 wherein:

the symbolic information includes information relating to a first node that receives the request; and

the historic point of access list is updated by;

listing the first node as the primary node; and

listing the secondary node as the secondary node.

17. The apparatus of claim 12, wherein the storage node is further configured to select a remote node that is an original secondary node in the historic point of access list maintained in the cache directory.

18. The apparatus of claim 12 wherein:

a copy of the modified data is maintained in two nodes in the storage cluster; and

the symbolic information is maintained in remaining nodes of the storage cluster.

19. The apparatus of claim 12, wherein the storage node is further configured to acquire a lock on associated tracks on relevant nodes in the storage cluster wherein the locking protocol provides for multiple readers and a single writer.

20. The apparatus of claim 12, wherein the storage node is further configured to:

detect a failure of a node in the storage cluster;

broadcast a failover recovery message to an additional storage node in the storage cluster; and

replicate the data from one node in the storage cluster to another node in the storage cluster.

21. The apparatus of claim 12, wherein the storage node is further configured to:

detect a failure of a node in the storage cluster;

broadcast a failover recovery message to an additional node in the storage cluster; and

destage the data from a node in the storage cluster to disk.

22. The apparatus of claim 12 further comprising a new node configured to:

apply for cluster admission;

request the symbolic information for new write requests;

request a modified track list comprising an identifier of modified data and an associated symbolic entry;

merge the modified track list with any new symbolic entries; and

broadcast availability to remaining nodes in the storage cluster.

23. An article of manufacture, embodying logic to perform a method of maintaining cache in a clustered environment, the method comprising:

receiving a request in a primary node of a storage cluster for accessing data;

selecting a secondary node for storing data in cache based on a historic point of access list maintained in a cache directory;

forwarding modified data and symbolic information to one or more relevant nodes in the storage cluster; and

updating the historic point of access list based on the symbolic information.

24. The article of manufacture 23 wherein the request is to write data.

25. The article of manufacture 23 wherein:

the historic point of access indicates that the data is not currently in cache of any node of the storage cluster; and

the secondary node selected is any node in the storage cluster.

26. The article of manufacture 23 wherein:

the historic point of access indicates that an original primary node maintains the data in cache; and

the secondary node selected is the original primary node.

27. The article of manufacture 23 wherein:

the symbolic information includes information relating to the first node; and

the historic point of access list is updated by;

listing the first node as the primary node; and

listing the secondary node as the secondary node.

28. The article of manufacture 23, the method further comprising selecting a remote node that is an original secondary node in the historic point of access list maintained in the cache directory.

29. The article of manufacture 23 wherein:

a copy of the modified data is maintained in the first node and the secondary node; and

the symbolic information is maintained in remaining nodes of the storage cluster.

30. The article of manufacture 23, the method further comprising acquiring a lock on associated tracks on nodes in the storage cluster wherein the locking protocol provides for multiple readers and a single writer.

31. The article of manufacture 23, the method further comprising:

detecting a failure of a node in the storage cluster;

broadcasting a failover recovery message to all nodes in the storage cluster; and

replicating the data from the primary node or the secondary node to another node in the storage cluster.

32. The article of manufacture 23, the method further comprising:

detecting a failure of a node in the storage cluster;

broadcasting a failover recovery message to all nodes in the storage cluster; and

destaging the data from the primary node or the secondary node to disk.

33. The article of manufacture 23, the method further comprising:

applying for cluster admission;

requesting the symbolic information for new write requests;

requesting a modified track list comprising an identifier of modified data and an associated symbolic entry;

merging the modified track list with any new symbolic entries; and

broadcasting availability to remaining nodes in the storage cluster.