Method and apparatus for cache synchronization in a clustered environment

• US 20020194429A1
• Filed: 05/07/2001
• Published: 12/19/2002
• Est. Priority Date: 05/07/2001
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Fuente, Carlos Francisco, Glider, Joseph Samuel, Chiu, Lawrence Yium-Chee
• Granted Patent: US 6,587,921 B2
• Time in Patent Office: Days
• US Class Current: 711/118
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0815 Cache consistency protocols
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 2212/284 being distributed
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users

Abstract

Providing high availability cache coherency in a distributed cache environment for a storage cluster. An extent owner node in a cluster controls extent disk requests and the locking and demotion of extent data. An extent client node may access the data but does not control destaging or locking. The extent client waits for a lock state grant prior to completing any I/O request. An extent client is allowed to receive data from a host for a write request and later let the extent owner sort out the proper cache coherency order. An extent client is not required to request a lock state change from the extent owner when a read cache hit occurs in the extent client. When a read miss occurs in an extent client, the extent client can initiate a stage request from disk and request the owner for a lock state change at a later time.

Claims

1. A method of writing to cache in a clustered environment comprising:

receiving a request to write data in a first node of a storage cluster from a host;

obtaining allocated memory space in a cache of the first node for said data;

receiving the data into the allocated memory space from the host prior to the first node receiving a requested lock state grant;

notifying a relevant secondary node in the storage cluster of the request to write data; and

transferring the data from the first node to the secondary node.

2. The method of claim 1 further comprising invalidating a track in memory of a remote node.

3. The method of claim 1 further comprising notifying the host of a completion of a write operation.

4. The method of claim 1 wherein:

the first node is an extent owner node responsible for maintaining a lock state of an extent in cache; and

the secondary node is an extent client node that maintains a backup copy of the data in an extent in cache.

5. The method of claim 1 wherein:

the secondary node is an extent owner node responsible for maintaining a lock state of an extent in cache; and

the first node is an extent client node that maintains a backup copy of the data in an extent in cache.

6. The method of claim 1 further comprising:

destaging the data from the cache of the first node to disk;

marking a track in the cache containing the data as unmodified; and

notifying the secondary node of the destaging.

7. The method of claim 6 further comprising:

marking a corresponding track in the cache of the secondary node as unmodified; and

demoting the corresponding track in the cache of the secondary node.

8. The method of claim 1 further comprising synchronizing cache after the data has been received in the first node from the host.

9. A method of reading data from cache in a clustered environment comprising:

receiving a request to read data in a first node of a storage cluster from a host;

allocating a track in cache of the first node for the requested data;

asking permission from the owner of the track to access the track;

obtaining the data into the cache;

providing the host with access to the data after obtaining permission from a node responsible for the data.

10. The method of claim 9 wherein the obtaining further comprises:

requesting the data from an extent owner node responsible for the data;

receiving notification in the first node from the extent owner node that the data is not in cache; and

staging the data from disk into the track in cache prior to obtaining permission from the extent owner node.

11. The method of claim 9 further comprising:

requesting the data from an extent owner node responsible for the data; and

receiving the data from the extent owner node.

12. The method of claim 9 wherein the data is located in the cache of the first node, the method further comprising forwarding the data to a second node prior to obtaining permission from a node responsible for the data.

13. The method of claim 9 wherein the asking and obtaining steps are performed in parallel.

15. The apparatus of claim 14 wherein at least one of the storage nodes is further configured to invalidate a track in a memory of a remote node.

16. The apparatus of claim 14 wherein at least one of the storage nodes is further configured to notify the host of a completion of a write operation.

17. The apparatus of claim 14 wherein:

the first node is an extent owner node and is further configured to be responsible for maintaining a lock state of an extent in cache; and

the second node is an extent client node that is configured to maintain a backup copy of the data in an extent in cache.

18. The apparatus of claim 14 wherein:

the second node is an extent owner node that is configured to be responsible for maintaining a lock state of an extent in cache; and

the first node is an extent client node that is configured to maintain a backup copy of the data in an extent in cache.

19. The apparatus of claim 14, wherein at least one of the storage nodes is further configured to:

destage the data from the cache of the storage node to disk;

mark a track in the cache containing the data as unmodified; and

notifying the other storage node of the destaging.

20. The apparatus of claim 19 wherein at least one of the storage nodes is further configured to:

mark a corresponding track in the cache of the other storage node as unmodified; and

demote the corresponding track in the cache of the other storage node.

21. The apparatus of claim 14, wherein at least one of the storage nodes is further configured to synchronize the cache after the data has been received from the host in one of the storage nodes.

14. An apparatus for writing cache in a clustered environment comprising:

(a) a cache;

(b) a first storage node and a second storage node organized in a storage cluster, each storage node having an interface for connecting to a host and a storage disk, wherein each storage node maintains cache, and wherein at least one of the storage nodes is configured to;

(i) receive a request to write data from a host;

(ii) obtain allocated memory space in the cache of the storage node;

(iii) receive the data into the allocated memory space from the host prior to the storage node receiving a requested lock state grant;

(iv) notify one of the storage nodes of the request to write data; and

(v) transfer the data from one of the storage nodes to the other storage node.

22. An apparatus for reading cache in a clustered environment comprising:

(a) a cache;

(b) a first storage node and a second storage node organized in a storage cluster, each storage node having an interface for connecting to a host and a storage disk, wherein each storage node maintains cache, and wherein at least one of the storage nodes is configured to;

(i) receive a request to read data from a host;

(ii) obtain allocated memory space in the cache;

(iii) ask permission from the owner of the track to access the track;

(iv) obtain the data into the cache; and

(v) provide the host with access to the data after obtaining permission from a node responsible for the data.

23. The apparatus of claim 22, wherein at least one of the storage nodes is configured to obtain the data into the cache by:

requesting the data from an extent owner node responsible for the data;

receiving notification from the extent owner node that the data is not in cache; and

staging the data from disk into the track in cache.

24. The apparatus of claim 22 wherein at least one of the storage nodes is configured to obtain the data by:

requesting the data from an extent owner node responsible for the data; and

receiving the data from the extent owner node.

25. The apparatus of claim 22 wherein the data is located in the cache of the first node, the first storage node further configured to forward the data to the second node prior to obtaining permission from a node responsible for the data.

26. The apparatus of claim 22 wherein the storage node is configured to ask permission and obtain the data in parallel.

28. The article of manufacture of claim 27, the method further comprising invalidating a track in a memory of a remote node.

29. The article of manufacture of claim 27, the method further comprising notifying the host of a completion of a write operation.

30. The article of manufacture of claim 27 wherein:

the first node is an extent owner node responsible for maintaining a lock state of an extent in cache; and

the second node is an extent client node that maintains a backup copy of the data in an extent in cache.

31. The article of manufacture of claim 27 wherein:

the second node is an extent owner node responsible for maintaining a lock state of an extent in cache; and

the first node is an extent client node that maintains a backup copy of the data in an extent in cache.

32. The article of manufacture of claim 27, the method further comprising:

destaging the data from the cache of the first node to disk;

marking a track in the cache containing the data as unmodified; and

notifying the second node of the destaging.

33. The article of manufacture of claim 32, the method further comprising:

marking a corresponding track in the cache of the second node as unmodified; and

demoting the corresponding track in the cache of the second node.

34. The article of manufacture of claim 27, the method further comprising synchronizing cache after the data has been received in the first node from the host.

36. The article of manufacture of claim 35, wherein the obtaining comprises:

requesting the data from an extent owner node responsible for the data;

receiving notification in the first node from the extent owner node that the data is not in cache; and

staging the data from disk into the track in cache.

37. The article of manufacture of claim 35, wherein the obtaining comprises:

requesting the data from an extent owner node responsible for the data; and

receiving the data from the extent owner node.

38. The article of manufacture of claim 35 wherein the data is located in the cache of the first node, the method further comprising forwarding the data to a second node prior to obtaining permission from a node responsible for the data.

39. The article of manufacture of claim 35 wherein the method asks permissions and obtains the data in parallel.

27. An article of manufacture, embodying logic to perform a method of writing cache in a clustered environment, the method comprising:

receiving a request to write data in a first node of a storage cluster from a host;

obtaining allocated memory space in cache of the first node for said data;

receiving the data into the allocated memory space from the host prior to the first node receiving a requested lock state grant;

notifying a relevant second node in the storage cluster of the request to write data; and

transferring the data from the first node to the second node.

35. An article of manufacture, embodying logic to perform a method of reading cache in a clustered environment, the method comprising:

receiving a request to read data in a first node of a storage cluster from a host;

allocating a track in cache of the first node for the requested data;

asking permission from the owner of the track to access the track;

obtaining the data into the cache; and

providing the host with access to the data after obtaining permission from a node responsible for the data.