Lightweight coherency control protocol for clustered storage system

• US 7,376,796 B2
• Filed: 11/01/2005
• Issued: 05/20/2008
• Est. Priority Date: 11/01/2005
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Corbett, Peter F., Wagner, J. Christopher
• Primary Examiner(s): NGUYEN, THAN VINH
• Application Number:
  US11/264,601
• Publication Number:
  US 20070101069A1
• Time in Patent Office: 931 Days
• Field of Search: None
• US Class Current: 711/141
• CPC Class Codes: G06F 12/0815 Cache consistency protocols
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 2212/263 Network storage, e.g. SAN or NAS
• CPC Class Codes: G06F 2212/284 being distributed

Abstract

A lightweight coherency control protocol ensures consistency of data containers, such as a file, and associated data buffers stored on one or more volumes served by a plurality of nodes, e.g., storage systems, connected as a cluster. Each data buffer is associated with a current cache sequence number comprising a load generation value and an update count value. The load generation value is incremented every time an inode of a file is loaded into a memory of the storage system. Once the inode is loaded and its load generation value is set, then the appropriate buffer of a buffer tree for the file is loaded into the memory. The update count value is incremented each time the buffer is updated with a write request/operation. Therefore, each buffer loaded into memory is tagged with the load generation value and an update count from the time that buffer is loaded.

Claims

1. A method for ensuring consistency of data containers and associated data buffers stored on one or more volumes served by a plurality of nodes connected as a cluster, comprising:

tagging each data buffer of each data container with a current cache sequence number comprising a load generation value and an update count value;

incrementing the load generation value every time an inode of a data container is loaded into a memory of a node;

loading a data buffer of the data container into the memory; and

incrementing the update count value of the data buffer each time the buffer is modified as long as its inode remains loaded in memory.

2. The method of claim 1 wherein each node comprises a disk element adapted to service the one or more volumes of the cluster and a network element adapted to redirect a data access request for a data container to any disk element of the cluster.

3. The method of claim 2 further comprising the step of maintaining an authoritative version of the data container and associated data buffer at the disk element.

4. The method of claim 3 further comprising the step of caching a version of the data buffer at the network element.

5. The method of claim 4 further comprising:

receiving a data access request from a client directed to the data container at the network element;

forwarding a first message to the appropriate disk element to validate the data buffer cached at the network element;

determining whether the data buffer cached at the network element is representative of a most recent version of the data buffer; and

if so, returning a response to the network element indicating that the cached buffer can be used to service the request.

6. The method of claim 5 wherein the step of determining further comprises the step of using the current cache sequence number to determine whether the data buffer cached at the network element is representative of a most recent version of the data buffer.

7. The method of claim 5 further comprising one of the steps of, if the data buffer cached at the network element is not representative of a most recent version of the data buffer:

returning a copy of the updated data buffer to the network element as part of the response from the disk element; and

returning the response indicating that the cached buffer is stale.

8. The method of claim 7 wherein the step of returning the response indicating that the cached buffer is stale comprises the step of forwarding a second message from the network element to the disk element requesting the updated data buffer.

9. A system configured to ensure consistency of data containers and associated data buffers stored on one or more volumes served by a plurality of nodes connected as a cluster, they system comprising:

a disk element of a node adapted to service the one or more volumes of the cluster, the disk element further adapted to maintain an authoritative version of a data container and associated data buffer stored on its serviced volumes;

a network element of a node adapted to redirect a data access request for the data container and associated data buffer to the disk element;

a local cache of the network element adapted to provide caching of the data buffer, the cached data buffer having an associated current cache sequence number wherein the current cache sequence number comprises a load generation value and an update count value, wherein the load generation value is incremented every time an inode of the data container is loaded into a memory of the disk element and wherein the update count value of the data buffer is incremented each time the buffer is modified as long as the inode remains loaded in memory; and

wherein the network element uses the cached data buffer to service the data access request when the current cache sequence number represents a most recent version of the data buffer.

10. Apparatus adapted to ensure consistency of data containers and associated data buffers stored on one or more volumes served by a plurality of nodes connected as a cluster, the apparatus comprising:

means for tagging each data buffer of each data container with a current cache sequence number comprising a load generation value and an update count value;

means for incrementing the load generation value every time an inode of a data container is loaded into a memory of a node;

means for loading a data buffer of the data container into the memory; and

means for incrementing the update count value of the data buffer each time the buffer is modified as long as its inode remains loaded in memory.

11. The apparatus of claim 10 wherein each node comprises a disk element adapted to service the one or more volumes of the cluster and a network element adapted to redirect a data access request for a data container to any disk element of the cluster.

12. The apparatus of claim 11 further comprising means for maintaining an authoritative version of the data container and associated data buffer at the disk element.

13. The apparatus of claim 12 further comprising means for caching a version of the data buffer at the network element.

14. The apparatus of claim 13 further comprising:

means for receiving a data access request from a client directed to the data container at the network element;

means for forwarding a message to the appropriate disk element to validate the data buffer cached at the network element;

means for determining whether the data buffer cached at the network element is representative of a most recent version of the data buffer; and

if so, means for returning a response to the network element indicating that the cached buffer can be used to service the request.

15. The apparatus of claim 14 further comprising one of, if the data buffer cached at the network element is not representative of a most recent version of the data buffer:

means for returning a copy of the updated data buffer to the network element as part of the response from the disk element; and

means for returning the response indicating that the cached buffer is stale.

16. A computer readable medium containing executable program instructions for ensuring consistency of data containers and associated data buffers stored on one or more volumes served by a plurality of nodes connected as a cluster, the executable instructions comprising one or more program instructions for:

tagging each data buffer of each data container with a current cache sequence number comprising a load generation value and an update count value;

incrementing the load generation value every time an inode of a data container is loaded into a memory of a node;

loading a data buffer of the data container into the memory; and

incrementing the update count value of the data buffer each time the buffer is modified as long as its inode remains loaded in memory.

17. The computer readable medium of claim 16 wherein each node comprises a disk element adapted to service the one or more volumes of the cluster and a network element adapted to redirect a data access request for a data container to any disk element of the cluster.

18. The computer readable medium of claim 17 further comprising one or more program instructions for maintaining an authoritative version of the data container and associated data buffer at the disk element.

19. The computer readable medium of claim 18 further comprising one or more program instructions for caching a version of the data buffer at the network element.

20. The computer readable medium of claim 19 further comprising one or more program instructions for:

receiving a data access request from a client directed to the data container at the network element;

forwarding a first message to the appropriate disk element to validate the data buffer cached at the network element;

determining whether the data buffer cached at the network element is representative of a most recent version of the data buffer; and

if so, returning a response to the network element indicating that the cached buffer can be used to service the request.

21. The computer readable medium of claim 20 further comprising one or more program instructions for, if the data buffer cached at the network element is not representative of a most recent version of the data buffer:

returning a copy of the updated data buffer to the network element as part of the response from the disk element;

or

returning the response indicating that the cached buffer is stale.