Processing and distributing write logs of nodes of a cluster storage system

• US 8,145,838 B1
• Filed: 03/10/2009
• Issued: 03/27/2012
• Est. Priority Date: 03/10/2009
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Miller, Steven C, Corbett, Peter F.
• Primary Examiner(s): Shah, Sanjiv
• Assistant Examiner(s): TSUI, DANIEL D
• Application Number:
  US12/401,533
• Time in Patent Office: 1,113 Days
• Field of Search: 711/114
• US Class Current: 711/114
• CPC Class Codes: G06F 11/2023 Failover techniques
• CPC Class Codes: G06F 11/2035 without idle spare hardware
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups

Abstract

A cluster storage system comprises a plurality of nodes that access a shared storage, each node having two or more failover partner nodes. A primary node produces write logs for received write requests and produces parity data for the write logs (storing the parity data to local non-volatile storage). By storing parity data rather than actual write logs, the non-volatile storage space within the cluster for storing write logs is reduced. Prior to failure of the primary node, the primary node also sub-divides the write logs into two or more sub-sets and distributes the sub-sets to the two or more partner nodes for storage at non-volatile storage devices. Thus, if the primary node fails, its write logs are already distributed among the partner nodes so each partner node may perform the allotted write logs on the storage, thus improving the response time to the primary node failure.

Claims

1. A cluster storage system for providing clients access to data of a shared storage, the cluster storage system comprising:

the shared storage comprising a plurality of aggregates, each aggregate comprising a predetermined set of storage devices for storing data of the aggregate;

a plurality of nodes, each node assigned to service data of a predetermined plurality of aggregates in the shared storage, wherein during normal operation when no node failure has occurred, each node can access only the aggregates assigned to the node, the plurality of nodes comprising a primary node and a plurality of partner nodes comprising a first partner node and a second partner node;

the primary node assigned to service data of a first aggregate and a second aggregate, the primary node comprising a local de-staging layer configured for;

producing a plurality of write logs for a plurality of received write requests for the first and second aggregates;

producing parity data for the plurality of write logs;

storing the parity data to local non-volatile storage, wherein the plurality of write logs are not stored to local non-volatile storage;

dividing the plurality of write logs on a per aggregate basis, wherein the plurality of write logs is divided into a first set of write logs for the first aggregate and a second set of write logs for the second aggregate; and

distributing the first set of write logs only to the first partner node and the second set of write logs only to the second partner node;

the first partner node comprising a first remote de-staging layer configured for;

receiving the first set of write logs for the first aggregate; and

performing the first set of write logs on the first aggregate upon failure of the primary node, wherein the first partner node is the only partner node performing write logs on the first aggregate upon failure of the primary node; and

the second partner node comprising a second remote de-staging layer configured for;

receiving the second set of write logs for the second aggregate; and

performing the second set of write logs on the second aggregate upon failure of the primary node, wherein the second partner node is the only partner node performing write logs on the second aggregate upon failure of the primary node.

2. The cluster storage system of claim 1, wherein each partner node is configured for resuming the data-access service of the primary node upon failure of the primary node.

3. The cluster storage system of claim 1, wherein each remote de-staging layer is further configured for:

storing the received write logs to local non-volatile storage, wherein the write logs are received and stored prior to failure of the primary node.

4. The cluster storage system of claim 1, wherein the local non-volatile storage comprises write logs received from the plurality of partner nodes.

5. The cluster storage system of claim 1, wherein each remote de-staging layer is further configured for:

upon failure of the primary node, reconstructing corrupt or lost write logs in the received write logs using the parity data; and

performing the reconstructed write logs on the shared storage.

6. The cluster storage system of claim 1, wherein the parity data is not distributed from the primary node to any partner nodes.

7. A method for implementing a cluster storage system for providing clients access to data of a shared storage, the method comprising:

storing data to the shared storage, the shared storage comprising a plurality of aggregates, each aggregate comprising a predetermined set of storage devices for storing data of the aggregate;

providing a plurality of nodes, each node assigned to service data of a predetermined plurality of aggregates in the shared storage, wherein during normal operation when no node failure has occurred, each node can access only the aggregates assigned to the node, the plurality of nodes comprising a primary node and a plurality of partner nodes comprising a first partner node and a second partner node;

assigning the primary node to service data of a first aggregate and a second aggregate;

at the primary node;

producing a plurality of write logs for a plurality of received write requests for the first and second aggregates;

producing parity data for the plurality of write logs;

storing the parity data to local non-volatile storage, wherein the plurality of write logs are not stored to local non-volatile storage;

dividing the plurality of write logs on a per aggregate basis, wherein the plurality of write logs is divided into a first set of write logs for the first aggregate and a second set of write logs for the second aggregate; and

distributing the first set of write logs only to the first partner node and the second set of write logs only to the second partner node;

at the first partner node;

receiving the first set of write logs for the first aggregate; and

performing the first set of write logs on the first aggregate upon failure of the primary node, wherein the first partner node is the only partner node performing write logs on the first aggregate upon failure of the primary node; and

at the second partner node;

receiving the second set of write logs for the second aggregate; and

performing the second set of write logs on the second aggregate upon failure of the primary node, wherein the second partner node is the only partner node performing write logs on the second aggregate upon failure of the primary node.

8. The method of claim 7, wherein each partner node is configured for resuming the data-access service of the primary node upon failure of the primary node.

9. The method of claim 7, further comprising:

at each partner node, storing the received write logs to local non-volatile storage, wherein the write logs are received and stored prior to failure of the primary node.

10. The method of claim 7, further comprising:

at each partner node, upon failure of the primary node, reconstructing corrupt or lost write logs in the received write logs using the parity data; and

at each partner node, performing the reconstructed write logs on the shared storage.

11. The method of claim 7, wherein the parity data is not distributed from the primary node to any partner nodes.

12. A non-transitory computer readable medium having instructions stored thereon when executed, implement a cluster storage system for providing clients access to data of a shared storage, the computer readable medium comprising sets of instructions for:

storing data to the shared storage, the shared storage comprising a plurality of aggregates, each aggregate comprising a predetermined set of storage devices for storing data of the aggregate;

providing a plurality of nodes, each node assigned to service data of a predetermined plurality of aggregates in the shared storage, wherein during normal operation when no node failure has occurred, each node can access only the aggregates assigned to the node, the plurality of nodes comprising a primary node and a plurality of partner nodes comprising a first partner node and a second partner node;

assigning the primary node to service data of a first aggregate and a second aggregate;

at the primary node;

producing a plurality of write logs for a plurality of received write requests for the first and second aggregates;

producing parity data for the plurality of write logs;

storing the parity data to local non-volatile storage, wherein the plurality of write logs are not stored to local non-volatile storage;

dividing the plurality of write logs on a per aggregate basis, wherein the plurality of write logs is divided into a first set of write logs for the first aggregate and a second set of write logs for the second aggregate; and

distributing the first set of write logs only to the first partner node and the second set of write logs only to the second partner node;

at the first partner node;

receiving the first set of write logs for the first aggregate; and

performing the first set of write logs on the first aggregate upon failure of the primary node, wherein the first partner node is the only partner node performing write logs on the first aggregate upon failure of the primary node; and

at the second partner node;

receiving the second set of write logs for the second aggregate; and

performing the second set of write logs on the second aggregate upon failure of the primary node, wherein the second partner node is the only partner node performing write logs on the second aggregate upon failure of the primary node.

13. The non-transitory computer readable medium of claim 12, wherein each partner node is configured for resuming the data-access service of the primary node upon failure of the primary node.

14. The non-transitory computer readable medium of claim 12, further comprising a set of instructions for:

at each partner node, storing the received write logs to local non-volatile storage, wherein the write logs are received and stored prior to failure of the primary node.

15. The non-transitory computer readable medium of claim 12, further comprising a set of instructions for:

at each partner node, upon failure of the primary node, reconstructing corrupt or lost write logs in the received write logs using the parity data; and

at each partner node, performing the reconstructed write logs on the shared storage.

16. The non-transitory computer readable medium of claim 12, wherein the parity data is not distributed from the primary node to any partner nodes.