Live NV replay for enabling high performance and efficient takeover in multi-node storage cluster
First Claim
1. A system comprising:
- one or more processors of a partner node interconnected to a local node of a multi-node storage cluster;
a non-volatile random access memory (NVRAM) connected to the one or more processors, the NVRAM configured to maintain operations serviced and logged by the local node, but not committed to storage devices of the local node; and
a memory coupled to the one or more processors and configured to store metadata describing a type of each operation logged in the NVRAM, the metadata further describing a location of the logged operation in the NVRAM and any dependency among the logged operation and other logged operations in the NVRAM, the memory further configured to store a storage operating system implemented as one or processes executable by the one or more processors to, upon failure of the local node, interpret the metadata to dynamically replay in parallel any logged operations not constrained by an ordering requirement.
1 Assignment
0 Petitions
Accused Products
Abstract
A live non-volatile (NV) replay technique enables a partner node to efficiently takeover a failed node of a high-availability pair in a multi-node storage cluster by dynamically replaying operations synchronously logged in a non-volatile random access memory (NVRAM) of the partner node, while also providing high performance during normal operation. Dynamic live replay may be effected through interpretation of metadata describing the logged operations. The metadata may specify a location and type of each logged operation within a partner portion of the NVRAM, as well as any dependency among the logged operation and any other logged operations that would impose an ordering constraint. During normal operation, the partner node may consult the metadata to identify dependent logged operations and dynamically replay those operations to satisfy one or more requests. Upon failure of the node, the partner node may replay, in parallel, those logged operations having no imposed ordering constraint, thereby reducing time needed to complete takeover of the failed node.
-
Citations
20 Claims
-
1. A system comprising:
-
one or more processors of a partner node interconnected to a local node of a multi-node storage cluster; a non-volatile random access memory (NVRAM) connected to the one or more processors, the NVRAM configured to maintain operations serviced and logged by the local node, but not committed to storage devices of the local node; and a memory coupled to the one or more processors and configured to store metadata describing a type of each operation logged in the NVRAM, the metadata further describing a location of the logged operation in the NVRAM and any dependency among the logged operation and other logged operations in the NVRAM, the memory further configured to store a storage operating system implemented as one or processes executable by the one or more processors to, upon failure of the local node, interpret the metadata to dynamically replay in parallel any logged operations not constrained by an ordering requirement. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A method comprising:
-
executing an operation at a local node of multi-node storage cluster, the local node coupled to a partner node of the cluster by a high availability (HA) interconnect; logging the operation in a first non-volatile random access memory (NVRAM) of the local node; generating metadata describing the logged operation at the local node; mirroring the logged operation from the first NVRAM to a second NVRAM on the partner node; transferring the metadata to a dependency index data structure of the partner node; receiving a request issued by an application at the partner node to redirect servicing of the request to the local node; interpreting the metadata at a semantic replay parser of the partner node to determine whether replay of the logged operation requires ordering due to a dependency among other operations logged in the second NVRAM; and dynamically replaying one or more of the logged operations having the dependency to service the request at the partner node without redirecting service to the local node. - View Dependent Claims (7, 8, 9, 10, 11, 12)
-
-
13. A system comprising:
-
one or more processors of a local node interconnected by a cluster switching fabric to a plurality of nodes, wherein the local node is further coupled to a partner node by a high availability (HA) interconnect; a first non-volatile random access memory (NVRAM) connected to the one or more processors, the first NVRAM having a local portion configured to maintain operations logged and serviced by the local node, but not yet committed to storage devices of the local node; and a memory coupled to the one or more processors and configured to store a storage operating system implemented as one or processes executable by the one or more processors to generate metadata describing a type of each operation logged in the local portion of the first NVRAM, the metadata further describing a location of each logged operation in the local portion of the first NVRAM and any dependency among the logged operations in the local portion of the first NVRAM. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20)
-
Specification