Write performance in fault-tolerant clustered storage systems

• US 9,454,435 B2
• Filed: 08/30/2013
• Issued: 09/27/2016
• Est. Priority Date: 12/19/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Belluomini, Wendy A., Gupta, Karan, Hildebrand, Dean, Povzner, Anna S., Pucha, Himabindu, Tewari, Renu
• Primary Examiner(s): Manoskey, Joseph D
• Application Number:
  US14/014,852
• Publication Number:
  US 20140173185A1
• Time in Patent Office: 1,124 Days
• Field of Search: 714/20, 714/2, 714/4.1, 714/6.1, 714/6.3, 707/822, 707/610, 707/619
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1415 at system level
• CPC Class Codes: G06F 11/1666 where the redundant component is memory or memory area
• CPC Class Codes: G06F 11/20 using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/2343 Locking methods, e.g. distributed locking or locking implementation details
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: G06F 2201/825 the problem or solution involving locking
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Embodiments of the invention relate to supporting transaction data committed to a stable storage. Committed data in the cluster is stored in the persistent cache layer and replicated and stored in the cache layer of one or more secondary nodes. One copy is designated as a master copy and all other copies are designated as replica, with an exclusive write lock assigned to the master and a shared write lock extended to the replica. An acknowledgement of receiving the data is communicated following confirmation that the data has been replicated to each node designated to receive the replica. Managers and a director are provided to support management of the master copy and the replicas within the file system, including invalidation of replicas, fault tolerance associated with failure of a node holding a master copy, recovery from a failed node, recovered of the file system from a power failure, and transferring master and replica copies within the file system.

Claims

1. A method comprising:

integrating a stable memory layer with a page cache layer in a file system to temporarily hold committed data in distributed non-volatile memory of nodes in a cluster;

in response to receiving a synchronous write transaction in the file system, placing data associated with the received write transaction in the page cache layer and replicating the received data within the page cache layer of one or more remote nodes in the cluster;

distinguishing between a master copy of the received data and a replica of the received data, including applying existing cache policies to the master copy of the received data; and

in response to flushing the master copy to persistent storage, invalidating each replica on the one or more remote nodes.

2. The method of claim 1, further comprising the master copy holding an exclusive cluster-wide write lock and the replica holding a shared cluster-wide write lock.

3. The method of claim 2, further comprising maintaining fault tolerance in response to a failure of a node holding the master copy, including electing a remote node to acquire the master copy by acquiring the exclusive cluster-wide write lock for the data.

4. The method of claim 3, further comprising recovering content on the recovered failed node in response to the recovery of the failed node, including reading data from non-volatile memory of a surviving node, wherein the surviving node is selected from the group consisting of:

an available node and the master node.

5. The method of claim 2, further comprising transferring designation of the master copy to a requestor node, including revoking the exclusive write lock from the master copy, and transferring the exclusive write lock to the requestor node.

6. The method of claim 1, further comprising in response to recovery of the file system from a power failure, recovering data from non-volatile memory content on each node, and identifying master and replica copies from a characteristic of data byte-range and validating master and replica copies by re-acquiring cluster-wide write locks.

7. A method comprising:

integrating a stable memory layer with a page cache layer in a file system to temporarily hold data committed to a stable storage in non-volatile memory of nodes in a cluster;

receiving a synchronous write transaction in the file system, including placing data associated with the transaction in the page cache layer of the file system and replicating received data within the page cache layer;

maintaining a master copy and a replica of the received data, the master copy having distinguishing characteristics from the replica; and

applying existing cache policies to the master copy of the received data; and

invalidating the replica following flushing of the master copy to persistent storage.

8. The method of claim 7, further comprising the master copy holding an exclusive cluster-wide write lock and the replica holding a shared cluster-wide write lock.

9. The method of claim 7, further comprising recovering data from non-volatile memory content on each node, identifying master and replica copies from a characteristic of data byte-range, and validating master and replica copies by re-acquiring cluster-wide write locks, responsive to recovery from a power failure.