DISTRIBUTED FILE SYSTEM USING CONSENSUS NODES

• US 20150067002A1
• Filed: 08/29/2013
• Published: 03/05/2015
• Est. Priority Date: 01/12/2005
• Status: Active Grant

• Current Assignee: Cirata Incorporated
• Original Assignee: WANdisco, Inc.
• Inventors: Sundar, Jagane, Aahlad, Yeturu, Shvachko, Konstantin V., Parkin, Micahel
• Granted Patent: US 9,361,311 B2
• Time in Patent Office: Days
• US Class Current: 707/827
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems

Abstract

A cluster of nodes in a distributed file system may include; at least two namenodes, each coupled to a plurality of data nodes and each configured to store a state of a namespace of the cluster and each being configured to respond to a request from a client while other(s) of the namenodes are responding to other requests from other clients; and a coordination engine coupled to each of the namenodes. The coordination engine may be configured to receive proposals from the namenodes to change the state of the namespace by replicating, deleting and/or adding data blocks stored in the data nodes and to generate, in response, an ordered set of agreements that specifies an order in which the namenodes are to change the state of the namespace. The namenodes are configured to delay making changes thereto until after the ordered set of agreements is received from the coordination engine.

Claims

1. A cluster of nodes comprising computing devices configured to implement a distributed file system, comprising:

a plurality of data nodes, each configured to store data blocks of client files;

at least two namenodes, each coupled to the plurality of data nodes and each configured to store a state of a namespace of the cluster and each being configured to respond to a request from a client while at least one other of the namenodes is responding to another request from another client;

a coordination engine coupled to each of the namenodes, the coordination engine being configured to receive proposals from the namenodes to change the state of the namespace and to generate, in response, an ordered set of agreements that specifies an order in which the namenodes are to change the state of the namespace, wherein the namenodes are configured to delay making changes to the state of the namespace until the ordered set of agreements is received from the coordination engine.

2. The cluster of nodes of claim 1, wherein each namenode is configured to generate a proposal responsive to a client issuing a request to at least one of replicating and deleting the data blocks in the data nodes.

3. The cluster of claim 1, wherein the coordination engine is further configured to assign a unique global sequence number (GSN) to each agreement, the GSN specifying an order in which the namenodes are to apply the changes to the state of the namespace.

4. The cluster of claim 3, wherein the namenodes are further configured to apply the changes to the state of the namespace in the order specified by the GSN of the received agreements.

5. The cluster of claim 1, further comprising local storage coupled to each of the plurality of namenodes, the local storage being configured to store an image of the namespace and entries specifying updates to the stored image.

6. The cluster of claim 1, wherein only one namenode in the cluster is configured to enable replication and deletion of data blocks stored in the plurality of data nodes.

7. The cluster of claim 6, wherein the namenode configured to enable replication and deletion of data blocks is further configured to periodically issue a block replicator heartbeat message.

8. The cluster of claim 7, wherein failure of the namenode configured to enable replication and deletion of data blocks to periodically issue the block replicator heartbeat triggers an election process to elect a new namenode that is to be solely configured to enable replication and deletion of data blocks.

9. The cluster of claim 8, wherein the election process is configured to be initiated by any of the namenodes.

10. The cluster of claim 1, wherein each of the namenodes is configured to enable data blocks to be generated and stored in the plurality of data nodes.

11. The cluster of claim 1, wherein each of the namenodes is configured to enable data blocks stored on one of the plurality of data nodes to be replicated to a predetermined other one or other ones of the plurality of data nodes.

12. The cluster of claim 11, wherein at least one of the data nodes, upon receipt of data blocks, is configured to send the data blocks to another of the plurality of data nodes identified by one of the namenodes.

13. The cluster of claim 1, wherein each of the plurality of data nodes is configured to periodically issue a heartbeat message to the namenodes indicating a current status of the data node.

14. The cluster of claim 13, wherein the namenodes are further configured to consider a data node from which the periodic heartbeat has not been timely received to have failed.

15. The cluster of claim 1, wherein one of the namenodes is further configured to cause data blocks stored in a failed data node to be replicated to another selected one of the plurality of data nodes.

16. The cluster of claim 1, wherein each of the namenodes is configured to assign a unique block identifier (ID) to data blocks from within a sub-range of a larger range of block ID numbers that is separate from other respective sub-ranges of the larger range of block IDs from which other ones of the namenodes assign block ID numbers.

17. The cluster of claim 1, further comprising a communication timeout configured such that, if a namenode having received a client request has not responded thereto at an expiry of the communication timeout, the client, request is made to another of the namenodes.

18. The cluster of claim 1, wherein all of the namenodes in the cluster are configured to actively support client requests simultaneously.

19. The cluster of claim 1 wherein, upon failure of one of the namenodes, client requests previously made to the failed namenode are made to an other one of the namenodes.

20. The cluster of claim 19 wherein, prior to making client requests to the other namenode, the client is configured to wait until the other namenode updates its namespace.

21. The cluster of claim 1, wherein each of the namenodes, to prevent stale reads, are configured to submit a coordinated read proposal to the coordination engine when a request is received to read a file that is determined to satisfy selected file name patterns and to enable the file to be read only upon receiving an agreement corresponding to the coordinated read proposal.

22. The cluster of claim 1, further configured to enable a new namenode to be brought into the cluster, wherein the new name node is configured to update the state of its namespace by at least one of:

loading a checkpoint comprising a prior image of the namespace, and

accepting agreements to its namespace from the coordination engine.

46. The cluster of claim 1, wherein the proposals to change the state of the namespace comprise at least one of replicating, deleting and adding data blocks in at least one of the plurality of data nodes.

48. The computing device of claim 46, wherein the first namenode is configured to delay making changes to the state of the namespace until the ordered set of agreements is received from the coordination engine.

49. The computing device of claim 46, wherein the proposals to change the state of the namespace comprise at least one of replicating, deleting and adding data blocks in at least one of the plurality of data nodes.

23-45. -45. (canceled)

47. A computing device configured as a part of a distributed file system, comprising:

a first namenode coupled to a plurality of data nodes, each of which is configured to store data blocks of client files, the first namenode being configured to store a state of a namespace of a cluster of computing nodes, the first namenode being configured to be coupled to a second namenode that is coupled to the plurality of data nodes and that is configured to store the state of the namespace, the first namenode being further configured to respond to a request from a client while the second namenode is responding to another request from another client; and

a first coordination engine coupled to the first namenode, the coordination engine being configured to receive proposals from the first namenode to change the state of the namespace and to generate, in response, an ordered set of agreements that specifies an order in which the first and the second namenodes are to change the state of the namespace.