DISTRIBUTED FILE SYSTEM USING CONSENSUS NODES

• US 20150067004A1
• Filed: 09/30/2013
• Published: 03/05/2015
• Est. Priority Date: 01/12/2005
• Status: Active Grant

• Current Assignee: Cirata Incorporated
• Original Assignee: WANdisco, Inc.
• Inventors: Sundar, Jagane, Aahlad, Yeturu, SHVACHKO, Konstantin V., Parkin, Micahel
• Granted Patent: US 9,424,272 B2
• Time in Patent Office: Days
• US Class Current: 707/827
• CPC Class Codes: G06F 16/166 File name conversion
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/285 Clustering or classification

Abstract

A cluster of nodes in a distributed file system may include; at least two namenodes, each coupled to a plurality of data nodes and each configured to store a state of a namespace of the cluster and each being configured to respond to a request from a client while other(s) of the namenodes are responding to other requests from other clients; and a coordination engine coupled to each of the namenodes. The coordination engine may be configured to receive proposals from the namenodes to change the state of the namespace by replicating, deleting and/or adding data blocks stored in the data nodes and to generate, in response, an ordered set of agreements that specifies an order in which the namenodes are to change the state of the namespace. The namenodes are configured to delay making changes thereto until after the ordered set of agreements is received from the coordination engine.

Claims

1-22. -22. (canceled)

23. A computer-implemented method of implementing a distributed file system comprising a plurality of data nodes configured to store data blocks of client files, the method, comprising:

coupling at least two namenodes to a plurality of data nodes, each of the namenodes being configured to store a state of a namespace of the cluster and each being configured to respond to a request from a client while at least one other of the namenodes is responding to another request from another client;

receiving proposals from the namenodes to change the state of the namespace, and

generating, in response to receiving the proposals, an ordered set of agreements that specifies an order in which the namenodes are to change the state of the namespace, such that the namenodes delay making changes to the state of the namespace until the namenodes receive the ordered set of agreements.

24. The computer-implemented method of claim 23, further comprising the namenodes generating the proposals responsive to clients issuing requests to at least one of replicating and deleting data blocks in the data nodes.

25. The computer-implemented method of claim 23, further comprising assigning a unique global sequence number (GSN) to each agreement of the ordered set of agreements, the GSN specifying an order in which the namenodes are to apply the changes to the state of the namespace.

26. The computer-implemented method of claim 25, further comprising the namenodes applying the changes to the state of the namespace in the order specified by the GSN of the received agreements.

27. The computer-implemented method of claim 23, further comprising storing an image of the namespace and entries specifying updates to the namespace in local storage coupled to each of the plurality of namenodes.

28. The computer-implemented method of claim 23, further comprising configuring only one namenode in the cluster to enable data blocks stored in the plurality of data nodes to be replicated or deleted.

29. The computer-implemented method of claim 28, further comprising configuring the namenode configured to enable replication and deletion of data blocks to periodically issue a block replicator heartbeat message.

30. The computer-implemented method of claim 29, further comprising triggering an election process to elect a new namenode that is to be solely configured to enable replication and deletion of data blocks upon failure of the namenode configured to enable replication and deletion of data blocks to periodically issue the block replicator heartbeat.

31. The method of claim 30 wherein the election process is configured to be initiated by any of the namenodes.

32. The computer-implemented method of claim 23, further comprising configuring each of the namenodes to enable data blocks to be generated and stored in the plurality of data nodes.

33. The computer-implemented method of claim 23, further comprising configuring each of the namenodes to enable data blocks stored on one of the plurality of data nodes to be replicated to a predetermined other one or other ones of the plurality of data nodes.

34. The computer-implemented method of claim 33, further comprising configuring at least one of the data nodes, upon receipt of data blocks, to send the data blocks to another of the plurality of data nodes identified by one of the namenodes.

35. The computer-implemented method of claim 23, further comprising configuring each of the plurality of data nodes to periodically issue a heartbeat message to the namenodes, indicating a current status of the data node.

36. The computer-implemented method of claim 35, further comprising configuring the namenodes to consider a data node from which the periodic heartbeat has not been timely received to have failed.

37. The computer-implemented method of claim 23, further comprising configuring only one of the namenodes to enable data blocks stored in a failed data node to be replicated to another selected one of the plurality of data nodes.

38. The computer-implemented method of claim 23, further comprising configuring each of the namenodes to assign a unique block identifier (ID) to data blocks from within a sub-range of a larger range of block ID numbers that is separate from other respective sub-ranges of the larger range of block IDs from which other ones of the namenodes assign block ID numbers.

39. The computer-implemented method of claim 23, further comprising setting a communication timeout configured such that, if a namenode having received a client request has not responded thereto at an expiry of the communication timeout, the client request is made to another of the namenodes.

40. The computer-implemented method of claim 23, further comprising enabling all of the namenodes in the cluster to actively support client requests simultaneously.

41. The computer-implemented method of claim 23 wherein, upon failure of one of the namenodes, making client requests previously made to the failed namenode to an other one of the namenodes.

42. The computer-implemented method of claim 41 wherein, prior to making client requests to the other namenode, waiting until the other namenode updates its namespace.

43. The computer-implemented method of claim 23, further comprising each of the namenodes, to prevent stale reads, to submit a coordinated read proposal to the coordination engine when a request is received to read a file that is determined to satisfy selected file name patterns and to enable the file to be read only upon receiving an agreement corresponding to the coordinated read proposal.

44. The computer-implemented method of claim 23, further comprising enabling a new namenode to be brought into the cluster, wherein the new name node is configured to update the state of its namespace by at least one of:

loading a checkpoint comprising a prior image of the namespace, and

accepting agreements to its namespace from the coordination engine.

46. The computer-implemented method of claim 23, wherein the received proposals are configured to change the namespace by at least one of replicating, deleting and adding data blocks in at least one of the plurality of data nodes.

45. A non-transitory machine-readable medium having data stored thereon representing sequences of instructions which, when executed by computing devices, cause the computing devices to implementing a distributed file system by:

coupling at least two namenodes to a plurality of data nodes, each of the namenodes being configured to store a state of a namespace of the cluster and each being configured to respond to a request from a client while at least one other of the namenodes is responding to another request from another client;

receiving proposals from the namenodes to change the state of the namespace, and

generating, in response to receiving the proposals, an ordered set of agreements that specifies an order in which the namenodes are to change the state of the namespace, such that the namenodes delay making changes to the state of the namespace until the namenodes receive the ordered set of agreements.

47. The non-transitory machine-readable medium of claim 45, wherein the received proposals are configured to change the namespace by at least one of replicating, deleting and adding data blocks in at least one of the plurality of data nodes.