GEOGRAPHICALLY-DISTRIBUTED FILE SYSTEM USING COORDINATED NAMESPACE REPLICATION OVER A WIDE AREA NETWORK

• US 20170193002A1
• Filed: 09/26/2016
• Published: 07/06/2017
• Est. Priority Date: 03/31/2014
• Status: Active Grant

• Current Assignee: Cirata Incorporated
• Original Assignee: WANdisco, Inc.
• Inventors: SHVACHKO, Konstantin V., AAHLAD, Yeturu, SUNDAR, Jagane, JELIAZKOV, Plamen Jeliazkov
• Granted Patent: US 10,795,863 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

A cluster of nodes implements a single distributed file system comprises at least first and second data centers and a coordination engine process. The first data center may comprise first DataNodes configured to store data blocks of client files, and first NameNodes configured to update a state of a namespace of the cluster. The second data center, geographically remote from and coupled to the first data center by a wide area network, may comprise second DataNodes configured to store data blocks of client files, and second NameNodes configured to update the state of the namespace. The first and second NameNodes are configured to update the state of the namespace responsive to data blocks being written to the DataNodes. The coordination engine process spans the first and second NameNodes and coordinates updates to the namespace stored such that the state thereof is maintained consistent across the first and second data centers.

Claims

1. A computer-implemented method of maintaining a state of a namespace of a distributed file system in a local area network (LAN), the method comprising:

storing the state of the namespace of the distributed file system in respective persistent storage coupled to each of a plurality of server node computing devices coupled to the LAN;

receiving, in at least some of the plurality of server node computing devices, at least one request to change the state of the namespace from at least some of a plurality of data node computing devices coupled to the LAN;

generating, by each of the plurality of server node computing nodes having received the at least one request to change the state of the namespace and for each of the at least one received request, a corresponding proposal to make the change to the state of the namespace;

sending the generated proposals to a coordination engine process that is coupled to the LAN and to each of the plurality of server node computing devices;

receiving, by the plurality of server node computing devices and for at least some of the generated proposals, an ordered set of agreements sent from the coordination engine process, the ordered set of agreements specifying an order in which the plurality server node computing devices are to update the state of the namespace stored in the respective persistent storages coupled thereto; and

updating, by the plurality of server node computing devices, the state of the namespace of the distributed file system stored in the respective persistent storages, in an order specified by the ordered set of agreements received from the coordination engine process.

2. The computer-implemented method of claim 1, wherein the coordination process comprises a coordinating engine agent for each of the plurality of server node computing devices.

3. The computer-implemented method of claim 1, wherein updating the state of the namespace by one of the plurality of server node computing devices is carried out while one or more others of the plurality of server node computing devices is also updating the state of the namespace.

4. The computer-implemented method of claim 1, wherein the distributed file system comprises a version of the Hadoop Distributed File System (HDFS) and wherein at least some of the plurality of server node computing devices comprises an HDFS NameNode.

5. The computer-implemented method of claim 1, wherein updating is performed by at least some of the plurality of server node computing devices at different times, but in the order specified by the ordered set of agreements received from the coordination engine process.

6. The computer-implemented method of claim 1, wherein not all of the persistent storages coupled to the plurality of server node computing devices store a same state of the namespace at a same time.

7. The computer-implemented method of claim 1, wherein the respective persistent storages coupled to the plurality of server node computing devices are not a shared resource accessible to each of the plurality of server node computing devices.

8. The computer-implemented method of claim 1, wherein the at least one request to change the state of the namespace comprises a request to one of add, delete or change a data block, file or directory.

9. The computer-implemented method of claim 1, wherein the ordered set of agreements sent from the coordination engine process is ordered according to a Global Sequence Number (GSN), configured as a monotonically increasing number.

10. The computer-implemented method of claim 1, further comprising one of the plurality of data node computing devices:

learning of the GSN of a latest agreement processed by the server node computing device to which it has been requesting changes to the state of the namespace;

changing to a selected other one of the plurality of server node computing devices;

waiting until the selected server node computing device has processed a request associated with the learned GSN before requesting the selected server node computing device to change the state of the namespace; and

submitting a new request to the selected server node computing device to change the state of the namespace after the selected server node computing device has processed the request associated with the learned GSN.

11. A system of maintaining a state of a namespace of a distributed file system in a local area network (LAN), the system comprising:

a plurality of server node computing devices coupled to the LAN;

a respective persistent storage coupled to each of the plurality of server node computing devices, each respective persistent storage being configured to store the state of the namespace of the distributed file system;

a plurality of data node computing devices coupled to the LAN, each being configured to request changes to the state of the namespace from one of the plurality of server node computing devices;

wherein each of the plurality of server node computing devices are configured to, upon receiving requests to change the state of the namespace from one of the plurality of data node computing devices;

generate, for each received request, a corresponding proposal to make the requested change to the state of the namespace; and

send the generated proposals to a coordination engine process that is coupled to the LAN and to each of the plurality of server node computing devices;

receive, from the coordination engine process, an ordered set of agreements that specifies an order in which the plurality server node computing devices are to update the state of the namespace stored in the respective persistent storages coupled thereto; and

update, by the plurality of server node computing devices, the state of the namespace of the distributed file system stored in the respective persistent storages, in an order specified by the ordered set of agreements received from the coordination engine process.

12. The system of claim 11, wherein the coordination process comprises a coordinating engine agent for each of the plurality of server node computing devices.

13. The system of claim 11, wherein one or more of the plurality of server node computing devices is configured to update the state of the namespace while one or more others of the plurality of server node computing devices is also updating the state of the namespace.

14. The system of claim 11, wherein the distributed file system comprises a version of the Hadoop Distributed File System (HDFS) and wherein at least some of the plurality of server node computing devices comprises an HDFS NameNode.

15. The system of claim 11, wherein the plurality of server node computing devices are configured to update the state of the namespace at different times, but in the order specified by the ordered set of agreements received from the coordination engine process.

16. The system of claim 11, wherein not all of the persistent storages coupled to the plurality of server node computing devices store a same state of the namespace at a same time.

17. The system of claim 11, wherein the respective persistent storages coupled to the plurality of server node computing devices are not a shared resource accessible to each of the plurality of server node computing devices.

18. The system of claim 11, wherein each of the requests to change the state of the namespace comprises a request to one of add, delete or change a data block, file or directory.

19. The system of claim 11, wherein the ordered set of agreements sent from the coordination engine process is ordered according to a Global Sequence Number (GSN), configured as a monotonically increasing number.

20. The system of claim 11, wherein one of the plurality of data node computing devices is further configured to:

learn of the GSN of a latest agreement processed by the server node computing device to which it has been requesting changes to the state of the namespace;

change to a selected other one of the plurality of server node computing devices;

wait until the selected server node computing device has processed a request associated with the learned GSN before requesting the selected server node computing device to change the state of the namespace; and

submit a new request to the selected server node computing device to change the state of the namespace after the selected server node computing device has processed the request associated with the learned GSN.