Geographically-distributed file system using coordinated namespace replication over a wide area network
First Claim
1. A computer-implemented method of maintaining a state of a single namespace of a distributed file system in a local area network (LAN), the method comprising:
- storing the state of the single namespace of the distributed file system in respective local persistent storage coupled to each of a plurality of simultaneously-active peer server node computing devices coupled to the LAN;
receiving, in at least some of the plurality of server node computing devices, at least one request to change the state of the single namespace from at least some of a plurality of data node computing devices coupled to the LAN;
generating, by each of the plurality of simultaneously-active peer server node computing devices having received the at least one request to change the state of the single namespace and for each of the at least one received request, a corresponding proposal to make the change to the state of the single namespace;
sending the generated proposals to a single coordination engine, the single coordination engine being coupled to the LAN and to each of the plurality of server node computing devices;
receiving, by the plurality of simultaneously-active peer server node computing devices and for at least some of the generated proposals, an ordered set of namespace modification agreements sent from the single coordination engine, the ordered set of namespace modification agreements specifying an order in which the plurality simultaneously-active peer server node computing devices are to update the state of the single namespace stored in the respective local persistent storages coupled thereto; and
updating, by the plurality of server node computing devices, the state of the single namespace of the distributed file system stored in the respective local persistent storages, in an order specified by the ordered set of namespace modification agreements received from the single coordination engine, such that the state of the single namespace stored in each of the respective local persistent storages is brought into consistency across the plurality of simultaneously-active peer server node computing devices.
1 Assignment
0 Petitions
Accused Products
Abstract
A cluster of nodes implements a single distributed file system comprises at least first and second data centers and a coordination engine process. The first data center may comprise first DataNodes configured to store data blocks of client files, and first NameNodes configured to update a state of a namespace of the cluster. The second data center, geographically remote from and coupled to the first data center by a wide area network, may comprise second DataNodes configured to store data blocks of client files, and second NameNodes configured to update the state of the namespace. The first and second NameNodes are configured to update the state of the namespace responsive to data blocks being written to the DataNodes. The coordination engine process spans the first and second NameNodes and coordinates updates to the namespace stored such that the state thereof is maintained consistent across the first and second data centers.
94 Citations
20 Claims
-
1. A computer-implemented method of maintaining a state of a single namespace of a distributed file system in a local area network (LAN), the method comprising:
-
storing the state of the single namespace of the distributed file system in respective local persistent storage coupled to each of a plurality of simultaneously-active peer server node computing devices coupled to the LAN; receiving, in at least some of the plurality of server node computing devices, at least one request to change the state of the single namespace from at least some of a plurality of data node computing devices coupled to the LAN; generating, by each of the plurality of simultaneously-active peer server node computing devices having received the at least one request to change the state of the single namespace and for each of the at least one received request, a corresponding proposal to make the change to the state of the single namespace; sending the generated proposals to a single coordination engine, the single coordination engine being coupled to the LAN and to each of the plurality of server node computing devices; receiving, by the plurality of simultaneously-active peer server node computing devices and for at least some of the generated proposals, an ordered set of namespace modification agreements sent from the single coordination engine, the ordered set of namespace modification agreements specifying an order in which the plurality simultaneously-active peer server node computing devices are to update the state of the single namespace stored in the respective local persistent storages coupled thereto; and updating, by the plurality of server node computing devices, the state of the single namespace of the distributed file system stored in the respective local persistent storages, in an order specified by the ordered set of namespace modification agreements received from the single coordination engine, such that the state of the single namespace stored in each of the respective local persistent storages is brought into consistency across the plurality of simultaneously-active peer server node computing devices. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A system of maintaining a state of a single namespace of a distributed file system in a local area network (LAN), the system comprising:
-
a plurality of simultaneously-active peer server node computing devices coupled to the LAN; a respective local persistent storage coupled to each of the plurality of server node computing devices, each respective local persistent storage being configured to store the state of the single namespace of the distributed file system; a plurality of data node computing devices coupled to the LAN, each being configured to request changes to the state of the single namespace from one of the plurality of server node computing devices; wherein each of the plurality of simultaneously-active peer server node computing devices are configured to, upon receiving requests to change the state of the single namespace from one of the plurality of data node computing devices; generate, for each received request, a corresponding proposal to make the requested change to the state of the single namespace; and send the generated proposals to a single coordination engine that is coupled to the LAN and to each of the plurality of server node computing devices; receive, from the single coordination engine, an ordered set of namespace modification agreements that specifies an order in which the plurality simultaneously-active peer server node computing devices are to update the state of the single namespace stored in the respective local persistent storages coupled thereto; and update, by the plurality of simultaneously-active peer server node computing devices, the state of the single namespace of the distributed file system stored in the respective local persistent storages, in an order specified by the ordered set of namespace modification agreements received from the single coordination engine, such that the state of the single namespace stored in the respective local persistent storages is brought into consistency across the plurality of simultaneously-active peer server node computing devices. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20)
-
Specification