Managing a global namespace for a distributed filesystem

• US 9,805,054 B2
• Filed: 09/06/2013
• Issued: 10/31/2017
• Est. Priority Date: 11/14/2011
• Status: Active Grant

• Current Assignee: Panzura, Inc.
• Original Assignee: Panzura, Inc.
• Inventors: Davis, Andrew P., Sharpe, Richard, Burrington, Allan
• Primary Examiner(s): Rahman, S M
• Application Number:
  US14/020,493
• Publication Number:
  US 20140006465A1
• Time in Patent Office: 1,516 Days
• Field of Search: 709219
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: G06F 16/1752 based on file chunks
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097

Abstract

The disclosed embodiments disclose techniques for managing a global namespace for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in a cloud storage system; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. Furthermore, a global namespace for the distributed filesystem is also split across these cloud controllers, with each cloud controller “owning” (e.g., managing write accesses for) a distinct portion of the global namespace and maintaining a set of namespace mappings that indicate which portion of the namespace is assigned to each cloud controller. During operation, an initial cloud controller receives a request from a client system to access a target file in the distributed system. This initial cloud controller uses the namespace mappings for the global namespace to determine a preferred cloud controller that will handle the request.

Claims

1. A computer-implemented method for managing a global namespace for a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises;

storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems; and

caching data for the distributed filesystem in each cloud controller;

dynamically managing a global namespace for the distributed filesystem across the two or more cloud controllers by assigning distinct portions of the global namespace across the two or more of the cloud controllers, wherein each cloud controller is configured to manage write accesses for its assigned portion of the global namespace, wherein a set of namespace mappings indicate the portions of the namespace that are assigned to each cloud controller, wherein the cloud controllers collectively change the set of namespace mappings dynamically overtime based on accesses to the distributed filesystem; and

receiving at an initial cloud controller a request from a client system to access a target file in the distributed filesystem;

using a namespace mapping for the global namespace to determine a preferred cloud controller for the request by;

determining from the set of namespace mappings that a target cloud controller manages the portion of the namespace that includes the target file;

determining the location and current load of the target cloud controller;

determining a local cloud controller that is co-located with the client system and the initial cloud controller, operational, and sufficiently lightly loaded to serve as the preferred cloud controller;

determining a remote cloud controller that is lightly loaded, operational, and located at a different geographic location as well as the amount of network bandwidth between the local cluster and the different geographic location; and

based on the location of the target cloud controller and determined network and load characteristics for the local cloud controller, the remote cloud controller, and the target cloud controller, choosing one of the target cloud controller, local cloud controller, and remote cloud controller to serve as the preferred cloud controller; and

wherein every cloud controller for the distributed filesystem can cache the target file and provide write access to clients seeking to modify the target file regardless of which cloud controller in the distributed filesystem has been assigned to manage the portion of the global namespace that includes the target file.

2. The computer-implemented method of claim 1, wherein each cloud controller caches the set of namespace mappings for the global namespace;

and

wherein each cloud controller is configured to notify the other cloud controllers for the distributed filesystem of changes to the set of namespace mappings.

3. The computer-implemented method of claim 2, wherein using the namespace mapping to determine the preferred cloud controller comprises:

determining from the set of namespace mappings that the preferred cloud controller manages the portion of the namespace that includes the target file; and

referring the client system to the preferred cloud controller;

wherein upon receiving the referral, the client system is configured to connect to the preferred cloud controller and send the request to the preferred cloud controller; and

wherein the preferred cloud controller services the request.

4. The computer-implemented method of claim 3, wherein determining the preferred cloud controller for the request comprises determining that the preferred cloud controller is operational and co-located with the client system and the initial cloud controller.

5. The computer-implemented method of claim 3, wherein determining the preferred cloud controller for the request comprises:

determining that the preferred cloud controller is operational and located at a different geographic location than the client system and the initial cloud controller;

determining that no operational and lightly-loaded cloud controller is available in the local cluster; and

determining that there is sufficient network bandwidth between the local cluster and the different geographic location to service the request.

6. The computer-implemented method of claim 2, wherein using the namespace mapping to determine the preferred cloud controller comprises:

determining from the set of namespace mappings that the target cloud controller manages the portion of the namespace that includes the target file;

determining that directly connecting the client system to the target cloud controller is unfavorable; and

referring the client system to the preferred cloud controller;

wherein upon receiving the referral, the client system is configured to connect to the preferred cloud controller and send the request to the preferred cloud controller;

wherein the preferred cloud controller is configured to contact the target cloud controller to request a write lock that is associated with the request; and

wherein, upon receiving the write lock from the target cloud controller, the preferred cloud controller uses the write lock to service the request.

7. The computer-implemented method of claim 6, wherein determining the preferred cloud controller for the request comprises:

determining that the target cloud controller is co-located with the client system, the initial cloud controller, and the preferred cloud controller; and

determining that the target cloud controller is unavailable to client connections.

8. The computer-implemented method of claim 6, wherein determining the preferred cloud controller for the request comprises:

determining that the target cloud controller is operational and located at a different location than the client system and the initial cloud controller;

determining that there is insufficient network bandwidth between the local cluster and the different location to service the request efficiently; and

determining that the preferred cloud controller is an operational and lightly-loaded cloud controller that is co-located with the client system.

9. The computer-implemented method of claim 6, wherein the method further comprises:

determining at a subsequent time that, based on tracked access patterns for the client system, the preferred cloud controller, and the target cloud controller, that the performance benefits of also connecting the client system directly to the target cloud controller outweigh the additional overhead of maintaining multiple connections for the client system; and

referring the client system to the target cloud controller;

wherein upon receiving the second referral, the client system is configured to connect to the target cloud controller and thereby maintain separate, simultaneous connections with the preferred cloud controller and the target cloud controller; and

wherein maintaining multiple simultaneous connections to different cloud controllers for the distributed filesystem consumes additional cloud controller and client system resources but improves the file access performance for the client system.

10. The computer-implemented method of claim 6, wherein the method further comprises:

tracking a set of accesses made by one or more client systems to determine patterns in data access and grouping for the distributed filesystem;

re-assigning a portion of the global namespace that includes the target file from the target cloud controller to the preferred cloud controller based on the determined patterns; and

updating the namespace mapping to reflect the reassignment;

wherein re-assigning the portion of the global namespace facilitates reducing the average load for the cloud controllers of the distributed filesystem and increasing the file access performance for the client system.

11. The computer-implemented method of claim 10,

wherein the determined patterns indicate a set of user and project files that are related;

wherein the portions of the global namespace associated with the set of user and project files are re-assigned to the preferred cloud controller; and

wherein re-assigning the portions of the global namespace associated with the set of user and project files to a single cloud controller facilitates improving file cache hit rates, reducing the number of connections to the distributed filesystem that are needed by the client systems accessing that portion of the global namespace, and improving the scalability of the distributed filesystem.

12. The computer-implemented method of claim 10,

wherein the determined patterns indicate the re-location of the client system from a first site that includes the target cloud controller to a different site that includes the preferred cloud controller;

and

wherein re-assigning the portion of the global namespace to the preferred cloud controller reduces the number of cloud controllers involved in servicing the client system'"'"'s requests.

13. The computer-implemented method of claim 10,

wherein the determined patterns indicate that the target cloud controller is overloaded;

and

wherein re-assigning the portion of the global namespace to the preferred cloud controller reduces the load for the target cloud controller.

14. The computer-implemented method of claim 10,

wherein the set of cloud controllers that manage the distributed filesystem track ongoing changes for the distributed filesystem and dynamically adjust the mapping of clients systems to cloud controllers and the assignment of namespace mappings to cloud controllers to improve and balance file access performance for the distributed filesystem;

wherein dynamically adjusting the mapping of clients to cloud controllers and the assignment of namespace mappings comprises;

selectively sending referrals to additional cloud controllers to client systems;

selectively disconnecting some client system connections from cloud controllers;

selectively encouraging or discouraging client system accesses that involve multiple cloud controllers and transferred write locks using referrals to cloud controllers; and

migrating the assignment of namespace mappings between cloud controllers to group namespace mappings for related files and directories.

15. The computer-implemented method of claim 2, wherein determining the preferred cloud controller for the request comprises considering:

the location and availability of a cloud controller that manages the portion of the namespace that includes the target file;

the configuration of a site that includes the client system;

the configuration of a second site that includes the cloud controller;

the bandwidth between the client system and the cloud controller;

load information for the cloud controller and one or more cloud controllers that are co-located with the client system;

other files that are in the portion of the namespace that is managed by the cloud controller; and

global cache knowledge for the distributed filesystem.

16. The computer-implemented method of claim 2,

wherein the target file that is being written to the distributed filesystem is a new file that is not yet associated with a portion of the global namespace;

and

wherein determining the preferred cloud controller for the request comprises considering;

the load of the cloud controllers that are co-located with the client system;

the namespace loads for the cloud controllers that are co-located with the client system;

the initial cloud controller that is contacted by the client system;

the location of the client system;

information associated with a user that is associated with the request; and

a locality policy that specifies selection criteria for choosing a cloud controller.

17. The computer-implemented method of claim 2, wherein collectively managing the data for the distributed filesystem further comprises:

maintaining in each cloud controller a metadata hierarchy that reflects the current state of the distributed filesystem, wherein changes to the metadata for the distributed filesystem are synchronized across the cloud controllers for the distributed filesystem to ensure that the clients of the distributed filesystem share a consistent view of the files in the distributed filesystem; and

collectively presenting the global namespace to the clients of the distributed filesystem via the two or more cloud controllers, wherein the clients access the distributed filesystem via the cloud controllers, wherein the file data for the distributed filesystem is stored in the cloud storage systems, wherein cloud controllers cache in their local storage devices a subset of the file data from the remote cloud storage system that is being actively accessed by each respective cloud controller'"'"'s clients, wherein new file data received by each cloud controller from its clients is written to the cloud storage systems.

18. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for managing a global namespace for a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises;

storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems; and

caching data for the distributed filesystem in each cloud controller;

dynamically managing a global namespace for the distributed file system across the two or more cloud controllers by assigning distinct portions of the global namespace across the two or more of the cloud controllers, wherein each cloud controller is configured to manage write accesses for its assigned portion of the global namespace, wherein a set of namespace mappings indicate the portions of the namespace that are assigned to each cloud controller, wherein the cloud controllers collectively change the set of namespace mappings dynamically overtime based on accesses to the distributed filesystem; and

receiving at an initial cloud controller a request from a client system to access a target file in the distributed filesystem;

using a namespace mapping for the global namespace to determine a preferred cloud controller for the request by;

determining from the set of namespace mappings that a target cloud controller manages the portion of the namespace that includes the target file;

determining the location and current load of the target cloud controller;

determining a local cloud controller that is co-located with the client system and the initial cloud controller, operational, and sufficiently lightly loaded to serve as the preferred cloud controller;

determining a remote cloud controller that is lightly loaded, operational, and located at a different geographic location as well as the amount or network bandwidth between the local cluster and the different geographic location; and

based on the location of the target cloud controller and determined network and load characteristics for the local cloud controller, the remote cloud controller, and the target cloud controller, choosing one of the target cloud controller, local cloud controller, and remote cloud controller to serve as the preferred cloud controller; and

wherein every cloud controller for the distributed filesystem can cache the target file and provide write access to clients seeking to modify the target file regardless of which cloud controller in the distributed filesystem has been assigned to manage the portion of the global namespace that includes the target file.

19. A cloud controller that manages a portion of a global namespace for a distributed filesystem, comprising:

a processor;

a storage mechanism that stores metadata for the distributed filesystem; and

a storage management mechanism;

wherein two or more cloud controllers collectively manage the data of the distributed filesystem;

wherein a global namespace for the distributed filesystem is dynamically managed by two or more cloud controllers;

wherein splitting the global namespace comprises assigning distinct portions of the global namespace across the two or more cloud controllers;

wherein each cloud controller is configured to manage write accesses for its assigned portion of the global namespace;

wherein a set of namespace mappings indicate the portions of the namespace that are assigned to each cloud controller;

wherein the cloud controllers collectively change the set of namespace mappings dynamically over time based on accesses to the distributed filesystem;

wherein the cloud controller is configured to receive a request from a client system to access a target file in the distributed filesystem;

wherein the cloud controller is further configured to use a namespace mapping for the global namespace to determine a preferred cloud controller for the request by;

determining from the set of namespace mappings that a target cloud controller manages the portion of the namespace that includes the target file;

determining the location and current load of the target cloud controller;

determining a local cloud controller that is co-located with the client system and the initial cloud controller, operational, and sufficiently lightly loaded to serve as the preferred cloud controller;

determining a remote cloud controller that is lightly loaded, operational, and located at a different geographic location as well as the amount of network bandwidth between the local cluster and the different geographic location; and

based on the location of the target cloud controller and determined network and load characteristics for the local cloud controller, the remote cloud controller, and the target cloud controller, choosing one of the target cloud controller, local cloud controller, and remote cloud controller to serve as the preferred cloud controller; and

wherein every cloud controller for the distributed filesystem can cache the target file and provide write access to clients seeking to modify the target file regardless of which cloud controller in the distributed filesystem has been assigned to manage the portion of the global namespace that includes the target file.