Managing a global namespace for a distributed filesystem
First Claim
1. A computer-implemented method for managing a global namespace for a distributed filesystem, the method comprising:
- collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises;
storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems; and
caching data for the distributed filesystem in each cloud controller;
dynamically managing a global namespace for the distributed filesystem across the two or more cloud controllers by assigning distinct portions of the global namespace across the two or more of the cloud controllers, wherein each cloud controller is configured to manage write accesses for its assigned portion of the global namespace, wherein a set of namespace mappings indicate the portions of the namespace that are assigned to each cloud controller, wherein the cloud controllers collectively change the set of namespace mappings dynamically over time based on accesses to the distributed filesystem; and
receiving at an initial cloud controller a request from a client system to access a target file in the distributed filesystem, wherein the initial cloud controller is configured to use a namespace mapping for the global namespace to determine a preferred cloud controller for the request by;
determining from the set of namespace mappings that a target cloud controller manages the portion of the namespace that includes the target file;
determining that directly connecting the client system to the target cloud controller is unfavorable;
determining the preferred cloud controller to be a viable intermediary to the target cloud controller; and
referring the client system to the preferred cloud controller;
wherein upon receiving the referral, the client system is configured to connect to the preferred cloud controller and send the request to the preferred cloud controller;
wherein the preferred cloud controller is configured to contact the target cloud controller to request a write lock that is associated with the request; and
wherein, upon receiving the write lock from the target cloud controller, the preferred cloud controller uses the write lock to service the request.
7 Assignments
0 Petitions
Accused Products
Abstract
The disclosed embodiments disclose techniques for managing a global namespace for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in a cloud storage system; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. Furthermore, a global namespace for the distributed filesystem is also split across these cloud controllers, with each cloud controller “owning” (e.g., managing write accesses for) a distinct portion of the global namespace and maintaining a set of namespace mappings that indicate which portion of the namespace is assigned to each cloud controller. During operation, an initial cloud controller receives a request from a client system to access a target file in the distributed system. This initial cloud controller uses the namespace mappings for the global namespace to determine a preferred cloud controller that will handle the request.
42 Citations
16 Claims
-
1. A computer-implemented method for managing a global namespace for a distributed filesystem, the method comprising:
-
collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises; storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems; and caching data for the distributed filesystem in each cloud controller; dynamically managing a global namespace for the distributed filesystem across the two or more cloud controllers by assigning distinct portions of the global namespace across the two or more of the cloud controllers, wherein each cloud controller is configured to manage write accesses for its assigned portion of the global namespace, wherein a set of namespace mappings indicate the portions of the namespace that are assigned to each cloud controller, wherein the cloud controllers collectively change the set of namespace mappings dynamically over time based on accesses to the distributed filesystem; and receiving at an initial cloud controller a request from a client system to access a target file in the distributed filesystem, wherein the initial cloud controller is configured to use a namespace mapping for the global namespace to determine a preferred cloud controller for the request by; determining from the set of namespace mappings that a target cloud controller manages the portion of the namespace that includes the target file; determining that directly connecting the client system to the target cloud controller is unfavorable; determining the preferred cloud controller to be a viable intermediary to the target cloud controller; and referring the client system to the preferred cloud controller; wherein upon receiving the referral, the client system is configured to connect to the preferred cloud controller and send the request to the preferred cloud controller; wherein the preferred cloud controller is configured to contact the target cloud controller to request a write lock that is associated with the request; and wherein, upon receiving the write lock from the target cloud controller, the preferred cloud controller uses the write lock to service the request. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for managing a global namespace for a distributed filesystem, the method comprising:
-
collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises; storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems; and caching data for the distributed filesystem in each cloud controller; dynamically managing a global namespace for the distributed filesystem across the two or more cloud controllers by assigning distinct portions of the global namespace across the two or more of the cloud controllers, wherein each cloud controller is configured to manage write accesses for its assigned portion of the global namespace, wherein a set of namespace mappings indicate the portions of the namespace that are assigned to each cloud controller, wherein the cloud controllers collectively change the set of namespace mappings dynamically over time based on accesses to the distributed filesystem; and receiving at an initial cloud controller a request from a client system to access a target file in the distributed filesystem, wherein the initial cloud controller is configured to use a namespace mapping for the global namespace to determine a preferred cloud controller for the request by; determining from the set of namespace mappings that a target cloud controller manages the portion of the namespace that includes the target file; determining that directly connecting the client system to the target cloud controller is unfavorable; determining the preferred cloud controller to be a viable intermediary to the target cloud controller; and referring the client system to the preferred cloud controller; wherein upon receiving the referral, the client system is configured to connect to the preferred cloud controller and send the request to the preferred cloud controller; wherein the preferred cloud controller is configured to contact the target cloud controller to request a write lock that is associated with the request; and wherein, upon receiving the write lock from the target cloud controller, the preferred cloud controller uses the write lock to service the request.
-
-
16. A cloud controller that manages a portion of a global namespace for a distributed filesystem, comprising:
-
a processor; a storage mechanism that stores metadata and namespace mappings for the distributed filesystem; and a storage management mechanism; wherein two or more cloud controllers collectively manage the data of the distributed filesystem; wherein a global namespace for the distributed filesystem is dynamically managed by two or more cloud controllers by assigning distinct portions of the global namespace across the two or more of the cloud controllers, wherein each cloud controller is configured to manage write accesses for its assigned portion of the global namespace, wherein a set of namespace mappings indicate the portions of the namespace that are assigned to each cloud controller, wherein the cloud controllers collectively change the set of namespace mappings dynamically over time based on accesses to the distributed filesystem; wherein the cloud controller is configured to receive a request from a client system to access a target file in the distributed filesystem, wherein the cloud controller is configured to use a namespace mapping for the global namespace to determine a preferred cloud controller for the request by; determining from the set of namespace mappings that a target cloud controller manages the portion of the namespace that includes the target file; determining that directly connecting the client system to the target cloud controller is unfavorable; determining the preferred cloud controller to be a viable intermediary to the target cloud controller; and referring the client system to the preferred cloud controller; wherein upon receiving the referral, the client system is configured to connect to the preferred cloud controller and send the request to the preferred cloud controller; wherein the preferred cloud controller is configured to contact the target cloud controller to request a write lock that is associated with the request; and wherein, upon receiving the write lock from the target cloud controller, the preferred cloud controller uses the write lock to service the request.
-
Specification