Accessing cached data from a peer cloud controller in a distributed filesystem
First Claim
1. A computer-implemented method for accessing cached data from a peer cloud controller in a distributed filesystem, the method comprising:
- collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises;
collectively presenting a unified namespace for the distributed filesystem to the clients of the distributed filesystem via the two or more cloud controllers, wherein the clients can only access the distributed filesystem via the cloud controllers, wherein the file data for the distributed filesystem is stored in a remote cloud storage system using fixed-size cloud files, wherein each cloud controller caches a subset of the file data from the remote cloud storage system that is being actively accessed by that cloud controller'"'"'s respective clients, wherein all new file data received by each cloud controller from its clients is written to the remote cloud storage system via the receiving cloud controller;
maintaining at each cloud controller a copy of the complete metadata for all of the files stored in the distributed filesystem, wherein each cloud controller communicates any changes to the metadata for the distributed filesystem to the other cloud controllers for the distributed filesystem to ensure that the clients of the distributed filesystem share a consistent view of each file in the distributed filesystem;
upon receiving in a cloud controller new data from a client, storing the new file data for the distributed filesystem as a cloud file in the remote cloud storage system, wherein the cloud file is sent from the cloud controller to the remote cloud storage system as part of an incremental data snapshot; and
upon receiving confirmation that the cloud file has been successfully stored in the remote cloud storage system, sending from the cloud controller an incremental metadata snapshot that includes new metadata for the distributed filesystem that describes the new data, wherein the incremental metadata snapshot is received by the other cloud controllers of the distributed filesystem;
receiving at the cloud controller a request from the client for a data block for a file in the distributed filesystem;
determining that the requested data block is not currently cached in the cloud controller;
determining that the peer cloud controller in the distributed filesystem is likely to cache the requested data block; and
sending a peer cache request for the requested data block from the cloud controller to the peer cloud controller.
9 Assignments
0 Petitions
Accused Products
Abstract
The disclosed embodiments provide a system that archives data for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. During operation, a cloud controller receives a request from a client for a data block of a file stored in the distributed filesystem. Upon determining that the requested data block is not currently cached in the cloud controller, the cloud controller sends a peer cache request for the requested data block to a peer cloud controller in the distributed filesystem.
42 Citations
20 Claims
-
1. A computer-implemented method for accessing cached data from a peer cloud controller in a distributed filesystem, the method comprising:
-
collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises; collectively presenting a unified namespace for the distributed filesystem to the clients of the distributed filesystem via the two or more cloud controllers, wherein the clients can only access the distributed filesystem via the cloud controllers, wherein the file data for the distributed filesystem is stored in a remote cloud storage system using fixed-size cloud files, wherein each cloud controller caches a subset of the file data from the remote cloud storage system that is being actively accessed by that cloud controller'"'"'s respective clients, wherein all new file data received by each cloud controller from its clients is written to the remote cloud storage system via the receiving cloud controller; maintaining at each cloud controller a copy of the complete metadata for all of the files stored in the distributed filesystem, wherein each cloud controller communicates any changes to the metadata for the distributed filesystem to the other cloud controllers for the distributed filesystem to ensure that the clients of the distributed filesystem share a consistent view of each file in the distributed filesystem; upon receiving in a cloud controller new data from a client, storing the new file data for the distributed filesystem as a cloud file in the remote cloud storage system, wherein the cloud file is sent from the cloud controller to the remote cloud storage system as part of an incremental data snapshot; and upon receiving confirmation that the cloud file has been successfully stored in the remote cloud storage system, sending from the cloud controller an incremental metadata snapshot that includes new metadata for the distributed filesystem that describes the new data, wherein the incremental metadata snapshot is received by the other cloud controllers of the distributed filesystem; receiving at the cloud controller a request from the client for a data block for a file in the distributed filesystem; determining that the requested data block is not currently cached in the cloud controller; determining that the peer cloud controller in the distributed filesystem is likely to cache the requested data block; and sending a peer cache request for the requested data block from the cloud controller to the peer cloud controller. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for accessing cached data from a peer cloud controller in a distributed filesystem, the method comprising:
-
collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises; collectively presenting a unified namespace for the distributed filesystem to the clients of the distributed filesystem via the two or more cloud controllers, wherein the clients can only access the distributed filesystem via the cloud controllers, wherein the file data for the distributed filesystem is stored in a remote cloud storage system using fixed-size cloud files, wherein each cloud controller caches a subset of the file data from the remote cloud storage system that is being actively accessed by that cloud controller'"'"'s respective clients, wherein all new file data received by each cloud controller from its clients is written to the remote cloud storage system via the receiving cloud controller; maintaining at each cloud controller a copy of the complete metadata for all of the files stored in the distributed filesystem, wherein each cloud controller communicates any changes to the metadata for the distributed filesystem to the other cloud controllers for the distributed filesystem to ensure that the clients of the distributed filesystem share a consistent view of each file in the distributed filesystem; upon receiving in a cloud controller new data from a client, storing the new file data for the distributed filesystem as a cloud file in the remote cloud storage system, wherein the cloud file is sent from the cloud controller to the remote cloud storage system as part of an incremental data snapshot; and upon receiving confirmation that the cloud file has been successfully stored in the remote cloud storage system, sending from the cloud controller an incremental metadata snapshot that includes new metadata for the distributed filesystem that describes the new data, wherein the incremental metadata snapshot is received by the other cloud controllers of the distributed filesystem; receiving at the cloud controller a request from the client for a data block for a file in the distributed filesystem; determining that the requested data block is not currently cached in the cloud controller; determining that the peer cloud controller in the distributed filesystem is likely to cache the requested data block; and sending a peer cache request for the requested data block from the cloud controller to the peer cloud controller. - View Dependent Claims (15, 16, 17, 18, 19)
-
-
20. A cloud controller that accesses cached data from a peer cloud controller in a distributed filesystem, comprising:
-
a processor; a storage mechanism that stores metadata for the distributed filesystem; and a storage management mechanism; wherein two or more cloud controllers collectively manage the data of the distributed filesystem, wherein collectively managing the data comprises; collectively presenting a unified namespace for the distributed filesystem to the clients of the distributed filesystem via the two or more cloud controllers, wherein the clients can only access the distributed filesystem via the cloud controllers, wherein the file data for the distributed filesystem is stored in a remote cloud storage system using fixed-size cloud files, wherein each cloud controller caches a subset of the file data from the remote cloud storage system that is being actively accessed by that cloud controller'"'"'s respective clients, wherein all new file data received by each cloud controller from its clients is written to the remote cloud storage system via the receiving cloud controller; maintaining at each cloud controller a copy of the complete metadata for all of the files stored in the distributed filesystem, wherein each cloud controller communicates any changes to the metadata for the distributed filesystem to the other cloud controllers for the distributed filesystem to ensure that the clients of the distributed filesystem share a consistent view of each file in the distributed filesystem; upon receiving in a cloud controller new data from a client, storing the new file data for the distributed filesystem as a cloud file in the remote cloud storage system, wherein the cloud file is sent from the cloud controller to the remote cloud storage system as part of an incremental data snapshot; and upon receiving confirmation that the cloud file has been successfully stored in the remote cloud storage system, sending from the cloud controller an incremental metadata snapshot that includes new metadata for the distributed filesystem that describes the new data, wherein the incremental metadata snapshot is received by the other cloud controllers of the distributed filesystem; wherein the storage management mechanism is configured to; receive a request from the client for a data block for a file in the distributed filesystem; determine that the requested data block is not currently cached in the cloud controller; determine that the peer cloud controller in the distributed filesystem is likely to cache the requested data block; and send a peer cache request for the requested data block from the cloud controller to the peer cloud controller.
-
Specification