ACCESSING CACHED DATA FROM A PEER CLOUD CONTROLLER IN A DISTRIBUTED FILESYSTEM

• US 20130117240A1
• Filed: 12/21/2012
• Published: 05/09/2013
• Est. Priority Date: 05/03/2010
• Status: Active Grant

• Current Assignee: Panzura, Inc.
• Original Assignee: Panzura, Inc.
• Inventors: Taylor, John Richard, Chou, Randy Yen-pang, Davis, Andrew P.
• Granted Patent: US 8,805,968 B2
• Time in Patent Office: Days
• US Class Current: 707/690
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/182 Distributed file systems

Abstract

The disclosed embodiments provide a system that archives data for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. During operation, a cloud controller receives a request from a client for a data block of a file stored in the distributed filesystem. Upon determining that the requested data block is not currently cached in the cloud controller, the cloud controller sends a peer cache request for the requested data block to a peer cloud controller in the distributed filesystem.

Claims

1. A computer-implemented method for accessing cached data from a peer cloud controller in a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises;

upon receiving in a cloud controller new data from a client, sending from the cloud controller an incremental metadata snapshot for the new data, wherein the incremental metadata snapshot is received by the other cloud controllers of the distributed filesystem;

storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems; and

sending an incremental data snapshot containing the new data from the cloud controller to a cloud storage system;

receiving at the cloud controller a request from the client for a data block for a file in the distributed filesystem;

determining that the requested data block is not currently cached in the cloud controller; and

sending a peer cache request for the requested data block from the cloud controller to a peer cloud controller in the distributed filesystem.

2. The computer-implemented method of claim 1, wherein sending a peer cache request further comprises sending the peer cache request in parallel with sending a request to the cloud storage system for a cloud file that contains the data block.

3. The computer-implemented method of claim 2, wherein if the peer cloud controller is currently caching the data block, the method further comprises:

sending the requested block from the peer cloud controller to the requesting cloud controller; and

sending a cancellation request from the cloud controller to the cloud storage system to cancel the request for the cloud file.

4. The computer-implemented method of claim 2, wherein if the peer cloud controller is not currently caching the data block, the method further comprises:

receiving the cloud file from the cloud storage system; and

extracting the requested data block from the received cloud file.

5. The computer-implemented method of claim 2, wherein sending the peer cache request further comprises:

determining a cloud virtual address (CVA) associated with a cloud file in the cloud storage system that contains the data block; and

determining from the CVA a unique cloud controller identifier associated with the cloud file, wherein the unique cloud controller identifier identifies an originating cloud controller that originally uploaded the cloud file to the cloud storage system; and

wherein if the originating cloud controller is different from the requesting cloud controller, sending the peer cache request to the originating cloud controller.

6. The computer-implemented method of claim 2, wherein sending the peer cache request further comprises:

determining the presence of a backup cloud controller for the distributed filesystem, wherein the backup cloud controller is configured to determine and cache a set of data that is most frequently accessed by the clients of the distributed filesystem; and

sending the peer cache request to the backup cloud controller.

7. The computer-implemented method of claim 2,

wherein the requesting cloud controller is co-located with one or more other cloud controllers for the distributed filesystem;

wherein the co-located cloud controllers are configured to collaboratively cache data for a set of clients by tracking the set of data that is being cached by each co-located cloud controller; and

wherein sending the peer cache request further comprises sending the peer cache request to a co-located cloud controller that is caching the requested data block.

8. The computer-implemented method of claim 2, wherein sending the peer cache request further comprises determining the peer cloud controller using a locality policy for the requesting cloud controller that specifies the peer cloud controller.

9. The computer-implemented method of claim 2, wherein sending the peer cache request further comprises determining the peer cloud controller by:

determining the network latency and network bandwidth between the cloud controller and the other cloud controllers of the distributed filesystem; and

sending the peer cache request to one or more cloud controllers that are determined to be in close network proximity to the requesting cloud controller.

10. The computer-implemented method of claim 2, wherein sending the peer cache request further comprises determining the peer cloud controller by:

determining the load for the other cloud controllers of the distributed filesystem; and

sending the peer cache request to one or more cloud controllers that are determined to have a low load.

11. The computer-implemented method of claim 2,

wherein the peer cloud controller is configured to cache data for an associated set of clients during a first time interval;

wherein the peer cloud controller is configured to cache data for the requesting cloud controller for a second time interval during which the associated set of clients are likely to be idle; and

wherein sending the peer cache request further comprises sending the peer cache request to the peer cloud controller during the second time interval.

12. The computer-implemented method of claim 2, wherein sending the peer cache request further comprises sending the peer cache request to two or more cloud controllers including:

an originating cloud controller that originally uploaded the cloud file to the cloud storage system;

a backup cloud controller for the distributed filesystem;

one or more cloud controllers that are co-located with the requesting cloud controller;

one or more peer cloud controllers specified in a locality policy for the requesting cloud controller;

one or more peer cloud controllers that are determined to be in close network proximity to the requesting cloud controller;

one or more peer cloud controllers that are determined to have a low load; and

the peer cloud controller during a time interval in which the peer cloud controller is configured to be caching data on behalf of the requesting cloud controller.

13. The computer-implemented method of claim 2,

wherein the cloud storage system is unaware of the organization and structure of the distributed filesystem;

wherein data stored in the distributed filesystem is indexed using a global address space;

wherein data is stored in the cloud storage system as cloud files, wherein each cloud file is uniquely indexed in the global address space;

wherein a second cloud controller receiving the incremental metadata snapshot uses a metadata entry in the incremental metadata snapshot to retrieve data in the incremental data snapshot; and

wherein using the metadata entry to retrieve data in the incremental data snapshot comprises;

determining from the metadata entry that a desired data block in the incremental data snapshot is not presently stored in the second cloud controller;

using a global address stored in the metadata entry to identify a cloud file in the cloud storage system that includes the data block;

downloading the identified cloud file to the second cloud controller; and

using an offset stored in the metadata entry to determine the location of the data block in the cloud file.

14. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for accessing cached data from a peer cloud controller in a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises;

upon receiving in a cloud controller new data from a client, sending from the cloud controller an incremental metadata snapshot for the new data, wherein the incremental metadata snapshot is received by the other cloud controllers of the distributed filesystem;

storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems; and

sending an incremental data snapshot containing the new data from the cloud controller to a cloud storage system;

receiving at the cloud controller a request from the client for a data block for a file in the distributed filesystem;

determining that the requested data block is not currently cached in the cloud controller; and

sending a peer cache request for the requested data block from the cloud controller to a peer cloud controller in the distributed filesystem.

15. The non-transitory computer-readable storage medium of claim 14, wherein sending a peer cache request further comprises sending the peer cache request in parallel with sending a request to the cloud storage system for a cloud file that contains the data block.

16. The non-transitory computer-readable storage medium of claim 15, wherein if the peer cloud controller is currently caching the data block, the method further comprises:

sending the requested block from the peer cloud controller to the requesting cloud controller; and

sending a cancellation request from the cloud controller to the cloud storage system to cancel the request for the cloud file.

17. The non-transitory computer-readable storage medium of claim 15, wherein if the peer cloud controller is not currently caching the data block, the method further comprises:

receiving the cloud file from the cloud storage system; and

extracting the requested data block from the received cloud file.

18. The non-transitory computer-readable storage medium of claim 15, wherein sending the peer cache request further comprises:

determining a cloud virtual address (CVA) associated with a cloud file in the cloud storage system that contains the data block; and

determining from the CVA a unique cloud controller identifier associated with the cloud file, wherein the unique cloud controller identifier identifies an originating cloud controller that originally uploaded the cloud file to the cloud storage system; and

wherein if the originating cloud controller is different from the requesting cloud controller, sending the peer cache request to the originating cloud controller.

19. The non-transitory computer-readable storage medium of claim 15, wherein sending the peer cache request further comprises:

determining the presence of a backup cloud controller for the distributed filesystem, wherein the backup cloud controller is configured to determine and cache a set of data that is most frequently accessed by the clients of the distributed filesystem; and

sending the peer cache request to the backup cloud controller.

20. A cloud controller that accesses cached data from a peer cloud controller in a distributed filesystem, comprising:

a processor;

a storage mechanism that stores metadata for the distributed filesystem; and

a storage management mechanism;

wherein two or more cloud controllers collectively manage the data of the distributed filesystem, wherein collectively managing the data comprises;

upon receiving in a cloud controller new data from a client, sending from the cloud controller an incremental metadata snapshot for the new data, wherein the incremental metadata snapshot is received by the other cloud controllers of the distributed filesystem;

storing the data for the distributed filesystem in one or more cloud storage systems, wherein the storage management mechanisms of the cloud controllers are configured to cache and ensure data consistency for data stored in the cloud storage systems; and

sending an incremental data snapshot containing the new data from the cloud controller to a first cloud storage system; and

wherein the storage management mechanism is configured to;

receive a request from the client for a data block for a file in the distributed filesystem;

determine that the requested data block is not currently cached in the cloud controller; and

send a peer cache request for the requested data block from the cloud controller to a peer cloud controller in the distributed filesystem.