RESTORING AN ARCHIVED FILE IN A DISTRIBUTED FILESYSTEM

• US 20140006357A1
• Filed: 09/05/2013
• Published: 01/02/2014
• Est. Priority Date: 11/14/2011
• Status: Active Grant

• Current Assignee: Panzura, Inc.
• Original Assignee: Panzura, Inc.
• Inventors: Davis, Andrew P., Parkison, Brian Christopher, Taylor, John Richard
• Granted Patent: US 9,804,928 B2
• Time in Patent Office: Days
• US Class Current: 707/667
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

The disclosed embodiments disclose techniques for restoring an archived file in a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. Furthermore, cloud controllers may archive infrequently-accessed files in an archival cloud storage system. During operation, a cloud controller receives a request from a client system to access an archived file, and restores this archived file from the archival cloud storage system.

Claims

1. A computer-implemented method for restoring an archived file in a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises;

storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems;

caching data for the distributed filesystem in each cloud controller; and

archiving one or more infrequently accessed files for the distributed filesystem in an archival cloud storage system;

receiving at a cloud controller a request from a client system to access an archived file in the distributed filesystem; and

restoring the archived file from the archival cloud storage system.

2. The computer-implemented method of claim 1, wherein receiving the request further involves:

receiving in the request a filename for a file in the distributed filesystem;

determining from metadata for the distributed filesystem that the file has been archived in the archival cloud storage system;

notifying the client system that the file is archived; and

receiving a restore request for the archived file from the client system.

3. The computer-implemented method of claim 1, wherein restoring the archived file comprises one or more of:

transferring one or more cloud files containing data associated with the archived file from the archival cloud storage system to a second cloud storage system; and

transferring the data blocks for the restored file to the cloud controller.

4. The computer-implemented method of claim 3, wherein restoring the archived file comprises:

tracking in the cloud controller status information for the cloud controller and the distributed filesystem; and

using the tracked status information to estimate a restore time interval for the archived file.

5. The computer-implemented method of claim 4, wherein restoring the archived file further comprises presenting the tracked status information and the estimated restore time interval to the client system via a status file that is associated with the archived file.

6. The computer-implemented method of claim 5,

wherein the status file represents a directory in the distributed filesystem and comprises status information and a restore time interval for all of the archived files in a sub-hierarchy of the distributed filesystem that can be accessed via the directory;

wherein the request from the client system initiates a restore operation for all of the archived files in the sub-hierarchy; and

wherein the sub-hierarchy includes the archived file.

7. The computer-implemented method of claim 5, wherein restoring the archived file further comprises:

determining a billing model for the archival cloud storage system;

adjusting the restore behavior for the distributed filesystem to balance the performance and cost of restore operations for the distributed filesystem.

8. The computer-implemented method of claim 7, wherein adjusting the restore behavior for the distributed filesystem comprises:

determining a set of restore options for the archived file based on the billing model, wherein the set of restore options span a range of different restore time intervals and costs that are available for restoring the archived file;

presenting the set of restore options to the client system using the status file; and

receiving a selected restore option from the client system.

9. The computer-implemented method of claim 7,

wherein the billing model comprises billing for restored data based on the maximum restore bandwidth that is used to access the archival cloud storage system in a given time interval;

and

wherein adjusting the restore behavior for the distributed filesystem comprises coordinating the restore bandwidth that is used by the set of cloud controllers that collectively manage the data of the distributed filesystem based on the billing model to ensure that restore operations for the distributed filesystem do not exceed a specified budget.

10. The computer-implemented method of claim 9, wherein tracking status information for the distributed filesystem further comprises tracking outstanding restore requests and in-progress restore operations for the distributed filesystem in the context of the billing model to provide an estimated restore time interval that meets the budget constraints of the distributed filesystem.

11. The computer-implemented method of claim 10,

wherein a restore request for the archived file indicates a high priority for the archived file and an increased restore budget for the distributed filesystem;

and

wherein restoring the archived file further comprises increasing the maximum restore bandwidth that is used to access the archival cloud storage system to reduce the restore time interval for the archived file.

12. The computer-implemented method of claim 10,

wherein a restore request for the archived file indicates a high priority for the archived file;

and

wherein restoring the archived file further comprises adjusting the scheduling of pending and in-progress restore operations for the distributed filesystem to reduce the estimated restore time interval for the archived file.

13. The computer-implemented method of claim 7, wherein the method further comprises:

checking for an update to the billing model for the archival cloud storage system;

retrieving an updated billing model for the archival cloud storage system; and

adjusting the schedule of pending and in-progress restore operations for the distributed filesystem to optimize restore performance for the distributed filesystem while ensuring that the restore operations for the distributed filesystem do not exceed the specified budget in the context of the updated billing model.

14. The computer-implemented method of claim 7,

wherein the billing model imposes a minimum storage term for a set of stored data;

wherein the billing model determines the restore bandwidth available for the distributed filesystem based on the amount of distributed filesystem data stored in the archival cloud storage system; and

wherein, upon receiving a request to delete the set of stored data from the archival cloud storage system before the end of the minimum storage term, the method further comprises preserving the set of stored data in the archival cloud storage system for the extent of the minimum storage term to maximize the restore bandwidth available for the distributed filesystem.

15. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for restoring an archived file in a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises;

storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems;

caching data for the distributed filesystem in each cloud controller; and

archiving one or more infrequently accessed files for the distributed filesystem in an archival cloud storage system;

receiving at a cloud controller a request from a client system to access an archived file in the distributed filesystem; and

restoring the archived file from the archival cloud storage system.

16. The non-transitory computer-readable storage medium of claim 15, wherein receiving the request further involves:

receiving in the request a filename for a file in the distributed filesystem;

determining from metadata for the distributed filesystem that the file has been archived in the archival cloud storage system;

notifying the client system that the file is archived; and

receiving a restore request for the archived file from the client system.

17. The non-transitory computer-readable storage medium of claim 15, wherein restoring the archived file comprises one or more of:

transferring one or more cloud files containing data associated with the archived file from the archival cloud storage system to a second cloud storage system; and

transferring the data blocks for the restored file to the cloud controller.

18. The non-transitory computer-readable storage medium of claim 17, wherein restoring the archived file comprises:

tracking in the cloud controller status information for the cloud controller and the distributed filesystem; and

using the tracked status information to estimate a restore time interval for the archived file.

19. The non-transitory computer-readable storage medium of claim 18, wherein restoring the archived file further comprises presenting the tracked status information and the estimated restore time interval to the client system via a status file that is associated with the archived file.

20. A cloud controller that restores an archived file in a distributed filesystem, comprising:

a processor;

a storage mechanism that stores metadata for the distributed filesystem; and

a storage management mechanism;

wherein two or more cloud controllers collectively manage the data of the distributed filesystem;

wherein the cloud controller is configured to archive one or more infrequently accessed files for the distributed filesystem in an archival cloud storage system;

wherein the cloud controller is further configured to receive a request from a client system to access an archived file in the distributed filesystem; and

wherein the cloud controller is further configured to restore the archived file from the archival cloud storage system.