Method and system for interfacing to cloud storage

• US 8,880,474 B2
• Filed: 07/24/2009
• Issued: 11/04/2014
• Est. Priority Date: 01/23/2009
• Status: Active Grant

• Current Assignee: Nasuni Corporation
• Original Assignee: Nasuni Corporation
• Inventors: Mason, Robert S., Rodriguez, Andres
• Primary Examiner(s): Shmatov, Alexey
• Assistant Examiner(s): PARK, GRACE A
• Application Number:
  US12/508,614
• Publication Number:
  US 20100191783A1
• Time in Patent Office: 1,929 Days
• Field of Search: 707/649
• US Class Current: 707/649
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097
• CPC Class Codes: G06F 16/1873 Versioning file systems, temporal file systems, e.g. file system supporting different historic versions of files
• CPC Class Codes: G06F 16/9027 Trees
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

An interface between a local file system and a data store (e.g., a “write-once”store) generates and exports to the data store a “versioned” file system. The state of the local file system at a given point in time can be determined using the versioned file system. The versioned file system comprises a set of structured data representations that comprise metadata associated with actual file system data. The interface creates and exports to the data store structured data representations, which correspond to versions of the local file system, as well as the actual file system data that these structured data representations represent. A cache associated with the interface stores recently used data and metadata, as well as write data and metadata waiting to be transferred to the data store. Prior to transfer, data transformations, such as compression, encryption, and de-duplication, may be applied to the data and metadata.

Claims

1. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by a processor, enable cloud-based storage of contents of a local file system, comprising:

code to intercept local file system data traffic and to generate a series of structured data representations of the local file system, each structured data representation representing a complete copy of a version of the local file system and corresponding to, and transformed from, a snapshot of the local file system at a point-in-time, the code to generate the series of structured data representations comprising;

code to aggregate changes to the local file system as a set of delta frames;

wherein each delta frame is defined by a tree structure that starts with a new root that represents a current version of the local file system, and that further includes a set of change events that have been generated as a result of modification to existing inodes, directories and files of the local file system, and pointers to those existing inodes, directories and files of the local file system, wherein a given delta frame represents a logical representation of a combination of a current version of the local file system and one or more prior versions of the local file system;

code to write-once export each structured data representation to one or more storage service providers, the structured data representations comprising a versioned file system (VFS) representing complete copies of versions of the local file system at various points-in-time, at least one structured data representation adapted to being accessed and returned to the local file system as needed to enable restore of a point-in-time version of the local file system by walking one and only one delta frame tree structure; and

code to restore the local file system to the point-in-time version by retrieving the at least one structured data representation from the one or more storage service providers and performing a restore operation at the local file system, the restore operation walking the one and only one delta frame tree structure.

2. The computer-readable storage medium as described in claim 1 further including:

code to apply one or more data transformations to the local file system data file prior to exporting a structured data representation that includes the file.

3. The computer-readable storage medium as described in claim 2 wherein the one or more data transformations are one of:

compression, encryption, de-duplication, and combinations thereof.

4. The computer-readable storage medium as described in claim 1 wherein at least one of structured data representations exported comprises a set of second objects, with each of the second objects representing a portion of the file as it is exists in the local file system.

5. The computer-readable storage medium as described in claim 1 wherein a structured data representation is an XML representation.

6. The computer-readable storage medium as described in claim 1 further including:

code to configure a volume in at least one of the storage service providers to store the exported structured data representations.

7. The computer-readable storage medium as described in claim 6 further including:

code to execute a management function with respect to the volume, wherein the management function is selected from one of;

moving the volume, copying the volume, linking the volume, recovering the volume, removing the volume, changing an attribute associated with the volume, and reporting on data associated with the volume.

8. The computer-readable storage medium as described in claim 1 wherein a structured data representation is generated upon a change within the file system.

9. The computer-readable storage medium as described in claim 8 wherein the change within the file system is one of:

a file creation, file deletion, file modification, directory creation, directory deletion and directory modification.

10. The computer-readable storage medium as described in claim 1 further including:

code to cache a portion of the file system data for use to restore the local file system to a point-in-time on an as-needed basis.

11. An apparatus for configuring one or more user local file systems to interface to cloud storage, comprising:

a processor;

a computer-readable medium having stored thereon program instructions executed by the processor to perform a configuration, the program instructions comprising;

program instructions to create a volume in cloud storage for use in storing a series of structured data representations that represent versions of a user'"'"'s local file system, each structured data representation representing a complete copy of a version of the local file system and corresponding to, and transformed from, a snapshot of the local file system at a point-in-time;

program instructions to associate to the volume a file system agent that executes in the user local file system, wherein the file system agent intercepts local file system data and generates the series of structured data representations by aggregating changes to the local file system as a set of delta frames, wherein each delta frame is defined by a tree structure that starts with a new root that represents a current version of the local file system, and that further includes a set of change events that have been generated as a result of modification to existing inodes, directories and files of the local file system, and pointers to those existing inodes, directories and files of the local file system, wherein a given delta frame represents a logical representation of a combination of a current version of the local file system and one or more prior versions of the local file system, wherein a walk of just the delta frame tree structure enables local restore of a point-in-time version of the local file system;

program instructions to identify one or more storage service providers to host the volume; and

program instructions to restore the local file system to the point-in-time version by retrieving at least one structured data representation from the one or more storage service providers and performing a restore operation at the local file system, the restore operation walking just the delta frame tree structure.

12. The apparatus as described in claim 11 wherein the program instructions further include program instructions to execute a management function with respect to the volume, wherein the management function is selected from one of:

associating or de-associating the volume with the file system agent, moving the volume, copying the volume, linking the volume, recovering the volume, removing the volume, changing an attribute associated with the volume, and reporting on data associated with the volume.

13. The apparatus as described in claim 11 wherein the program instructions further include program instructions to provide an encryption key to the user local file system for use in encrypting the structured data representations.

14. A method to interface a local file system to cloud-based storage, comprising:

configuring a volume in at least one cloud-based data store using a web-accessible interface executable on a hardware element;

intercepting local file system data traffic and generating a series of structured data representations of the local file system using software executing on hardware in association with the local file system, each structured data representation representing a complete copy of a version of the local file system and corresponding to, and transformed from, a snapshot of the local file system at a point-in-time, the series of structured data representations generated by;

aggregating changes to the local file system as a set of delta frames;

wherein each delta frame is defined by a tree structure that starts with a new root that represents a current version of the local file system, and that further includes a set of change events that have been generated as a result of modification to existing inodes, directories and files of the local file system, and pointers to those existing inodes, directories and files of the local file system, wherein a given delta frame represents a logical representation of a combination of a current version of the local file system and one or more prior versions of the local file system;

write-once exporting each structured data representation to the cloud-based data store for storage in the volume, the structured data representations comprising a versioned file system (VFS) representing complete copies of versions of the local file system at various points-in-time, at least one structured data representation adapted to being accessed and returned to the local file system as needed to enable restore of a point-in-time version of the local file system by walking one and only one delta frame tree structure; and

upon a given occurrence, restoring the local file system to the point-in-time version by retrieving from the cloud-based data store the at least one structured data representation and performing a restore operation at the local file system using the retrieved structured data representation, the restore operation walking the one and only one delta frame tree structure.

15. The method as described in claim 14 further including applying a transformation to local file system data prior to exporting a structured data representation associated with the data.

16. The method as described in claim 15 wherein the transformation is one of:

compression, encryption, de-duplication, and combinations thereof.

17. The method as described in claim 14 further including:

executing a management function with respect to the volume, wherein the management function is selected from one of;

associating or de-associating the volume, moving the volume, copying the volume, linking the volume, recovering the volume, removing the volume, changing an attribute associated with the volume, and reporting on data associated with the volume.