Method and apparatus for integrating primary data storage with local and remote data protection

• US 7,925,623 B2
• Filed: 09/10/2003
• Issued: 04/12/2011
• Est. Priority Date: 09/10/2002
• Status: Active Grant

• Current Assignee: ExaGrid Systems, Inc.
• Original Assignee: ExaGrid Systems, Inc.
• Inventors: Therrien, David G., VanderSpek, Adrian, Sawyer, Cory Lee, Pownell, James E., Hansen, Thomas G., Ramu, Ashok T., Kenna, Herman Robert
• Primary Examiner(s): Rones; Charles
• Assistant Examiner(s): Adams; Charles D
• Application Number:
  US10/659,129
• Publication Number:
  US 20040088331A1
• Time in Patent Office: 2,771 Days
• Field of Search: 707/200, 707/202, 707/204, 709/321, 714/6, 711/162
• US Class Current: 707/610
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

Embodiments of this invention provide primary magnetic disk data storage capacity to clients while at the same time making sure that client data is replicated locally and at an offsite location to protect from all forms of data loss.

Claims

1. A data protection system, comprising:

a fileserver configured to contain shares of data and to be in communication with at least one local repository that is in communication with at least one remote repository, wherein two or more repositories are configured to store a replica of a file, wherein each repository includes multiple repository nodes, at least one repository node of each repository is configured to store the replica of the file, wherein a storage location and a number of replicas in each repository is configured to be changed over time by a user;

wherein based on a criticality of the file, the number of stored replicas of the file is increased or decreased in at least one repository;

wherein shares of data are directories or folders of storage capacity created on the fileserver;

the fileserver includes;

a filter driver configured to intercept input or output activity initiated by client file requests, including modification of any existing stored files and creation of new files as modification of any existing stored files and creation of new files occur, and further configured to capture a snapshot of a set of the shares of data at a particular point in time and to maintain a list of modified and created files since a last snapshot occurred;

a file system in communication with the filter driver and configured to store client files;

the fileserver is configured to store a unique protection policy for each share of data on the fileserver, the protection policy defines;

repositories used to protect each share of data;

frequency of data protection;

number of replicas of each file that are maintained in each repository; and

,

maintenance of modifications to each share of data;

based on the definitions in the protection policy, the filter driver is configured to capture the snapshot.

2. The system of claim 1, wherein the fileserver further comprises:

a location cache configured to determine based on the protection policy which repository will be used to protect each share of data; and

a location manager coupled to the location cache and configured to update the location cache when the fileserver protects a new share of data in a specific repository node.

3. The system of claim 2, wherein the local repository in communication with the fileserver is adapted for receiving files from the fileserver;

the local repository is further configured to receive replicated files from the fileserver; and

the local repository includes a protection policy component configured to determine whether new versions of existing files should be compressed and whether older versions of existing files should be maintained.

4. The system of claim 3, wherein the remote repository in communication with the local repository is adapted for receiving files from the local repository;

the remote repository is further configured to receive replicated files from the local repository; and

the remote repository includes a protection policy component configured to determine whether new versions of existing files should be compressed and whether older versions of existing files should be maintained.

16. The system of claim 1, wherein said fileserver is configured to

backup said modified files into repositories identified in said protection policy based on said backup frequency;

and

store a latest version of a file in a repository where a prior version of said file is stored;

determine a difference between said latest version of said file and said prior version of said file;

apply reverse delta compression to said difference;

replace said prior version of said file with said reverse delta compressed difference between said latest version and said prior version of said file.

17. The system of claim 1, wherein, based in the protection policy, the fileserver is configured to determine the location of repositories and a number of replicas of the files to be stored in each repository.

18. The system of claim 1, wherein, based on the protection policy, the fileserver is further configured to determine whether to purge a file from a repository after the file has been deleted from a set of files.

19. The system of claim 1, wherein, based on the protection policy, the fileserver is further configured to determine whether to keep a version history of a file in the set of files.

20. The system of claim 1, wherein, based on the protection policy, the fileserver is further configured to determine a specified backup frequency for a repository.

21. The system of claim 1, wherein, based on the protection policy, the fileserver is further configured to determine a specified type of compression for a file in the set of files.

22. The system of claim 1, wherein, based on the protection policy, the fileserver is further configured to determine a specified caching level of a repository.

5. A method for protecting data comprising:

providing a fileserver configured to contain sets of files and to be in communication with at least one local repository that is in communication with at least one remote repository, wherein two or more repositories are configured to store a replica of a file;

wherein sets of files are directories or folders of storage capacity created on the fileserver;

the fileserver including

a filter driver configured to intercept input or output activity initiated by client file requests, including modification of any existing stored files and creation of new files as modification of any existing stored files and creation of new files occur;

a file system in communication with the filter driver and operative to store client files;

storing a version of a file within a set of files on the fileserver;

using the filter driver, capturing a snapshot of the set of files at a particular point in time based on a backup frequency defined in a protection policy;

using the filter driver, maintaining a list of modified and created files since last captured snapshot;

examining the protection policy associated with the set of files to determine where and how to protect files associated with the set of files;

wherein the protection policy defines;

repositories used to protect each set of files;

frequency of data protection;

number of replicas of each file that are maintained in each repository; and

,

maintenance of modifications to each set of files; and

,

replicating the version of the file to two or more repositories specified by the protection policy, wherein the repositories include at least one local repository and at least one remote repository, wherein a storage location and a number of replicas of the version of the file is configured to be changed over time by a user;

wherein each repository includes multiple repository nodes, at least one repository node of each repository is configured to store the replica of the file;

wherein based on the criticality of the file, the number of stored replicas of the file is increased or decreased in at least one repository;

wherein the protection policy is configured to be uniquely defined for each set of files.

6. The method of claim 5 wherein the file is configured to have at least one version.

7. The method of claim 6 wherein the method further comprises:

applying reverse delta compression to the version of the file when a successive version of the file is stored in the repository.

8. The method of claim 7 wherein the step of applying reverse delta compression comprises

creating another version of the file, wherein the another version of the file is a version of the file successive to the version of the file;

replicating the another version of the file into the local repository and the remote repository;

replacing the replicated version of the file in the local repository with a reverse delta compressed version representing a difference between the version of the file and the another version of the file and replicating;

transmitting the reverse delta compressed version to the remote repository; and

in the remote repository, replacing the version of the file with the reverse delta compressed version to store the another version and the reverse delta compressed version.

9. The method of claim 5 wherein examining a protection policy associated with the set of files to determine where and how to protect files associated with the set of files comprises:

determining the location of repositories and a number of replicas of the files to be stored in each repository.

10. The method of claim 5 wherein examining a protection policy associated with the set of files to determine where and how to protect files associated with the set of files comprises:

determining whether to purge the file from a repository after the file has been deleted from a set of files.

11. The method of claim 5 wherein examining a protection policy associated with the set of files to determine where and how to protect files associated with the set of files comprises:

determining whether to keep a version history of a file in the set of files.

12. The method of claim 5 wherein examining a protection policy associated with the set of files to determine where and how to protect files associated with the set of files comprises:

determining a specified backup frequency for a repository.

13. The method of claim 5 wherein examining a protection policy associated with the set of files to determine where and how to protect files associated with the set of files comprises:

determining a specified type of compression for a file in the set of files.

14. The method of claim 5 wherein examining a protection policy associated with the set of files to determine where and how to protect files associated with the set of files comprises:

determining a specified caching level of a repository.

15. A data protection system comprising:

a fileserver configured to contain shares of data and to be in communication with at least one local repository that is in communication with at least one remote repository, wherein two or more repositories are configured to store a replica of a file, wherein each repository includes multiple repository nodes, at least one repository node of each repository is configured to store the replica of the file, wherein a storage location and a number of replicas in each repository is configured to be changed over time by a user;

wherein based on a criticality of the file, the number of stored replicas of the file is increased or decreased in at least one repository;

wherein shares of data are directories or folders of storage capacity created on the fileserver;

said fileserver includes;

filter driver means for intercepting input or output activity initiated by client file requests, including modification of any existing stored files and creation of new files as modification of any existing stored files and creation of new files occur, and for capturing a snapshot of a set of the shares of data at a particular point in time and for maintaining a list of modified and created files since a last snapshot occurred;

file system means in communication with the filter driver, the file system means for storing client files;

the fileserver is configured to store a unique protection policy for each share of data on the fileserver, the protection policy defines;

repositories used to protect each share of data;

frequency of data protection;

number of replicas of each file that are maintained in each repository; and

,

maintenance of modifications to each share of data;

based on the definitions in the protection policy, said filter driver means is configured to capture the snapshot.

23. The system of claim 15, wherein the fileserver further includes

backup means for backing up the modified files into repositories identified in the protection policy based on the backup frequency;

storage means for storing a latest version of a file in a repository where a prior version of the file is stored;

means for determining a difference between the latest version of the file and the prior version of the file;

means for applying reverse delta compression of the difference; and

means for replacing the prior version of the file with the reverse delta compressed difference between the latest version and the prior version of the file.