Systems and methods for on-line backup and disaster recovery with local copy

• US 9,483,359 B2
• Filed: 01/23/2015
• Issued: 11/01/2016
• Est. Priority Date: 12/21/2012
• Status: Expired due to Fees

• Current Assignee: Zetta LLC
• Original Assignee: Zetta Incorporated
• Inventors: Montulli, Louis, Harrison, Jason, Whitehead, Jeffrey, Schin, Chris, Seah, Maggie
• Primary Examiner(s): NGUYEN, CAM LINH T
• Application Number:
  US14/604,599
• Publication Number:
  US 20150301899A1
• Time in Patent Office: 648 Days
• Field of Search: 707/610, 707/639, 707/649, 707/655
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/17 Details of further file system functions
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Systems and methods are disclosed for rapidly restoring client data set for a computer by storing the client data and one or more pat sets required to revert to one or more version of the client data on a remote server; storing a local copy of the replicated client data on a local data storage device coupled to the computer; receiving a request to revert to a predetermined version of the client data; using the local copy as a seed, receiving a patch set corresponding to a predetermined version; and updating the local copy using the patch set to generated the predetermined version.

Claims

1. A method for rapidly restoring client data set for a computer, comprising:

storing the client data and one or more patch sets required to revert to one or more version of the client data on a remote server, wherein the remote server stores heterogeneous data regardless of file format;

storing a local copy of the replicated client data on a local data storage device coupled to the computer;

receiving a request to revert to a predetermined version of the client data;

using the local copy as a seed, receiving a patch set corresponding to a predetermined version;

updating the local copy using the patch set to generated the predetermined version; and

generating a local manifest of the client data set during a replication;

storing the local manifest on a remote server along with a server data set corresponding to the client data set;

subsequently updating the local manifest for all write operations directed at the client data set;

receiving back up or replication job request; and

using the local manifest in place of the manifest on the remote server to determine files for copying to the server data set.

2. The method of claim 1, identifying changed files based on comparisons with a manifest.

3. The method of claim 1, identifying changed files based on meta-data retrieved from a remote server.

4. The method of claim 1, comprising:

managing replication of the client data set with an application software on the computer;

generating a local manifest of the client data set during a replication;

storing the local manifest on a remote server along with a server data set corresponding to the client data set;

subsequently updating the local manifest for all write operations directed at the client data set;

receiving back up or replication job request; and

using the local manifest in place of the manifest on the remote server to determine files for copying to the server data set.

5. The method of claim 1, comprising generating a signature for each file and last modified time meta data and file size meta data.

6. The method of claim 1, comprising generating a secure hash algorithm (SHA) signature for each file and last modified time meta data and file size meta data.

7. The method of claim 1, wherein the manifest includes a header, one or more lines comprising a file or directory entry, one or more name value pairs, and a validation of the manifest.

8. The method of claim 7, comprising generating a hash to validate the manifest.

9. The method of claim 1, comprising wiping, patching, or pushing down the manifest if a discrepancy between the client and server data set is found.

10. The method of claim 1, comprising customizing a Web-based Distributed Authoring and Versioning application program interface to access the files.

11. The method of claim 1, comprising overlaying access to every file to query, set, or override the meta data.

12. The method of claim 1, comprising creating a metadata database as a shadow of a user file in a hidden folder and applying the meta data database for a random access item lookup.

13. The method of claim 1, comprising storing meta data in a single index database with a meta data database file size below the size of a single block.

14. The method of claim 1, comprising translating between an original native file system and a split file and metadata system.

15. The method of claim 1, comprising using a software client as a translation agent.

16. The method of claim 1, comprising reading each file data and meta data and sending both through Web-based Distributed Authoring and Versioning application program interface.

17. The method of claim 1, comprising storing file data and meta data separately on a file system, and linking both through folder proximity and file name.

18. The method of claim 1, comprising storing as metadata file modification time, file creation time, file access time, file size, access control list (ACLs), permission list, hash of file data.

19. The method of claim 1, comprising storing metadata in sidecar files for any type of attribute.

20. The method of claim 1, comprising transferring back-up data files using a client software on the computer, wherein the client software receives inputs from a web user portal to control operations of the client software and a system management platform to back up the customer computer with one or more data storage silos, each including a plurality of storage directors communicating with the client software;

and a clustered data storage array.

21. The method of claim 1, comprising restoring files through a group consisting of:

the client software, a web based file browser replicating a look and feel of a native file explorer, and a remote data drive at the data storage silo mounted by a user simulating a local drive.

22. A system, comprising:

a plurality of customer computers to be backed-up, each customer computer running a client software to transfer back-up data files by storing the client data and one or more pat sets required to revert to one or more version of the client data on a remote server;

computer code for;

storing a local copy of the replicated client data on a local data storage device coupled to the computer;

receiving a request to revert to a predetermined version of the client data;

using the local copy as a seed, receiving a patch set corresponding to a predetermined version;

updating the local copy using the patch set to generated the predetermined version, wherein the remote server accepts heterogeneous data regardless of file format;

generating a local manifest of the client data set during a replication;

storing the local manifest on a remote server along with a server data set corresponding to the client data set;

subsequently updating the local manifest for all write operations directed at the client data set;

receiving back up or replication job request; and

using the local manifest in place of the manifest on the remote server to determine files for copying to the server data set; and

a system management platform coupled to the client software over the Internet, the system management platform receiving inputs from a web user portal to control operations of the client software and the system management platform to back up the customer computer; and

two or more data storage silos, each including;

a plurality of storage directors communicating with the client software; and

a clustered data storage array.

23. The system of claim 22, wherein the client software communicates using Web-based Distributed Authoring and Versioning to collaborative updating and management of files on remote web servers and wherein the client software selects the number of threads.

24. The system of claim 22, wherein the client software comprises a configurable parallelism selection for multi-processor efficiency and high latency connections, or for backing up NAS/SAN or RAID arrays with multiple disks.

25. The system of claim 22, wherein the data storage silos comprises a pair of head nodes and a plurality of storage nodes, wherein each storage node comprises a computer with disks and an on-disk encryptor to generate clear text and sent over an Internet Small Computer System Interface (iSCSI), wherein iSCSI traffic is load balanced and fails over network links and connected to a plurality of switches and wherein iSCSI data comprise whole disk mappings and wherein the head nodes create RAID stripes across the storage nodes.