Systems and methods for state consistent replication

• US 8,977,594 B2
• Filed: 12/21/2012
• Issued: 03/10/2015
• Est. Priority Date: 12/21/2012
• Status: Active Grant

• Current Assignee: Arcserve LLC
• Original Assignee: Zetta Incorporated
• Inventors: Whitehead, Jeffrey, Montulli, Louis, Harrison, Jason, Schin, Chris, Seah, Maggie, Cook, Casey
• Primary Examiner(s): NGUYEN, CAM LINH T
• Application Number:
  US13/725,756
• Publication Number:
  US 20140181027A1
• Time in Patent Office: 809 Days
• Field of Search: 707/610, 707/639, 707/649, 707/655
• US Class Current: 707/639
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 2201/83 the solution involving signatures
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

Systems and methods are disclosed for state consistent replication of client data set on a client computer by generating a snapshot of the client data set on a local volume; synchronizing with a remote server volume corresponding to the local volume to create a copy of the client data set on the remote server; performing a master to slave replication of the data set; and taking a snapshot of the server data set to create a mirror of the snapshot of the client data set on the server.

Claims

1. A method for replicating a client data set on a client computer or an appliance, comprising:

generating a snapshot of the client data set on a local volume, wherein the client computer uses a file storage format;

synchronizing with a remote server volume corresponding to the local volume to create a copy of the client data set on the remote server, wherein the remote server accepts heterogeneous format;

performing a master to slave replication of the data set;

reading an existing snapshot history and making a snapshot mirror on the remote server by starting with an old client snapshot, syncing the snapshot, selecting another snapshot and syncing the snapshot and for each syncing, taking a corresponding server snapshot;

creating series of snapshots on the remote server to mirror the snapshots on either the client computer or the appliance; and

transferring back-up data files using a client software on the computer, wherein the client software receives inputs from a web user portal to control operations of the client software and a system management platform to back up the customer computer with one or more data storage silos, each including a plurality of storage directors communicating with the client software; and

a clustered data storage array.

2. The method of claim 1, comprising taking a series of snapshots on the server that exactly mirror the snapshots on the computer.

3. The method of claim 1, comprising deleting the snapshot on the client side as the server holds a copy thereof.

4. The method of claim 1, comprising mirroring contents of a client file system on a server file system.

5. The method of claim 4, comprising iterating over a dataset by starting with an initial directory and recursively syncing a directory tree.

6. The method of claim 4, comprising replicating files and directories that exist on the client file system and not the server file system.

7. The method of claim 4, comprising copying files changed on the client file system to the server file system.

8. The method of claim 7, comprising copying only changes to the client data set rather than the whole data set.

9. The method of claim 4, comprising deleing files or directories found on the server but not the client computer from the server.

10. A system, comprising:

a plurality of appliances or customer computers to be backed-up, each customer computer running a client software to transfer back-up data files by generating a snapshot of the client data set on a local volume;

synchronizing with a remote server volume corresponding to the local volume to create a copy of the client data set on the remote server, wherein the remote server accepts heterogeneous format;

performing a master to slave replication of the data set;

taking a snapshot of the server data set to create a mirror of the snapshot of the client data set on the server and reading an existing snapshot history and making a snapshot minor on the remote server by starting with an old client snapshot, syncing the snapshot, selecting another snapshot and syncing the snapshot and for each syncing, taking a corresponding server snapshot and creating series of snapshots on the remote server to mirror the snapshots on either the client computer or the appliance;

transferring back-up data files using a client software on the computer;

a system management platform coupled to the client software over the Internet, the system management platform receiving inputs from a web user portal to control operations of the client software and the system management platform to back up the customer computer; and

two or more data storage silos, each including;

a plurality of storage directors communicating with the client software; and

a clustered data storage array.

11. The system of claim 10, wherein the client software communicates using Web-based Distributed Authoring and Versioning to collaborative updating and management of files on remote web servers and wherein the client software selects the number of threads, and wherein the client software comprises a configurable parallelism selection for multi-processor efficiency and high latency connections, or for backing up NAS/SAN or RAID arrays with multiple disks.

12. The system of claim 10, wherein the data storage silos comprises a pair of head nodes and a plurality of storage nodes, wherein each storage node comprises a computer with disks and an on-disk encryptor to generate clear text and sent over an Internet Small Computer System Interface (iSCSI), wherein iSCSI traffic is load balanced and fails over network links and connected to a plurality of switches and wherein iSCSI data comprise whole disk mappings and wherein the head nodes create RAID stripes across the storage nodes.

13. The system of claim 10, comprising means for taking a series of snapshots on the server that exactly mirror the snapshots on the computer.

14. The system of claim 10, comprising means for deleting the snapshot on the client side as the server holds a copy thereof.

15. The system of claim 10, comprising means for mirroring contents of a client file system on a server file system.

16. The system of claim 15, comprising means for iterating over a dataset by starting with an initial directory and recursively syncing a directory tree.

17. The system of claim 15, comprising means for replicating files and directories that exist on the client file system and not the server file system.

18. The system of claim 15, comprising means for copying files changed on the client file system to the server file system.

19. The system of claim 15, comprising means for deleting files or directories found on the server but not the client computer from the server.