Data replication techniques using incremental checkpoints

• US 9,495,264 B2
• Filed: 08/09/2010
• Issued: 11/15/2016
• Est. Priority Date: 08/09/2010
• Status: Active Grant

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Inventors: Desai, Samir, Pendharkar, Niranjan
• Primary Examiner(s): Ruiz, Angelica
• Application Number:
  US12/853,234
• Publication Number:
  US 20120036106A1
• Time in Patent Office: 2,290 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Incremental checkpoint, for use in data replication, track the changes made to a file system after a point in time at which the incremental checkpoint is created. Data replication techniques using the incremental checkpoints may include taking a regular checkpoint of the file system and creating the first time full copy on remote node using the regular checkpoint. Changes made to the file system are then tracked in an incremental checkpoint that are stored on the remote node. The processes of taking the incremental checkpoint and storing the incremental checkpoint are iteratively performed. The first time fully copy and the incremental checkpoints may then be used for data replication, backup, continuous data protection (CDP), or the like.

Claims

1. A method comprising:

starting generation of a full copy of a file system on a primary node at a first time, wherein the file system is locked while the full copy is generated;

starting generation of an incremental checkpoint at substantially the first time;

receiving input/output requests to the file system on the primary node;

storing queued changes to the file system in a sequential log;

unlocking and making changes to the file system on the primary node at a second time later than the first time in response to corresponding input/output requests;

completing generation of the full copy at a third time concurrent with or later than the second time;

storing the full copy on a remote node;

completing generation of the incremental checkpoint after the third time, wherein the incremental checkpoint consists of changes to the file system stored in the sequential log;

storing the incremental checkpoint on the remote node; and

generating a replication copy of the file system by applying the changes in the incremental checkpoint to the full copy of the file system.

2. The method according to claim 1, further comprising:

taking a regular checkpoint of the file system on the primary node;

creating the first time full copy on the remote node using the regular checkpoint;

taking the incremental checkpoint of the file system on the primary node; and

iteratively repeating taking the incremental checkpoint and storing the incremental checkpoint on a predetermined interval, wherein changes to the file system are paused while switching from a current incremental checkpoint to a new incremental checkpoint.

3. The method according to claim 2, further comprising:

retrieving the first time full copy from the remote node;

retrieving each successive incremental checkpoint from the remote node; and

creating the replication copy by applying changes in each successive incremental checkpoint to the first time full copy.

4. The method according to claim 1, wherein the incremental checkpoint mirrors changes to files and metadata of the file system from a point of creation of the current incremental checkpoint to a point of creation of the new incremental checkpoint.

5. The method according to claim 1, wherein the incremental checkpoint mirrors overlays to the non-written data on the original file data.

6. The method according to claim 1, wherein the incremental checkpoint comprises a block level snapshot.

7. The method according to claim 1, wherein the incremental checkpoint comprises a file level snapshot.

8. One or more non-transitory computing device readable media having computing device executable instructions which when executed perform a method comprising:

starting generation of a full copy of a file system on a primary node at a first time, wherein the file system is locked while the full copy is generated;

starting generation of an incremental checkpoint at substantially the first time;

receiving input/output requests to the file system on the primary node;

storing queued changes to the file system in a sequential log;

unlocking and making changes to the file system on the primary node at a second time later than the first time in response to corresponding input/output requests;

completing generation of the full copy at a third time concurrent with or later than the second time;

storing the full copy on a remote node;

completing generation of the incremental checkpoint after the third time, wherein the incremental checkpoint consists of changes to the file system stored in the sequential log;

storing the incremental checkpoint on the remote node; and

generating a replication copy of the file system by applying the changes in the incremental checkpoint to the full copy of the file system.

9. The one or more non-transitory computing device readable media having computing device executable instructions which when executed perform the method of claim 8, further comprising:

taking a regular checkpoint of the file system on the primary node;

creating the first time full copy on a remote node using the regular checkpoint;

taking the incremental checkpoint of the file system on the primary node; and

iteratively repeating taking the incremental checkpoint and storing the incremental checkpoint on a predetermined interval, wherein changes to the file system are paused while switching from a current incremental checkpoint to a new incremental checkpoint.

10. The one or more non-transitory computing device readable media having computing device executable instructions which when executed perform the method of claim 9, further comprising:

retrieving the first time full copy from the remote node;

retrieving each successive incremental checkpoint from the remote node; and

creating the replication copy by applying changes in each successive incremental checkpoint to the first time full copy.

11. The one or more non-transitory computing device readable media having computing device executable instructions which when executed perform the method of claim 8, wherein the incremental checkpoint mirrors changes to files and metadata of the file system from a point of creation of the current incremental checkpoint to a point of creation of the new incremental checkpoint.

12. The one or more non-transitory computing device readable media having computing device executable instructions which when executed perform the method of claim 8, wherein the incremental checkpoint mirrors overlays the non-written data on the original file data.

13. The one or more non-transitory computing device readable media having computing device executable instructions which when executed perform the method of claim 8, wherein the incremental checkpoint comprises a block level snapshot.

14. The one or more non-transitory computing device readable media having computing device executable instructions which when executed perform the method of claim 8, wherein the incremental checkpoint comprises a file level snapshot.

15. A storage system comprising:

one or more processors coupled to one or more computing device readable storage media and executing computing device readable code which implement one or more modules, wherein the one or more modules;

start generation of a full copy of a file system on a primary node at a first time, wherein the file system is locked while the full copy is generated;

start generation of an incremental checkpoint at substantially the first time;

receive input/output requests to the file system on the primary node;

store queued changes to the file system in a sequential log;

unlock and make changes to the file system on the primary node at a second time later than the first time in response to corresponding input/output requests;

complete generation of the full copy at a third time concurrent with or later than the second time;

store the full copy on a remote node;

complete generation of the incremental checkpoint after the third time, wherein the incremental checkpoint consists of changes to the file system stored in the sequential log;

store the incremental checkpoint on the remote node; and

generate a replication copy of the file system by applying the changes in the incremental checkpoint to the full copy of the file system.

16. The storage system of claim 15, wherein the one or more modules:

take a regular checkpoint of the file system on the primary node;

create the first time full copy on remote node using the regular checkpoint;

take the incremental checkpoint of the file system on the primary node; and

iteratively repeat taking the incremental checkpoint and storing the incremental checkpoint on a predetermined interval, wherein changes to the file system are paused while switching from a current incremental checkpoint to a new incremental checkpoint.

17. The storage system of claim 16, wherein the one or more modules:

retrieve the first time full copy from the remote node;

retrieve each successive incremental checkpoint from the remote node; and

create replication copy by applying changes in each successive incremental checkpoint to the first time full copy.

18. The storage system of claim 15, wherein the incremental checkpoint mirrors changes to files and metadata of the file system from a point of creation of the current incremental checkpoint to a point of creation of the new incremental checkpoint.

19. The storage system of claim 15, wherein the incremental checkpoint comprises a block level snapshot.

20. The storage system of claim 15, wherein the incremental checkpoint comprises a file level snapshot.