Method, system, and apparatus for providing resilient data transfer in a data protection system

• US 9,372,906 B2
• Filed: 06/10/2013
• Issued: 06/21/2016
• Est. Priority Date: 09/09/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Zizys, Giedrius, Patiejunas, Kestutis
• Primary Examiner(s): WILSON, KIMBERLY LOVEL
• Application Number:
  US13/914,389
• Publication Number:
  US 20130275374A1
• Time in Patent Office: 1,107 Days
• Field of Search: 707/676
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning

Abstract

Restarting transmission and application of changed data. A method includes receiving from the second location a first checkpoint. The first checkpoint points to a last record applied to replication data at the second location to replicate data from the first location. The method further includes receiving from the second location a second checkpoint. The second checkpoint is received by the second location from the first location. The second checkpoint identifies a last record in the set of transmitted records. The method further includes transmitting the received first checkpoint back to the second location with a command to restart applying records from the last record applied pointed to in the first checkpoint. The method further includes transmitting the received second checkpoint to the first location with a command for the first location to restart transmission of records from the last record in the set of records.

Claims

1. A method, implemented at a data protection system that includes one or more processors, for restarting a transmission of data from a first location to a second location, and for applying changed data, the method comprising:

receiving from the second location a first checkpoint, the first checkpoint having been received by the second location from the first location, the first checkpoint including (i) a first pointer to a first position in a first log at the first location that corresponds to a last data record transmitted by the first location to the second location as part of the transmission of data from the first location to the second location, and (ii) a second pointer to a second position in a second log at the second location storing the first checkpoint;

receiving from the second location a second checkpoint, the second checkpoint including a third pointer to a third position in the second log at the second location that corresponds to a last record applied to a data store at the second location; and

based at least on determining that the transmission of data from the first location to the second location is to be restarted, performing at least the following;

transmitting one or more first start commands to the second location, the one or more first start commands identifying;

a fourth position in the second log from which to start applying data records to the data store, based on the third position pointed to by third pointer of the second checkpoint; and

a fifth position in the second log to which to begin storing received data records from the first location, based on the second position pointed to by the second pointer of the first checkpoint; and

transmitting one or more second start commands to the first location, the one or more second start commands identifying a sixth position in the first log from which to send data records to the second location, based on first position pointed to by the first pointer of the first checkpoint.

2. The method of claim 1, further comprising:

determining if a validation process was in progress; and

in response to determining that validation was in progress, restarting validation.

3. The method of claim 1, further comprising transmitting a confirmation of receipt of the first checkpoint to the first location, signaling to the first location that records sent to the second location prior to sending the first checkpoint to the second location may be purged from the first log.

4. The method of claim 1, further comprising transmitting a confirmation of receipt of the second checkpoint to the second location, signaling to the second location that records prior to the third position in the second log may be purged from the second log.

5. The method of claim 1, further comprising storing the first checkpoint and the second checkpoint in a database.

6. The method of claim 1, wherein the first log is a change log and the second log is a spill log.

7. The method of claim 1, wherein the first location is a production location and the second location is a storage location.

8. A hardware storage device having stored thereon computer executable instructions that are executable by one or more processors of a computer system to cause the computer system to restart a transmission of data from a first location to a second location, and for applying changed data, the computer executable instructions including instructions that are executable to cause the computer system to perform at least the following:

receive from the second location a first checkpoint, the first checkpoint having been received by the second location from the first location, the first checkpoint including (i) a first pointer to a first position in a first log at the first location that corresponds to a last data record transmitted by the first location to the second location as part of the transmission of data from the first location to the second location, and (ii) a second pointer to a second position in a second log at the second location storing the first checkpoint;

receiving from the second location a second checkpoint, the second checkpoint including a third pointer to a third position in the second log at the second location that corresponds to a last record applied to a data store at the second location; and

based at least on determining that the transmission of data from the first location to the second location is to be restarted, performing at least the following;

transmitting one or more first start commands to the second location, the one or more first start commands identifying;

a fourth position in the second log from which to start applying data records to the data store, based on the third position pointed to by third pointer of the second checkpoint; and

a fifth position in the second log to which to begin storing received data records from the first location, based on the second position pointed to by the second pointer of the first checkpoint; and

transmitting one or more second start commands to the first location, the one or more second start commands identifying a sixth position in the first log from which to send data records to the second location, based on first position pointed to by the first pointer of the first checkpoint.

9. The hardware device of claim 8, the computer executable instructions also including instructions that are executable to cause the computer system to:

determine if a validation process was in progress; and

in response to determining that validation was in progress, restart validation.

10. The hardware device of claim 8, the computer executable instructions also including instructions that are executable to cause the computer system to:

transmit a confirmation of receipt of the first checkpoint to the first location, signaling to the first location that records sent to the second location prior to sending the first checkpoint to the second location may be purged from the first log.

11. The hardware device of claim 8, the computer executable instructions also including instructions that are executable to cause the computer system to:

transmit a confirmation of receipt of the second checkpoint to the second location, signaling to the second location that records prior to the third position in the second log may be purged from the second log.

12. The hardware device of claim 8, the computer executable instructions also including instructions that are executable to cause the computer system to store the first checkpoint and the second checkpoint in a database.

13. The hardware device of claim 8, wherein the first log is a change log and the second log is a spill log.

14. A computer system comprising:

one or more hardware processors; and

one or more hardware storage devices having stored thereon computer executable instructions that are executable by the one or more processors to cause the computer system to restart a transmission of data from a first location to a second location, and for applying changed data, the computer executable instructions including instructions that are executable to cause the computer system to perform at least the following;

receive from the second location a first checkpoint, the first checkpoint having been received by the second location from the first location, the first checkpoint including (i) a first pointer to a first position in a first log at the first location that corresponds to a last data record transmitted by the first location to the second location as part of the transmission of data from the first location to the second location, and (ii) a second pointer to a second position in a second log at the second location storing the first checkpoint;

receiving from the second location a second checkpoint, the second checkpoint including a third pointer to a third position in the second log at the second location that corresponds to a last record applied to a data store at the second location; and

based at least on determining that the transmission of data from the first location to the second location is to be restarted, performing at least the following;

transmitting one or more first start commands to the second location, the one or more first start commands identifying;

a fourth position in the second log from which to start applying data records to the data store, based on the third position pointed to by third pointer of the second checkpoint; and

a fifth position in the second log to which to begin storing received data records from the first location, based on the second position pointed to by the second pointer of the first checkpoint and

transmitting one or more second start commands to the first location, the one or more second start commands identifying a sixth position in the first log from which to send data records to the second location, based on first position pointed to by the first pointer of the first checkpoint.

15. The computer system of claim 14, the computer executable instructions also including instructions that are executable to cause the computer system to:

determine if a validation process was in progress; and

in response to determining that validation was in progress, restart validation.

16. The computer system of claim 14, the computer executable instructions also including instructions that are executable to cause the computer system to:

transmit a confirmation of receipt of the first checkpoint to the first location, signaling to the first location that records sent to the second location prior to sending the first checkpoint to the second location may be purged from the first log.

17. The computer system of claim 14, the computer executable instructions also including instructions that are executable to cause the computer system to:

transmit a confirmation of receipt of the second checkpoint to the second location, signaling to the second location that records prior to the third position in the second log may be purged from the second log.

18. The computer system of claim 14, the computer executable instructions also including instructions that are executable to cause the computer system to store the first checkpoint and the second checkpoint in a database.

19. The computer system of claim 14, wherein the first log is a change log and the second log is a spill log.

20. The computer system of claim 14, wherein the first location is a production location and the second location is a storage location.