Method, system, and apparatus for providing resilient data transfer in a data protection system

• US 8,145,601 B2
• Filed: 09/09/2004
• Issued: 03/27/2012
• Est. Priority Date: 09/09/2004
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Patiejunas, Kestutis, Zizys, Giedrius
• Primary Examiner(s): LOVEL, KIMBERLY M
• Application Number:
  US10/937,060
• Publication Number:
  US 20060053121A1
• Time in Patent Office: 2,756 Days
• Field of Search: 707/1, 707/204, 707/640, 707/999.202
• US Class Current: 707/640
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning

Abstract

A method for transmitting records of changes to data from a production location to a storage location. The method stores in a log, records of changes to data stored at a production location. Those records are transmitted and a transmitted records checkpoint is generated that is transmitted at the end of the transmitted records. A records checkpoint conformation is received and the plurality of transmitted records are purged from the log.

Claims

1. A method for transmitting records of changes to data from a production location to a storage location, comprising:

storing in a first log, records of changes to data stored at a production location in response to a change to protected data made by a user, wherein the records are transmitted at a predetermined time from the production location to the storage location;

providing users with a logical representation of data stored at the production location and allowing the users to logically identify what data at the production location to protect;

transmitting a plurality of the stored records from the production location to the storage location, the transmission between the production location and the storage location is initiated in response to the user making a change to protected data or at a scheduled job execution time;

generating a transmitted records checkpoint that includes an identification of a location within the first log of a last transmitted record and appending the transmitted records checkpoint to the end of the transmission records, wherein the transmitted records checkpoint is generated if the amount of data exceeds a predefined minimum transmission size;

configuring a second log to maintain a plurality of records received from the production location and adding a location identifier to the transmitted records checkpoint that identifies the location of the transmitted records checkpoint in the second log;

transmitting the transmitted records checkpoint to a data protector that monitors the transmission of data between the storage location and the production location and confirms the accuracy of the transmission by receiving the transmitted records checkpoint from the storage location;

receiving a transmitted records checkpoint confirmation indentifying to the production location that records prior to the last transmitted record identified in the transmitted records checkpoint have been received at the storage location; and

in response to receiving the transmitted records checkpoint confirmation removing the plurality of transmitted records from the first log.

2. The method of claim 1, wherein subsequent to transmitting a plurality of the stored records, additional records of changes to data are stored in the first log.

3. The method of claim 1, wherein the received transmitted records checkpoint confirmation indicates that each of the plurality of the stored records transmitted prior to the transmission of the transmitted records checkpoint have been received at a storage location.

4. The method of claim 1, wherein the received transmitted records checkpoint is received from a data management system.

5. The method of claim 1, wherein the plurality of the records includes metadata records about the data stored at the production location.

6. A system for transferring data from a production location to a storage location, comprising:

a processor for executing the following components;

a data protector configured to monitor and control a transfer of data from the production location to the storage location, the data transfer between the production location and the storage location is initiated in response to a user making a change to protected data, wherein the data protector provides users with a logical representation of data stored at the production location and allows the users to logically identify what data at the production location to protect;

a first log configured to maintain a plurality of records of changes made to data at the production location, wherein the records are transmitted at a predetermined time from the production location to the storage location, in response to the change to protected data made by the user;

a first checkpoint generation device located at the production location that generates a first checkpoint that is included with the transmitted records, wherein the first checkpoint is generated if the amount of data exceeds a predefined minimum transmission size, and wherein the first checkpoint comprises a first location identifier of a location within the first log of a last transmitted record;

a second log configured to maintain a plurality of records received from the production location; and

a second checkpoint generation device located at the storage location that adds a second location identifier to the first checkpoint that identifies the location of the first checkpoint in the second log, wherein the first checkpoint with the first location identifier and the second location identifier are sent to the data protector;

wherein the data protector monitors the transmission of data between the storage location and the production location and confirms the accuracy of the transmission by receiving the first checkpoint with the first location identifier and the second location identifier from the storage location; and

wherein the data protector initiates validation of data between the production location and the storage location and adds any differences to a difference log, wherein the difference log includes a list of items of data stored at the storage location that are different than items of data stored at the production location.

7. The system of claim 6, wherein the data protector maintains a database of first checkpoints.