Method and system for no downtime resychronization for real-time, continuous data protection

• US 7,680,834 B1
• Filed: 06/08/2004
• Issued: 03/16/2010
• Est. Priority Date: 06/08/2004
• Status: Active Grant

• Current Assignee: Quest Software, Inc.
• Original Assignee: BakBone Software, Inc. (Quest Software, Inc.)
• Inventors: Sim-Tang, Siew Yong
• Primary Examiner(s): Rones; Charles
• Assistant Examiner(s): Khoshnoodi; Fariborz
• Application Number:
  US10/862,971
• Time in Patent Office: 2,107 Days
• Field of Search: 707/201, 707/202, 714/4, 714/48
• US Class Current: 1/1
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users

Abstract

A data management system or “DMS” provides an automated, continuous, real-time data protection service to one or more data sources associated with a set of application host servers. To facilitate the service, a host driver embedded in an application server captures real-time data transactions. When a data protection command for a given data source is forwarded to a host driver, an event processor enters into an initial upload state. During this state, the event processor gathers a list of data items to be protected and creates a data list. Then, the event processor moves the data to a DMS core to create initial baseline data. The upload is a stream of application-aware data chunks that are attached to upload events. A resynchronization state is entered when there is a suspicion that the state of the data in the host is out-of-sync with the state of the most current data in the DMS. During upload or upward resynchronization, the application does not have to be shut down.

Claims

1. A machine-implemented method of synchronizing data as a data protection service is being provided to a given data source in a first processing environment, the method comprising:

generating, at the first processing environment, a continuous, application-aware data stream by capturing data changes associated with one or more events associated with the given data source, the data stream comprising, for each event, data identifying the event, given metadata, and the data changes;

transferring the continuous, application-aware data stream to a data store remote from the first processing environment;

determining whether a state of given data in the first processing environment is out of synchronization with respect to a state of the given data in the data store remote from the first processing environment; and

upon determining that the state of the given data in the first processing environment is out of synchronization with respect to the state of the given data in the data store, and upon determining the given data in the first processing environment is not corrupt, initiating a given operation with respect to the data source in a first processing environment to synchronize the given data by the following sub-steps that are carried out in the first processing environment;

ordering into a data structure a set of data items that reference the given data source;

flagging the data items in the set of data items that are out of synchronization;

requesting metadata from the data store;

using the metadata received from the data store to if identify at least one data item in the set of data items that is out of synchronization with a corresponding data item in the remote store;

with respect to the at least one data item in the set of data items that is out of synchronization, generating a difference value; and

placing the difference value into the continuous, application-aware data stream;

wherein the given operation occurs while the given application continues to execute and as the continuous, application-aware data stream continues to be generated and transferred to the data store to provide the data protection.

2. The method as described in claim 1 wherein the given operation is initiated after a blackout when the given data in the first processing environment is changed.

3. The method as described in claim 2 further including the step of updating the state of the given data in the first processing environment during the blackout.

4. The method as described in claim 1 wherein the event associated with the given data source is a checkpoint.

5. The method as described in claim 1 wherein the given operation is initiated after a host in the first processing environment is rebooted and a determination is made that the state of the given data in the data store is unknown.

6. The method as described in claim 5 further including the step of examining a state of each of a set of data items associated with the data source.

7. The method as described in claim 1 wherein the data source is a file system associated with the first processing environment.

8. The method as described in claim 1 wherein the data source is a database associated with the first processing environment.

9. The method as described in claim 1 wherein the data structure is a sorted source tree.

10. A system for protecting a data source associated with a host in a first processing environment, comprising:

a processor;

a data structure having a list of data items associated with the data source; and

a computer program stored on a computer recordable medium, comprising;

code executed in the processor and responsive to initiation of a data protection service for transferring to a data store remote from the first processing environment a continuous, application-aware data stream, the continuous, application-aware data stream being generated by capturing data changes associated with one or more events associated with the data source, the data stream comprising, for each event, data identifying the event, given metadata, and the data changes; and

code executed in the processor and responsive to information in the data structure for synchronizing a state of given data at the host with a state of the given data at the data store by the following method;

ordering into the data structure a set of data items that reference the data source;

flagging the data items in the set of data items that are out of synchronization;

requesting metadata from the data store;

using the metadata received from the data store to identify at least one data item in the set of data items that is out of synchronization with a corresponding data item in the remote data store; and

with respect to the at least one data item in the set of data items that is out of synchronization, generating a difference value; and

placing the difference value into the continuous, application-aware data stream;

wherein the synchronization occurs as the continuous, application-aware data stream continues to be generated and transferred to the data store for data protection.

11. The system as described in claim 10 wherein the data structure is a sorted source tree.

12. The system as described in claim 10 wherein the data source is a file system.

13. The system as described in claim 10 wherein the data source is a database.

14. The system as described in claim 10 wherein the application-aware data stream includes checkpoint data.

15. A machine-implemented method of synchronizing data as a data protection service, comprising:

monitoring and capturing real-time events as an application interfaces to a local data store in the first processing environment;

forwarding the real-time events to a data store remote from the first processing environment as a continuous, application-aware data stream;

ordering into a sorted data structure a set of data items associated with an application executing in a first processing environment, each data item having a pointer associated therewith;

flagging the set of data items to identify one or more data items that may be are out of synchronization with the associated data items in the data store remote from the first processing environment; and

processing the flagged data item in the set of data items as follows;

(a) requesting metadata from the data store;

(b) receiving the metadata from the data store;

(c) using the metadata to make a determination that the flagged data item is out of synchronization with respect to an associated data item in the data store;

(d) upon determining that the flagged data item is out of synchronization with respect to the associated data item in the data store, placing a difference value into the continuous, application-aware data stream; and

(e) unflagging the data item;

wherein the flagging and processing steps occur as the continuous, application-aware data stream continues to be generated and transferred from the first processing environment to the data store for data protection.