Systems and methods for continuous data replication

• US 20080028009A1
• Filed: 12/21/2006
• Published: 01/31/2008
• Est. Priority Date: 07/27/2006
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Ngo, David
• Granted Patent: US 8,726,242 B2
• Time in Patent Office: Days
• US Class Current: 707/204
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Systems and methods for intelligent data replication are provided, which may include the identification and use of consistency points in application data, such that one or more replication volumes may be created using an “application aware” replication process that allows the system to intelligently replicate data or other information (e.g., metadata) relating to a particular process, application or specified set of data.

Claims

1. A method of copying application data indicative of a known recoverable state of an operating software application, the method comprising:

identifying a set of data indicative of the known recoverable state of the software application from a group of software application data generated, at least in part, by the operating software application;

generating an indicator and storing the indicator in a data structure associated with the set of data indicative of the known recoverable state; and

copying the set of data indicative of the known recoverable state designated by the indicator such that the copied set of data may be used to restore the operating software application to a known stable state.

2. The method of claim 1 wherein the identifying further comprises quiescing the software application.

3. The method of claim 2 wherein the quiescing of the software application is requested by a software module in communication with the software application.

4. The method of claim 3 wherein the software module is a data agent.

5. The method of claim 1 wherein the identifying further comprises analyzing the group of software application data that represents the operating software application to determine the set of data indicative of the known recoverable state of the software application

6. The method of claim 4 wherein the analyzing of the group of software application data is performed by a software module in communication with the software application.

7. The method of claim 6 wherein the software module is a data agent.

8. The method claim 1 wherein the identifying further comprises creating a first data log of the group of software application data that represents the operating software application.

9. The method of claim 8 wherein the quiescing of the operating software application comprises:

suspending any further data write operations, and

allowing pending data write operations to complete such that the application data in the first data log includes the result of the pending data operations.

10. The method of claim 9 wherein the generating further comprises storing the indicator data structure associated with the first data log at a point immediately after the log entry representing the result of the last pending data write operation such that the indicator designates within the first data log the set of data indicative of the known recoverable state.

11. The method of claim 10 further comprising associating a first time stamp with the indicator that represents a point in time at which the operating software application was quiesced.

12. The method of claim 11 wherein the copying further comprises:

periodically copying the data stored in the first data log to a second data log;

copying data from the second data log to one or more first storage devices, based, at least in part, on data or application type;

suspending copy operations from the second data log to one or more first storage devices when the indicator is encountered; and

copying data from the one or more first storage devices to one or more second storage devices while the copy operations from the second data log to the one or more first storage devices are suspended.

13. The method of claim 12 wherein the copying data from the one or more first storage devices to the one or more second storage devices comprises taking a snapshot of the data on the one or more first storage devices.

14. The method of claim 12 wherein the copying further comprises copying the data from the second data log to the one or more storage devices with a plurality of replay threads.

15. The method of claim 12 further comprising:

operating a plurality of replay threads in parallel on the data in the second data log to facilitate copying of the data to the one or more storage devices; and

notifying a management module when the plurality of replay threads encounter the indicator; and

suspending copy operations when each of the plurality of replay threads notifies the management module that the indicator has been encountered.

16. The method of claim 12 further comprising associating a second time stamp with the copying of data from the one or more first storage devices to one or more second storage devices.

17. The method of claim 16 further comprising:

relating the first time stamp with the second time stamp;

obtaining location information identifying the location of the data copied to the one or more second storage devices; and

associating the location information with the first and second time stamp information for subsequent use in locating the set of data representative of the known recoverable state copied at a time indicated by the first time stamp.

18. A computer-readable medium having stored thereon a plurality of sequences of instructions including sequences of instructions which, when executed by one or more processors cause an electronic device to:

identify a set of software application data indicative of a known recoverable state of a software application from a group of software application data that represents an operating software application;

generate an indicator and insert the indicator at a point in the group of software application data that represents the set of software application data indicative of the known recoverable state; and

copy the set of data indicative of the known recoverable state designated by the indicator such that the copied set of data may be used to restore the operating software application to a known stable state.

19. A method of correlating copies of application data of an operating software application in a storage system, the method comprising:

identifying a set of data indicative of the operating software application;

creating a first data log and copying the software application data that represents the operating software application to the first data log;

generating an indicator and placing the indicator at a point in the software application data;

assigning a time stamp to the indicator;

periodically copying the data stored in the first data log to a second data log;

copying data from the second data log to one or more first storage devices until the indicator is reached;

associating a second time stamp with the copying of data to the to the one or more first storage devices; and

associating the first time stamp to the second time stamp such that the application data stored in the first data log is correlated with the application data stored on the one or more first storage devices.

20. The method of claim 19 further comprising periodically copying the application data stored on the one or more first storage devices to one or more second storage devices.

21. The method of claim 20 wherein the application data is copied to the one or more second storage devices when the indicator is encountered during a copy operation.

22. The method of claim 21 wherein the copying of application data to the one or more second storage devices comprises taking a snapshot of the application data on the one or more first storage devices.

23. The method of claim 22 further comprising assigning a third time stamp to the snapshot.

24. The method of claim 23 further comprising:

associating the first time stamp with the third time stamp; and

associating a location of the application data stored on the one or more second storage devices with the first time stamp such that the application data stored on the first data log becomes synchronized with the application data stored on the one or more second storage devices.

25. The method of claim 24 further comprising copying location information associated with the application data stored on the one or more second storage devices to an index.

26. The method of claim 24 further comprising suspending copy operations from the second data log to one or more first storage devices when the indicator is encountered;

and

copying data from the one or more first storage devices to the one or more second storage devices while the copy operations from the second data log to the one or more first storage devices are suspended.

27. The method of claim 26 wherein the indicator is placed at a point in the group of software application data that represents a set of software application data indicative of known recoverable state of the software application.

28. A computer-readable medium having stored thereon a plurality of sequences of instructions including sequences of instructions which, when executed by one or more processors cause an electronic device to:

identify a set of software application data indicative of an operating software application;

create a first data log and copy the software application data that represents the operating software application to the first data log;

generate an indicator and place the indicator at a point in the software application data;

assign a time stamp to the indicator;

periodically copy the data stored in the first data log to a second data log;

copy data from the second data log to one or more first storage devices until the indicator is reached;

associate a second time stamp with the copying of data to the to the one or more first storage devices; and

associate the first time stamp to the second time stamp such that the application data stored in the first data log is correlated with the application data stored on the one or more first storage devices.

29. A system for of copying application data indicative of a known recoverable state of an operating software application, the method comprising:

A software module installed on a source computing device for identifying a set of data indicative of the known recoverable state of the software application from a group of software application data that represents the operating software application;

the software module generating an indicator and placing the indicator at a point in the group of software application data that represents the set of data indicative of the known recoverable state;

a first log file for receiving a copy of the set of data indicative of the known recoverable state designated by the indicator;

a second log file that periodically receives a copy of the application data in the first log file;

a first data store that receives a copy data in the second log file until the indicator is reached; and

a second data store that receives a copy data in the second log file such that the copied set of data received by the second data store may be correlated to the data in the first log file and used to restore the operating software application to a known stable state.

30. the system of claim 29 wherein the software module further comprises one or more filter drivers.

31. the system of claim 30 wherein the one or more filter drivers are application specific.

32. The system of claim 29 wherein the application data is copied from the first log file to the second log file over a network.

33. The system of claim 29 further comprising a storage manager component.

34. The system of claim 29 further comprising a replication manager that interoperates with the software module for coordinating the transfer of application from the first log file to the second log file.

35. The system of claim 29 further comprising a replication manager that instantiates a copy module for copying application data from the second log file to the first data store.

36. The system of claim 29 further comprising a replication manager that instantiates a plurality of replay threads that traverse the second log file in parallel for copying application data from the second log file to the first data store.

37. The system of claim 29 wherein the first log file includes one or more filter driver logs.

38. The system of claim 29 wherein the second log file includes one or more replication logs.

39. The system of claim 29 wherein the first data store includes at least one replication volume.

40. The system of claim 29 wherein the second data store includes snapshots of the replication volume.