Replication using deduplicated secondary copy data

• US 10,481,824 B2
• Filed: 09/30/2016
• Issued: 11/19/2019
• Est. Priority Date: 05/26/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Vijayan, Manoj Kumar, Job, Joe Sabu Thyvelikkakakth
• Primary Examiner(s): Morrison, Jay A
• Assistant Examiner(s): Hoang, Ken
• Application Number:
  US15/282,445
• Publication Number:
  US 20170083563A1
• Time in Patent Office: 1,145 Days
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1666 where the redundant component is memory or memory area
• CPC Class Codes: G06F 11/2089 Redundant storage control functionality
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/835 Timestamp
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0683 Plurality of storage devices

Abstract

An information management system according to certain aspects uses backup copies or other secondary copies of production data for the purposes of replicating production data to another client. The secondary copies can be deduplicated copies. By utilizing available secondary copies of the data for replication, the system can reduce the impact on the production machines associated with replication. Utilizing deduplicated copies not only reduces the amount of stored data, but also reduces the amount of data that is communicated between the source and the destination, increasing the speed of the replication process.

Claims

1. A system for replicating primary data from a source system to a destination system using secondary copy data in secondary storage, the system comprising:

a source system comprising;

one or more source system client computing devices comprising hardware and one or more source system primary storage devices that store primary data, the source system primary data stored in a native format;

one or more source system secondary storage controller computers comprising hardware that create deduplicated secondary data with corresponding signature values when the number of writes of source system primary data on the source system primary storage devices exceed a predefined threshold, the deduplicated secondary data stored in one or more source system secondary storage devices in a secondary storage format that is different than the native format of the source system primary data;

at least one source system secondary storage replication agent comprising computer hardware, the source system secondary storage replication agent configured to generate one or more secondary data replication streams comprising replication command messages having timestamps associated therewith, the one or more secondary data replication streams further comprising the deduplicated secondary data, and embedded cone soon-dine denature values, the embedded corresponding signature values stored contiguously with the deduplicated secondary data in the one or more secondary data replication streams; and

a destination system comprising;

one or more destination client computing devices comprising hardware and one or more destination primary storage devices that store replicated destination system primary data in a native format;

at least one destination system secondary storage replication agent comprising computer hardware, the destination system secondary storage replication agent receives the one or more secondary data replication streams, the destination system secondary storage replication agent configured to create a replicated deduplicated secondary copy based on the deduplicated secondary data and the embedded corresponding signature values stored contiguously with the deduplicated secondary data, the replicated deduplicated secondary copy is stored in one or more destination system secondary storage devices m the secondary storage format; and

at least one destination data agent configured to apply the replication command messages received in the one or more secondary data replication streams to a restored version, of the replicated deduplicated secondary copy based on the timestamps to create the replicated destination system primary data consistent with the source system primary data based on the timestamps, the replicated destination system primary data stored in the one or more destination primary storage devices in the native format of the primary data.

2. The system of claim 1, wherein the replicated deduplicated secondary copy is one of a plurality of replicated deduplicated secondary copies that are part of a continuous replication process.

3. The system of claim 2 wherein the plurality of replicated deduplicated secondary copies are created based at least in part on the time of a last write of the source system primary data on the source system primary storage devices.

4. The system of claim 2 wherein the plurality of replicated deduplicated secondary copies are created based at least in part on the time since a last write of the source system primary data on the source system primary storage devices.

5. The system of claim 2 wherein the plurality of replicated deduplicated secondary copies are created based at least in part on availability of the source system client computing device.

6. The system of claim 2 wherein the plurality of replicated deduplicated secondary copies are created based at least in part on usage of a the source system client computing device.

7. The system of claim 2 wherein the restored version of the replicated deduplicated secondary copy is one of a plurality of restored versions or a plurality of replicated deduplication secondary copies, wherein each of the plurality of restored versions corresponds to one of the plurality of replicated deduplicated secondary copies.

8. The system of claim 7 wherein the plurality of restored versions are created at an interval such that changes to the source system primary data are replicated to the destination system in a period of time not significantly greater than the interval.

9. The system of claim 1 wherein the deduplicated secondary data and corresponding signature values are stored in the source system secondary storage devices in physically contiguous memory space.

10. The system of claim 1 wherein the deduplicated secondary data and corresponding signature values are stored in logically contiguous memory space in the one or more source system secondary storage devices.

11. A method of replicating primary data from a source system to a destination system using secondary copy data in secondary storage, the method comprising:

creating with one or more source secondary storage controllers deduplicated secondary data that is a copy of source system primary data on one or more source system primary storage devices when the number of writes of the source system primary data on the source system primary storage devices exceed a predefined threshold, the source system primary data stored in a native format and the deduplicated secondary data stored in a backup format that is different than the native format of the source system primary data;

sending with a source system secondary storage replication agent comprising computer hardware, one or more secondary data replication streams comprising replication command messages having timestamps associated therewith, the one or more secondary data replication streams further comprising the deduplicated secondary data and embedded corresponding signature values, the embedded corresponding signature voices stored contiguously with the deduplicated secondary data in the one or more secondary data replication streams

receiving the one or more secondary data replication streams with at least one destination system secondary storage replication agent comprising computer hardware, creating a replicated deduplicated secondary copy based on the deduplicated secondary data and the embedded corresponding signature values stored contiguously with the deduplicated secondary data, and storing the replicated deduplicated secondary COPY In one or more destination system secondary storage devices in the secondary storage format; and

creating a restored version of the replicated deduplicated secondary copy and applying, with a destination data agent, the replication command messages received m the one or more secondary data replication streams to the restored version of the replicated deduplicated secondary copy based on the timestamps to create replicated destination system primary data consistent with the source system primary data based on the timestamps, the replicated destination system primary data stored in the one or more destination primary storage devices in the native formal of the source system primary data.

12. The method of claim 11 wherein the replicated deduplicated secondary copy is one of a plurality of replicated deduplicated secondary copies that are part of a continuous replication process.

13. The method of claim 12 wherein the plurality of replicated deduplicated secondary copies are created based at least in part on the time of a last write of the source system primary data on the source system primary storage devices.

14. The method of claim 12 wherein the plurality of replicated deduplicated secondary copies are created based at least in part on the time since a last write of the source system primary data on the source system primary storage devices.

15. The method of claim 12 wherein the plurality of replicated deduplicated secondary copies are created based at least in part on availability of a source system client computing device.

16. The method of claim 12 wherein the plurality of replicated deduplicated secondary copies are created based at least in part on usage of the source system client computing device.

17. The method of claim 12 wherein the restored version of the replicated deduplicated secondary copy is one of a plurality of restored versions of a plurality of replicated deduplicated secondary copies, wherein each of the plurality of restored versions corresponds to one of the plurality of deduplicated secondary copies.

18. The method of claim 17 wherein the plurality of restored versions are created at an interval such that changes to the primary data on the source system are replicated to the destination system in a period of time not significantly greater than the interval.

19. The method of claim 11 wherein the deduplicated secondary data and corresponding signature values are arranged in the one or more secondary data replication streams in physically contiguous memory space.

20. The method of claim 11 wherein the deduplicated secondary data and corresponding signature values are arranged in the one or more secondary data replication streams in logically contiguous memory space.