ANY POINT IN TIME REPLICATION TO THE CLOUD

• US 20200134079A1
• Filed: 10/25/2018
• Published: 04/30/2020
• Est. Priority Date: 10/25/2018
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Shemer, Jehuda, Wolfson, Kfir, Azaria, Itay
• Granted Patent: US 10,860,608 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

Systems, apparatus, and methods for any point in time replication to the cloud. Data is replicated by replicating data to a remote storage or a data bucket in the cloud. At the same time, a metadata stream is generated and stored. The metadata stream establishes a relationship between the data and offsets of the data in the production volume. This allows continuous replication without having to maintain a replica volume. The replica volume can be generated during a rehydration operation that uses the metadata stream to construct the production volume from the cloud data.

Claims

1. A method for replicating data, the method comprising:

capturing transactions occurring on production data, wherein the transactions relate to changes in the production data;

transmitting data associated with the transactions to a remote storage as objects such that the production data can be replicated from the objects;

generating a metadata stream associated with the transactions, wherein each entry in the metadata stream corresponds to a transaction; and

storing the metadata stream in a metadata storage.

2. The method of claim 1, wherein the transactions include at least one of writes and deletes.

3. The method of claim 1, further comprising chunking the data into objects.

4. The method of claim 3, wherein each of the objects has a predetermined size.

5. The method of claim 3, wherein each object is associated with an offset in the production data and each object is associated with an identifier that uniquely identifies the object.

6. The method of claim 1, further comprising consolidating the metadata stream and the objects using a consolidation interval.

7. The method of claim 1, further comprising rehydrating the production data from the objects and the metadata stream.

8. The method of claim 1, wherein the production data is continuously replicated without maintaining a replica volume in the remote storage.

9. The method of claim 1, further comprising storing an initial production copy in the remote storage.

10. A non-transitory computer readable medium comprising instructions that, when executed by a processor, perform the method of claim 1.

11. A method for rehydrating a production volume, the method comprising:

creating a blank metadata volume in preparation for rehydrating the production volume;

selecting a point in time in a metadata stream that corresponds to a restore point;

accessing the metadata stream and writing entries in the metadata stream to the blank metadata volume, wherein each entry includes an offset of an identifier of an object, wherein the identifier is written to a location in the blank metadata volume corresponding to the offset in the entry to prepare an organized metadata volume;

generating the production volume from the organized metadata volume and data stored in a remote storage, wherein objects in the remote storage corresponding to the identifiers stored in the organized metadata volume are written to corresponding offsets in the production volume.

12. The method of claim 11, wherein the generated production volume is a replica of an original production volume.

13. The method of claim 11, further comprising selecting a point in time for the production volume, wherein the point in time corresponds to a particular entry in the metadata stream.

14. The method of claim 11, wherein the production volume includes initialized production data.

15. The method of claim 14, wherein the objects in the remote storage are written to the initialized production data in the production volume.

16. The method of claim 11, wherein the organized metadata volume is prepared one entry at a time and wherein unnecessary writes are avoided.

17. The method of claim 11, wherein the objects in the remote storage are deduplicated.

18. A non-transitory computer readable medium comprising instructions that, when executed, perform the method of claim 11.