METHODS AND APPARATUS FOR OPTIMAL JOURNALING FOR CONTINUOUS DATA REPLICATION

• US 20080082770A1
• Filed: 09/28/2006
• Published: 04/03/2008
• Est. Priority Date: 09/28/2006
• Status: Active Grant

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: EMC Israel Development Center Limited (Dell Technologies Inc.)
• Inventors: Drukh, Evgeny, Natanzon, Assaf, Kedem, Oded, Ahal, Shlomo, Sechner, Tzach
• Granted Patent: US 7,516,287 B2
• Time in Patent Office: Days
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1474 in transactions G06F16/20 takes precedence
• CPC Class Codes: G06F 11/2035 without idle spare hardware
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 11/2074 Asynchronous techniques

Abstract

Methods and systems for continuous data replication, including receiving a write transaction from a production site, the write transaction including raw data (RWNEW), a size indicator (SZ1) for the size of the raw data, and a start address (ADD1) for writing the raw data, copying the raw data (RWNEW) into a first journal data stream at a current journal address (JADD1), and copying the size indicator (SZ1), the start address (ADD1) and the journal address (JADD1) into a first journal meta-data stream.

Claims

1. A method for continuous data replication, comprising:

receiving a write transaction from a production site, the write transaction including first raw data (RWNEW), a first size indicator (SZ1) for the size of the raw data, and a first start address (ADD1) for writing the first raw data;

copying the first raw data (RWNEW) into a first journal data stream starting at a first current journal address (JADD1); and

copying the first size indicator (SZ1), the first start address (ADD1), and the first current journal address (JADD1) into a first journal meta-data stream.

2. The method according to claim 1, further including copying the first size indicator (SZ1), the first start address (ADD1), and the first current journal address (JADD1) into the end of the first journal meta-data stream.

3. The method of claim 2 further comprising:

reading a second size indicator (SZ2), a second start address (ADD2), and a second journal address (JADD), from the beginning of the first journal meta-data stream;

reading second raw data (RW) from the first journal data stream beginning at the second journal address (JADD) and extending to a size of the second size indicator (SZ2); and

copying the second raw data (RW) to a storage unit, beginning at a third start address (ADD).

4. The method of claim 3, further comprising, prior to the copying of the second raw data (RW) to the storage unit:

reading third raw data (RWOLD) from the storage unit beginning at the second start address (ADD2) and extending to a size of the second size indicator (SZ2);

copying the third raw data (RWOLD) to an undo journal data stream at the second current journal address (JADD2); and

copying size the second indicator (SZ2), the second start address (ADD2), and the third journal address (JADD2) into an undo journal meta-data stream, at the end of the undo journal meta-data stream.

5. The method of claim 3, further comprising, prior to the copying of the second raw data (RW) to the storage unit:

conditionally reading the third raw data (RWOLD), from the storage unit beginning at the second start address (ADD2) and extending to a size of the second size indicator (SZ2) if a size of the first journal data stream exceeds a selected length;

conditionally copying the third raw data (RWOLD) to a second journal data stream at the third current journal address (JADD2) if the size of the first journal data stream exceeds the selected length; and

conditionally copying the second size indicator (SZ2), the second start address (ADD2), and the third journal address (JADD2) into a second journal meta-data stream, at the end of the second journal meta-data stream, if the size of the first journal data stream exceeds the selected length.

6. The method of claim 5 wherein the selected length corresponds to a percentage of maximum capacity for the first journal data stream.

7. The method of claim 5 wherein the selected length corresponds to a recovery time.

8. An article comprising:

a storage medium having stored thereon instructions that when executed by a machine result in the following;

receiving a write transaction from a production site, the write transaction including first raw data (RWNEW), a first size indicator (SZ1) for the size of the raw data, and a first start address (ADD1) for writing the first raw data;

copying the first raw data (RWNEW) into a first journal data stream starting at a first current journal address (JADD1); and

copying the first size indicator (SZ1), the first start address (ADD1), and the first current journal address (JADD1) into a first journal meta-data stream.

9. The article according to claim 8, further including instructions to enable copying the first size indicator (SZ1), the first start address (ADD1), and the first current journal address (JADD1) into the end of the first journal meta-data stream.

10. The article of claim 9 further including instructions to enable:

reading a second size indicator (SZ2), a second start address (ADD2), and a second journal address (JADD), from the beginning of the first journal meta-data stream;

reading second raw data (RW) from the first journal data stream beginning at the second journal address (JADD) and extending to a size of the second size indicator (SZ2); and

copying the second raw data (RW) to a storage unit, beginning at a third start address (ADD).

11. The article of claim 9, further including instructions to enable, prior to the copying of the second raw data (RW) to the storage unit:

reading third raw data (RWOLD) from the storage unit beginning at the second start address (ADD2) and extending to a size of the second size indicator (SZ2);

copying the third raw data (RWOLD) to an undo journal data stream at the second current journal address (JADD2); and

copying size the second indicator (SZ2), the second start address (ADD2), and the third journal address (JADD2) into an undo journal meta-data stream, at the end of the undo journal meta-data stream.

12. The article of claim 10, further including instructions to enable, prior to the copying of the second raw data (RW) to the storage unit:

conditionally reading the third raw data (RWOLD), from the storage unit beginning at the second start address (ADD2) and extending to a size of the second size indicator (SZ2) if a size of the first journal data stream exceeds a selected length;

conditionally copying the third raw data (RWOLD) to a second journal data stream at the third current journal address (JADD2) if the size of the first journal data stream exceeds the selected length; and

conditionally copying the second size indicator (SZ2), the second start address (ADD2), and the third journal address (JADD2) into a second journal meta-data stream, at the end of the second journal meta-data stream, if the size of the first journal data stream exceeds the selected length.

13. The article of claim 12, wherein the selected length corresponds to a percentage of maximum capacity for the first journal data stream.

14. The article of claim 12 wherein the selected length corresponds to a recovery time.

15. A system, comprising:

a source side storage system to store write transaction data, the source side storage system including a source data protection appliance coupled to a source journal logical unit and a source side logical unit;

a target side storage system to replicate write transaction data stored in the source side storage system, the target side storage system including a target data protection appliance coupled to a target journal logical unit;

the target side storage system having a memory for storing instructions to enable;

receiving a write transaction including first raw data (RWNEW), a first size indicator (SZ1) for the size of the raw data, and a first start address (ADD1) for writing the first raw data;

copying the first raw data (RWNEW) into a first journal data stream starting at a first current journal address (JADD1); and

copying the first size indicator (SZ1), the first start address (ADD1), and the first current journal address (JADD1) into a first journal meta-data stream.

16. The system according to claim 15, further including instructions to enable copying the first size indicator (SZ1), the first start address (ADD1), and the first current journal address (JADD1) into the end of the first journal meta-data stream.

17. The system of claim 16 further including instructions to enable:

reading a second size indicator (SZ2), a second start address (ADD2), and a second journal address (JADD), from the beginning of the first journal meta-data stream;

reading second raw data (RW) from the first journal data stream beginning at the second journal address (JADD) and extending to a size of the second size indicator (SZ2); and

copying the second raw data (RW) to a storage unit, beginning at a third start address (ADD).

18. The system of claim 17, further including instructions to enable, prior to the copying of the second raw data (RW) to the storage unit:

reading third raw data (RWOLD) from the storage unit beginning at the second start address (ADD2) and extending to a size of the second size indicator (SZ2);

copying the third raw data (RWOLD) to an undo journal data stream at the second current journal address (JADD2); and

copying size the second indicator (SZ2), the second start address (ADD2), and the third journal address (JADD2) into an undo journal meta-data stream, at the end of the undo journal meta-data stream.

19. The system of claim 18, further including instructions to enable, prior to the copying of the second raw data (RW) to the storage unit:

conditionally reading the third raw data (RWOLD), from the storage unit beginning at the second start address (ADD2) and extending to a size of the second size indicator (SZ2) if a size of the first journal data stream exceeds a selected length;

conditionally copying the third raw data (RWOLD) to a second journal data stream at the third current journal address (JADD2) if the size of the first journal data stream exceeds the selected length; and

conditionally copying the second size indicator (SZ2), the second start address (ADD2), and the third journal address (JADD2) into a second journal meta-data stream, at the end of the second journal meta-data stream, if the size of the first journal data stream exceeds the selected length.

20. The system of claim 19 wherein the selected length corresponds to a percentage of maximum capacity for the first journal data stream and/or a recovery time.