Distributed asynchronous ordered replication

• US 7,624,109 B2
• Filed: 02/25/2005
• Issued: 11/24/2009
• Est. Priority Date: 02/27/2004
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Texas Memory Systems Incorporated (International Business Machines Corporation)
• Inventors: Testardi, Richard
• Primary Examiner(s): Vo; Tim T.
• Assistant Examiner(s): Morrison; Jay A
• Application Number:
  US11/067,453
• Publication Number:
  US 20050204106A1
• Time in Patent Office: 1,733 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/855 Details of asynchronous mirroring using a journal to transfer not-yet-mirrored changes
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

Described are methods, systems, and apparatus, including computer program products, for achieving distributed asynchronous ordered replication. Distributed asynchronous ordered replication includes creating a first journal for a first set of I/O data, creating a second journal for a second set of I/O data, and temporarily preventing committal, of the second set of I/O data until the second journal is created. In some examples, the first and second journals comprise entries. The entries of the first and second journals include counter values. The entries of the first journal typically have a different counter value than the entries of the second journal.

Claims

1. A method of maintaining committal of I/O data to achieve distributed asynchronous ordered replication, the method comprising:

creating a first journal for a first set of I/O data to be committed to a local storage device, the first journal comprising entries, the entries comprising a first counter value that is the same for all entries of the first journal and the order of entries in the first journal being independent of the order of the first set of I/O data committed to the local storage;

creating a second journal for a second set of I/O data to be committed to the local storage device, the second journal comprising entries, the entries comprising a second counter value that is the same for all entries of the second journal and the order of entries in the second journal being independent of the order of the second set of I/O data committed to the local storage;

temporarily preventing committal, to the local storage device, of the second set of I/O data until the second journal is created; and

beginning committal of the second set of I/O data to the local storage device before the first set of I/O data is finished committal.

2. The method of claim 1 further comprising committing the first set to I/O data to the local storage device and to the first journal at substantially the same time.

3. The method of claim 1 wherein the local storage device comprises a plurality of storage devices.

4. The method of claim 3 wherein the first journal is a plurality of journals.

5. The method of claim 3 wherein the second journal is a plurality of journals.

6. The method of claim 1 wherein the second journal is created in response to the first journal filling up.

7. The method of claim 1 wherein the second journal is created in response to a replication interval occurring.

8. The method of claim 7 wherein the replication interval occurs every 30 seconds.

9. The method of claim 1 further comprising sending the first journal to a remote storage device.

10. The method of claim 9 further comprising after sending the first journal to the remote storage device, applying the first journal to the remote storage device.

11. The method of claim 9 further comprising after sending the first journal to the remote storage device, sending the second journal to the remote storage device.

12. The method of claim 11 further comprising applying the entries of the first and second journals to the remote storage device, ordered by the entries'"'"' counter value.

13. The method of claim 12 wherein entries with equal counter values are applied in a preferred order.

14. The method of claim 13 wherein the preferred order is a random order.

15. The method of claim 13 wherein the preferred order is that which the journal entries are read in.

16. The method of claim 13 wherein the preferred order is according to the age of first and second journals.

17. A system for distributed asynchronous ordered replication, the system comprising:

a fast path for performing journal operations;

a control path; and

a first storage device comprising

a first journal for a first set of I/O data, the first journal comprising journal entries, the entries comprising a first counter value that is the same for all entries of the first journal, the order of entries in the first journal being independent of the order of the first set of I/O data committed to the local storage, and representing an I/O operation of the first set of I/O data performed by the fast path; and

a second journal for a second set of I/O data, the second journal comprising journal entries, the entries comprising a second counter value that is the same for all entries of the second journal, and the order of entries in the second journal being independent of the order of the second set of I/O data committed to the local storage, and representing I/O operation of the second set of I/O data performed by the fast path;

wherein the second counter value is different than the first counter value and wherein the control path temporarily prevents the fast path from performing the second set of I/O data until the second journal is created, allowing for performance of the second set of I/O data before finishing performance of the first set of I/O data.

18. The system of claim 17 wherein journal entries of the first and second journals represent I/O operations performed by the fast path on the first storage.

19. The system of claim 18 further comprising a second storage for backing up the first storage.

20. The system of claim 19 wherein journal entries of the first and second journals are applied to the second storage ordered by counter value, wherein entries with equal counter values are applied in a preferred order.

21. The system of claim 20 wherein the preferred order is a random order.

22. The system of claim 20 wherein the preferred order is that which the journal entries are read in.

23. The system of claim 20 wherein the preferred order is according to the age of the first and second journals.

24. The system of claim 17 wherein the journal entries of the first journal are written at substantially the same time the fast path performs the I/O operations represented by the first set of I/O data and the journal entries of the second journal are written at substantially the same time the fast path performs the I/O operations represented by the second set of I/O data.

25. A computer program product, tangibly embodied in a machine-readable storage device, for maintaining committal of I/O data to achieve distributed asynchronous ordered replication, the computer program product including instructions being operable to cause a data processing apparatus to:

create a first journal for a first set of I/O data to be committed to a local storage device, the first journal comprising entries, the entries comprising a first counter value that is the same for all entries of the first journal and the order of entries in the first journal being independent of the order of the first set of I/O data committed to the local storage;

create a second journal for a second set of I/O data to be committed to the local storage device, the second journal comprising entries, the entries comprising a second counter value that is the same for all entries of the second journal and the order of entries in the second journal being independent of the order of the second set of I/O data committed to the local storage;

temporarily prevent committal, to the local storage device, of the second set of I/O data until the second journal is created; and

begin committal of the second set of I/O data to the local storage device before the first set of I/O data is finished committal.

26. A system for maintaining committal of I/O data to achieve distributed asynchronous ordered replication without using a universal timestamp for ordering journal entries, the system comprising:

a local storage device;

means for creating a first journal for a first set of I/O data to be committed to the local storage device, the first journal comprising entries, the entries comprising a first counter value that is the same for all entries of the first journal and the order of entries in the first journal being independent of the order of the first set of I/O data committed to the local storage;

means for creating a second journal for a second set of I/O data to be committed to the local storage device, the second journal comprising entries, the entries comprising a second counter value that is the same for all entries of the second journal and the order of entries in the second journal being independent of the order of the second set of I/O data committed to the local storage;

means for temporarily preventing committal, to the local storage device, of the second set of I/O data until the second journal is created; and

means for beginning committal of the second set of I/O data to the local storage device before the first set of I/O data is finished committal.