Data synchronization of multiple remote storage after remote copy suspension

• US 20030177321A1
• Filed: 03/11/2003
• Published: 09/18/2003
• Est. Priority Date: 01/03/2002
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Watanabe, Naoki
• Granted Patent: US 7,139,932 B2
• Time in Patent Office: Days
• US Class Current: 711/161
• CPC Class Codes: G06F 11/2058 using more than 2 mirrored copies
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2069 Management of state, configuration or failover
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

A method and apparatus are provided for enhancing the performance of storage systems to allow recovery after all types of suspensions in remote copy operations. Data is synchronized after an interruption in transfer between a first storage volume of a primary storage system and a first storage volume of a secondary storage system which also includes a second storage volume. After the interruption is detected, at the primary storage system, a record is provided of the data written onto the first storage volume of the primary storage system, and at the secondary storage volume a record is provided of the data written onto the first storage volume of the secondary storage system. Then, at least a partial copy of the record of the data written onto the first storage volume of the primary storage system is written onto the second storage volume. Using the copy, the first storage volume of the secondary storage system is synchronized with the second storage volume of the secondary storage system.

Claims

1. A storage system comprising:

a local storage system having storage media for storing data;

a remote storage system having at least two remote storage media for storing data, the remote storage being operable to mirror the local storage;

a detection circuit for detecting an interruption in communications between the local storage and the remote storage; and

a control circuit operating in response to the detection circuit for creating a record of the changes made to the local storage and the remote storage up to the time of the interruption, and for providing a copy of that record to other storage volumes in the local and the remote storage system.

2. A storage system as in claim 1 wherein:

the local storage includes a first volume mirrored by a first volume in the remote storage; and

the control circuit creates a copy of the changes made to the first volume of the local storage on the second volume of the remote storage.

3. A storage system as in claim 2 further comprising a processor for controlling synchronization of the first volume of the remote storage with the second volume of the remote storage.

4. A storage system as in claim 1 wherein the control circuit makes a copy of the record of the changes to the local storage on another local storage if it is available, and otherwise makes a copy to the remote storage.

5. A storage system comprising:

a cascaded series of storage volumes having a first storage volume being mirrored to a second storage volume;

a detection circuit for detecting an interruption in communications between the first storage volume and second storage volume; and

a control circuit operating in response to the detection circuit for creating a record of the changes made to the first storage volume and the second storage volume, and for providing a copy of that record to other storage volumes in the cascaded series.

6. A storage system as in claim 5 wherein the record is used to synchronize a new pair of the cascaded series.

7. A method of synchronizing data stored in a storage system after an interruption in a transfer of data between a first storage volume of a primary storage system and a first storage volume of a secondary storage system, the secondary storage system also including a second storage volume, the method comprising:

detecting an interruption in the transfer of data from the primary storage system to the secondary storage system;

after detecting the interruption in the transfer of data;

at the primary storage system, providing a record of the data written onto the first storage volume of the primary storage system, and

at the secondary storage volume providing a record of the data written onto the first storage volume of the secondary storage system;

creating on the second storage volume of the secondary storage system at least a partial copy of the record of the data written onto the first storage volume of the primary storage system; and

using the at least partial copy on the second storage volume and the record of the data written onto the first storage volume of the secondary storage system, synchronizing the first storage volume of the secondary storage system with the second storage volume of the secondary storage system.

8. A method as in claim 7 wherein the partial copy is a copy of the record of the data written onto the first storage volume of the primary storage system.

9. A method as in claim 7 wherein the record of data regarding each of the first storage volumes comprises a bitmap.

10. A method as in claim 9 wherein the step of using the at least partial copy on the second storage volume and the record of the data written onto the first storage volume of the secondary storage system, synchronizing the first storage volume of the secondary storage system with the second storage volume of the secondary storage system further comprises:

exchanging bitmaps between the first and the second volumes of the secondary storage system;

using the exchanged bitmaps, determining a set of necessary write operations to synchronize the primary storage system and the secondary storage system; and

performing the set of necessary write operations.

11. A method as in claim 7 wherein at least one of the steps of providing a record of the data written onto the first storage volume comprises:

determining if the storage volume for which the record will be created is a primary storage volume;

if it is a primary storage volume, and there is another primary storage volume, then creating a record at the another primary storage volume;

if it is a primary storage volume, and there is not another primary storage volume, but there is another secondary storage volume, then creating a record at the another secondary storage volume; and

if it is not a primary storage volume, then creating a record at another secondary storage volume.

13. A method as in claim 11 wherein the first storage volume is synchronized with the third storage volume.

14. A method as in claim 11 wherein the second storage volume is synchronized with the fourth storage volume.

12. A method of synchronizing data stored in a cascaded storage system having first, second, third and fourth storage volumes serially connected, after an interruption in a transfer of data between a the second storage volume and the third storage volume, the method comprising:

detecting an interruption in the transfer of data from the second storage volume to the third storage volume;

after detecting the interruption;

at the second storage volume, providing a first record of the data written onto the second storage volume, and

at the third storage volume providing a second record of the data written onto the third storage volume;

copying at least portions of the first record to the first storage volume; and

copying at least portions of the second record to the fourth storage volumes;

using at least one of the portions copied, synchronizing at least one of the second and third storage volumes to at least one of the first and fourth storage volumes.

15. A method of synchronizing data stored in a secondary storage system having at least a first storage media and a second storage media with data stored in a primary storage system after an interruption in a transfer of data between the primary storage system and the first storage media in the secondary storage system, the method comprising:

determining in the transfer of data from the primary storage system to the first storage media in the secondary storage system when the interruption in the transfer of data occurred;

at the primary storage system providing a record of the data written onto the primary storage system and at the secondary storage system providing a record of the data written onto the first storage media in the secondary storage system;

creating on the second storage media at the secondary storage system a copy of the record of the data written onto the first storage media in the secondary storage system; and

using the copy on the second storage media of the record of the data written onto the first storage media, synchronizing the primary storage system with the second storage media in the secondary storage system.

16. In a storage system having primary and secondary storage volumes, a method of creating a bitmap for synchronizing storage volumes after suspension of a remote copy pair comprising:

determining if the storage volume for which the bitmap will be created is a primary storage volume;

if it is a primary storage volume, and there is another primary storage volume, then creating the bitmap at the another primary storage volume;

if it is a primary storage volume, and there is not another primary storage volume, but there is another secondary storage volume, then creating the bitmap at the another secondary storage volume; and

if it is not a primary storage volume, then creating a bitmap at another secondary storage volume.

17. A remote copy method using a first volume in a first storage subsystem, a second volume in a second storage subsystem and a third volume in a third storage subsystem, wherein the first volume and the second volume provide a first mirrored pair, wherein the first volume and the third volume provide a second mirrored pair, the method comprising the steps of:

at the first storage subsystem creating and storing a record of changes made to the first volume while the first mirrored pair is suspended,

at the first storage subsystem transmitting the record to the third volume, and

at the third storage subsystem storing the record.

18. A method as in claim 17, further comprising the step of resynchronizing the second volume with the third volume using the record stored in the third storage subsystem when the first storage subsystem is unavailable while the first mirrored pair is suspended.