Data recovery with internet protocol replication with or without full resync

• US 7,275,177 B2
• Filed: 06/25/2003
• Issued: 09/25/2007
• Est. Priority Date: 06/25/2003
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Bergant, Milena, Armangau, Philippe, Srinivasan, Karthikeyan
• Primary Examiner(s): Beausoliel; Robert
• Assistant Examiner(s): Wilson; Yolanda L
• Application Number:
  US10/603,951
• Publication Number:
  US 20050015663A1
• Time in Patent Office: 1,553 Days
• Field of Search: 714/15, 714/5
• US Class Current: 714/5.1
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2069 Management of state, configuration or failover
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

In an asynchronous remote copy system, a primary site keeps a list of data blocks for which changes have been made to a primary file system and transmitted to a secondary file system at a secondary site. When the primary site becomes inoperative, the secondary site begins read/write access to the secondary file system and keeps a snapshot copy of the restart point. Once the primary site becomes operative, the primary file system is restored to the state of the secondary file system at the restart point by using the list of data blocks for obtaining from the snapshot copy the data of the blocks for which changes had been made to the primary file system after the restart point. Then the primary file system is synchronized to the secondary file system, and read/write access is switched back from the secondary file system to the primary file system.

Claims

1. A method of recovery in an asynchronous remote copy system having a primary file system at a primary site and a secondary file system at a secondary site, said method comprising:

in response to the primary site becoming inoperative during read/write access to the primary file system and asynchronous remote copy of changes made to the primary file system being copied to the secondary file system, beginning read/write access to the secondary file system, making a snapshot copy of the secondary file system at the beginning of read/write access to the secondary file system, and keeping a record of changes made to the secondary file system during the read/write access to the secondary file system; and

thereafter,

when the primary site becomes operative, using the snapshot copy to restore the primary file system to the state of the secondary file system existing when read/write access of the secondary file system was begun, and then writing into the primary file system the changes made to the secondary file system during the read/write access to the secondary file system; and

terminating read/write access to the secondary file system, and once the changes made to the secondary file system have been written into the primary file system, restarting read/write access to the primary file system and asynchronous remote copy of changes made to the primary file system being copied to the secondary file system.

2. The method as claimed in claim 1, wherein the asynchronous remote copy of changes made to the primary file system being copied to the secondary file system includes using the Internet Protocol to transmit the changes made to the primary file system over a data network between the primary site and the secondary site.

3. The method as claimed in claim 2, which includes, in response to the primary site becoming operative, transmitting over a first IP pipe changes having been made to the secondary file system during the read/write access to the secondary file system prior to the primary site becoming operative, and concurrently transmitting over a second IP pipe changes having been made to the secondary file system during read/write access to the secondary file system after the primary site becomes operative.

4. The method as claimed in claim 1, wherein the state of the secondary file system existing when read/write access of the secondary file system was begun is a prior state of the primary file system existing before the primary site became inoperative, and the method includes the primary site keeping a list of blocks that have been changed in the primary file system during read/write access to the primary file system, and the snapshot copy is used to restore the primary file system to the state of the secondary file system existing when read/write access of the secondary file system was begun by accessing the list of blocks that have been changed in the primary file system during the read/write access to the primary file system to determine the blocks that have been changed in the primary file system since said prior state of the primary file system, and copying from the snapshot copy to the primary file system the blocks that have been changed in the primary file system since said prior state of the primary file system.

5. A method of recovery in an asynchronous remote copy system having a primary file system at a primary site and a secondary file system at a secondary site, said method comprising:

in response to the primary site becoming inoperative during the asynchronous remote copy of changes made to the primary file system being copied to the secondary file system, beginning read/write access to the secondary file system, and then;

in response to the primary site becoming operative, synchronizing the primary file system to the secondary file system by beginning asynchronous remote copy of changes made to the secondary file system being copied to the primary file system, making a snapshot copy of the secondary file system at the beginning of the asynchronous remote copy of changes made to the secondary file system being copied to the primary file system, synchronizing the primary file system to the snapshot copy of the secondary file system, and once the primary file system has been synchronized to the snapshot copy of the secondary file system, terminating read/write access to the secondary file system and completing the asynchronous remote copy of changes made to the secondary file system being copied to the primary file system; and

once the asynchronous remote copy of changes made to the secondary file system has been completed, restarting the read/write access to the primary file system and asynchronous remote copy of changes made to the primary file system being copied to the secondary file system;

wherein the asynchronous remote copy of changes made to the secondary file system being copied to the primary file system includes transmitting changes made to the secondary file system over a first IP pipe from the secondary site to the primary site, and the synchronizing of the primary file system to the snapshot copy of the secondary file system includes transmitting changes made to the secondary file system over a second IP pipe from the secondary site to the primary site, and wherein the transmission of the changes made to the secondary file system over the first IP pipe is concurrent with the transmission of the changes made to the secondary file system over the second IP pipe.

6. A method of recovery in an asynchronous remote copy system having a primary file system at a primary site and a secondary file system at a secondary site, said method comprising:

in response to the primary site becoming inoperative during read/write access to the primary file system and asynchronous remote copy of changes made to the primary file system being copied to the secondary file system, beginning read/write access to the secondary file system, making a first snapshot copy of the secondary file system at the beginning of read/write access to the secondary file system, and keeping a record of changes made to the secondary file system during the read/write access to the secondary file system; and

thereafter,

when the primary site becomes operative, making a second snapshot copy of the secondary file system, beginning asynchronous remote copy of changes made to the secondary file system since the second snapshot copy being copied to the primary file system, using the first snapshot copy to restore the primary file system to the state of the secondary file system existing when read/write access of the secondary file system was begun, and then writing into the primary file system the changes made to the secondary file system during the read/write access to the secondary file system between the time of the first snapshot copy and the second snapshot copy; and

terminating read/write access to the secondary file system, and once the changes made to the secondary file system have been written into the primary file system, restarting the read/write access to the primary file system and asynchronous remote copy of changes made to the primary file system being copied to the secondary file system.

7. The method as claimed in claim 6, wherein the asynchronous remote copy of changes made to the primary file system being copied to the secondary file system includes using the Internet Protocol to transmit the changes made to the primary file system over a data network between the primary site and the secondary site.

8. The method as claimed in claim 6, wherein the asynchronous remote copy of changes made to the secondary file system since the second snapshot copy being copied to the primary file system includes transmitting changes made to the secondary file system over a first IP pipe from the secondary site to the primary site, and which further includes transmitting changes made to the secondary file system between the first snapshot copy and the second snapshot copy over a second IP pipe from the secondary site to the primary site, and wherein the transmission of the changes made to the secondary file system over the first IP pipe is concurrent with the transmission of the changes made to the secondary file system over the second IP pipe.

9. In an asynchronous remote copy system in which changes made to data blocks of a primary file system at a primary site are transmitted to a secondary file system at a secondary site, the primary site storing a list of the data blocks that have been changed in the primary file system, a method of recovery from a disruption at the primary site, said method comprising:

accessing the list of the data blocks that have been changed in the primary file system to restore the primary file system to a prior state at a restart point, the prior state at the restart point including changes made to the primary file system that have been transmitted to the secondary site, the primary file system being restored by determining from the list the data blocks that have been changed in the primary file system since the restart point, and obtaining from the secondary site the data existing at the time of the restart point in the data blocks that have been changed in the primary file system since the restart point, and writing into the primary file system the data existing at the time of the restart point in the data blocks that have been changed in the primary file system since the restart point.

10. The method as claimed in claim 9, which further includes the secondary site responding to the disruption by making a snapshot copy of the secondary file system at the restart point once all of the changes to the primary file system that have been transmitted to the secondary file system have been written into the secondary file system, and wherein the data existing at the time of the restart point in the data blocks that have been changed in the primary file system since the restart point are obtained from the snapshot copy at secondary site.

11. The method as claimed in claim 9, which further includes the secondary site activating the secondary file system for read/write access once all of the changes to the primary file system that have been transmitted to the secondary file system prior to the disruption have been written into the secondary file system.

12. The method as claimed in claim 9, which further includes the secondary site responding to the disruption by activating the secondary file system for read/write access, and wherein the state of the primary file system at the restart point is the state of the secondary file system when the secondary file system is activated for read/write access.

13. The method as claimed in claim 12, which further includes the secondary site keeping a record of changes made to the secondary file system since the restart point, and once the primary site is operative and after writing into the primary file system the data existing at the time of the restart point in the data blocks that have been changed in the primary file system since the restart point, writing the changes made to the secondary file system since the restart point into the primary file system.

14. The method as claimed in claim 12, which further includes the secondary site keeping a record of changes made to the secondary file system since the restart point, and once the primary site is operative, beginning at an intermediate point replication of changes made to the secondary file system, and after writing into the primary file system the data existing at the time of the restart point in the data blocks that have been changed in the primary file system since the restart point, copying changes made to the secondary file system since the restart point and up to the intermediate point into the primary file system.

15. The method as claimed in claim 14, which further includes beginning playback of the replicated changes made to the secondary file system into the primary file system once the changes made to the secondary file system since the restart point and up to the intermediate point have been copied to the primary file system.

16. The method as claimed in claim 15, which further includes ending read/write access to the secondary file system thereby terminating the replication of changes made to the secondary file system, and once all of the changes made to the secondary file system have been played back into the primary file system, the primary site restarts the transmission of changes made to data blocks of the primary file system to the secondary file system.

17. An asynchronous remote copy system comprising a primary data storage system and a secondary data storage system, the primary data storage system having a primary file system and the secondary data storage system having a secondary file system, the primary data storage system being programmed for read/write access to the primary file system and asynchronous remote copy of changes made to the primary file system being copied to the secondary file system,

wherein the secondary data storage system is programmed to respond to the primary data storage system becoming inoperative during the asynchronous remote copy of changes made to the primary file system being copied to the secondary file system by beginning read/write access to the secondary file system, making a snapshot copy of the secondary file system at the beginning of read/write access to the secondary file system, and keeping a record of changes made to the secondary file system during the read/write access to the secondary file system;

and

wherein the primary data storage system and the secondary data storage system are programmed for recovery when the primary data storage system becomes operative by using the snapshot copy to restore the primary file system to the state of the secondary file system existing when read/write access of the secondary file system was begun, and then writing into the primary file system the changes made to the secondary file system during the read/write access to the secondary file system, terminating read/write access to the secondary file system, and once the changes made to the secondary file system have been written into the primary file system, restarting read/write access to the primary file system and asynchronous remote copy of changes made to the primary file system being copied to the secondary file system.

18. The asynchronous remote copy system as claimed in claim 17, wherein the primary data storage system is programmed to perform the asynchronous remote copy of changes made to the primary file system being copied to the secondary file system by using the Internet Protocol to transmit the changes made to the primary file system over a data network between the primary data storage system and the secondary data storage system.

19. The asynchronous remote copy system as claimed in claim 18, wherein the secondary data storage system is programmed to respond to the primary data storage system becoming operative by transmitting over a first IP pipe changes having been made to the secondary file system during the read/write access to the secondary file system prior to the primary data storage system becoming operative, and concurrently transmitting over a second IP pipe changes having been made to the secondary file system during read/write access to the secondary file system after the primary data storage system becomes operative.

20. The asynchronous remote copy system as claimed in claim 17, wherein the state of the secondary file system existing when read/write access of the secondary file system was begun is a prior state of the primary file system existing before the primary data storage system became inoperative, and wherein the primary data storage system is programmed to keep a list of blocks that have been changed in the primary file system during the read/write access to the primary file system, and to restore the primary file system to the state of the secondary file system existing when read/write access of the secondary file system was begun by accessing the list of blocks that have been changed in the primary file system during the read/write access to the primary file system to determine the blocks that have been changed in the primary file system since said prior state of the primary file system, and copying from the snapshot copy to the primary file system the blocks that have been changed in the primary file system since said prior state of the primary file system.

21. An asynchronous remote copy system comprising a primary data storage system and a secondary data storage system, the primary data storage system having a primary file system and the secondary data storage system having a secondary file system, the primary data storage system being programmed for read/write access to the primary file system and asynchronous remote copy of changes made to the primary file system being copied to the secondary file system,

wherein the secondary data storage system is programmed to respond to the primary data storage system becoming inoperative during the asynchronous remote copy of changes made to the primary file system being copied to the secondary file system by the secondary data storage system beginning read/write access to the secondary file system, and

wherein the primary data storage system and the secondary data storage system are programmed to respond to the primary data storage system becoming operative by synchronizing the primary file system to the secondary file system by beginning asynchronous remote copy of changes made to the secondary file system being copied to the primary file system, making a snapshot copy of the secondary file system at the beginning of the asynchronous remote copy of changes made to the secondary file system being copied to the primary file system, synchronizing the primary file system to the snapshot copy of the secondary file system, and once the primary file system has been synchronized to the snapshot copy of the secondary file system, terminating read/write access to the secondary file system and completing the asynchronous remote copy of changes made to the secondary file system being copied to the primary file system;

and once the asynchronous remote copy of changes made to the secondary file system has been completed, restarting read/write access to the primary file system and asynchronous remote copy of changes made to the primary file system being copied to the secondary file system;

wherein the secondary data storage system is programmed to perform the asynchronous remote copy of changes made to the secondary file system being copied to the primary file system by transmitting changes made to the secondary file system over a first IP pipe from the secondary data storage system to the primary data storage system, and the secondary data storage system is programmed to synchronize the primary file system to the snapshot copy of the secondary file system by transmitting changes made to the secondary file system over a second IP pipe from the secondary data storage system to the primary data storage system, wherein the transmission of the changes made to the secondary file system over the first IP pipe is concurrent with the transmission of the changes made to the secondary file system over the second IP pipe.

22. An asynchronous remote copy system comprising a primary data storage system and a secondary data storage system, the primary data storage system having a primary file system and the secondary data storage system having a secondary file system, the primary data storage system being programmed for read/write access to the primary file system and asynchronous remote copy of changes made to the primary file system being copied to the secondary file system,

wherein the secondary data storage system is programmed to respond to the primary data storage system becoming inoperative during the asynchronous remote copy of changes made to the primary file system being copied to the secondary file system by beginning read/write access to the secondary file system, making a first snapshot copy of the secondary file system at the beginning of read/write access to the secondary file system, and keeping a record of changes made to the secondary file system during the read/write access to the secondary file system;

wherein the secondary data storage system is programmed to respond to the primary data storage system becoming operative by making a second snapshot copy of the secondary file system, and beginning asynchronous remote copy of changes made to the secondary file system since the second snapshot copy being copied to the primary file system; and

wherein the primary data storage system and the secondary data storage system are programmed for using the first snapshot copy to restore the primary file system to the state of the secondary file system existing when read/write access of the secondary file system was begun, and then writing into the primary file system the changes made to the secondary file system during the read/write access to the secondary file system between the time of the first snapshot copy and the second snapshot copy, and terminating read/write access to the secondary file system, and once the changes made to the secondary file system have been written into the primary file system, restarting read/write access to the primary file system and asynchronous remote copy of changes made to the primary file system being copied to the secondary file system.

23. The asynchronous remote copy system as claimed in claim 22, wherein the primary data storage system is programmed to perform the asynchronous remote copy of changes made to the primary file system being copied to the secondary file system by using the Internet Protocol to transmit the changes made to the primary file system over a data network between the primary data storage system and the secondary data storage system.

24. The asynchronous remote copy system as claimed in claim 22, wherein the secondary data storage system is programmed to perform the asynchronous remote copy of changes made to the secondary file system since the second snapshot copy being copied to the primary file system by transmitting changes made to the secondary file system since the second snapshot copy over a first IP pipe from the secondary data storage system to the primary data storage system, and the secondary data storage system is programmed to transmit the changes made to the secondary file system during the read/write access to the secondary file system between the time of the first snapshot copy and the second snapshot copy over a second IP pipe from the secondary data storage system to the primary data storage system, wherein the transmission of the changes made to the secondary file system over the first IP pipe is concurrent with the transmission of the changes made to the secondary file system over the second IP pipe.

25. An asynchronous remote copy system comprising a primary data storage system and a secondary data storage system, the primary data storage system having a primary file system and the secondary data storage system having a secondary file system, the primary data storage system being programmed for read/write access to the primary file system and asynchronous remote copy of changes made to the primary file system being copied to the secondary file system, the primary data storage system storing a list of the data blocks that have been changed in the primary file system;

wherein the primary data storage system and the secondary data storage system are programmed for recovering from a disruption in the asynchronous remote copy of changes made to the primary file system being copied to the secondary file system by accessing the list of the data blocks that have been changed in the primary file system to restore the primary file system to a prior state at a restart point, the prior state at the restart point including changes made to the primary file system that have been transmitted to the secondary data storage system, the primary file system being restored by determining from the list the data blocks that have been changed in the primary file system since the restart point, and obtaining from the secondary data storage system the data existing at the time of the restart point in the data blocks that have been changed in the primary file system since the restart point, and writing into the primary file system the data existing at the time of the restart point in the data blocks that have been changed in the primary file system since the restart point.

26. The asynchronous remote copy system as claimed in claim 25, wherein the secondary data storage system is programmed to respond to the disruption by making a snapshot copy of the secondary file system at the restart point once all of the changes to the primary file system that have been transmitted to the secondary file system have been written into the secondary file system, and wherein the secondary file system is programmed to obtain from the snapshot copy the data existing at the time of the restart point in the data blocks that have been changed in the primary file system since the restart point.

27. The asynchronous remote copy system as claimed in claim 25, wherein the secondary data storage system is programmed to activate the secondary file system for read/write access once all of the changes to the primary file system that have been transmitted to the secondary file system prior to the disruption have been written into the secondary file system.

28. The asynchronous remote copy system as claimed in claim 25, wherein the secondary data storage system is programmed to respond to the disruption by activating the secondary file system for read/write access, and the state of the primary file system at the restart point is the state of the secondary file system when the secondary file system is activated for read/write access.

29. The asynchronous remote copy system as claimed in claim 28, wherein the secondary data storage system is programmed for keeping a record of changes made to the secondary file system since the restart point, and wherein the primary data storage system is programmed for writing into the primary file system the data existing at the time of the restart point in the data blocks that have been changed in the primary file system since the restart point and then writing the changes made to the secondary file system since the restart point into the primary file system.

30. The asynchronous remote copy system as claimed in claim 28, wherein the secondary data storage system is programmed for keeping a record of changes made to the secondary file system since the restart point, and for responding to the primary data storage system becoming operative after the disruption by beginning at an intermediate point replication of changes made to the secondary file system, and copying to the primary data storage system changes made to the secondary file system since the restart point and up to the intermediate point.

31. The asynchronous remote copy system as claimed in claim 30, wherein the primary data storage system is programmed to write into the primary file system the changes made to the secondary file system since the restart point and up to the intermediate point, and then begin playback of the replicated changes made to the secondary file system into the primary file system.

32. The asynchronous remote copy system as claimed in claim 31, wherein the primary data storage system is programmed to restart read/write access to the primary file system and asynchronous remote copy of changes made to the primary file system being copied to the secondary data storage system once all of the changes made to the secondary file system have been played back into the primary file system.