Disaster recovery

• US 7,814,050 B2
• Filed: 10/22/2002
• Issued: 10/12/2010
• Est. Priority Date: 10/22/2002
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Brocade Communications Systems, Inc. (Broadcom, Inc.)
• Inventors: Van Horn, Steve, Mehta, Rahul, Glitsch, Hans, Place, Paul
• Primary Examiner(s): Wong; Don
• Assistant Examiner(s): Thai; Hanh B
• Application Number:
  US10/277,431
• Publication Number:
  US 20040078397A1
• Time in Patent Office: 2,912 Days
• Field of Search: 707 1- 2, 707/104.1, 707609-610, 714 1- 4, 714/10, 714/100, 711/147, 711/151, 711/158
• US Class Current: 707/609
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups

Abstract

File system disaster recovery techniques provide automated monitoring, failure detection and multi-step failover from a primary designated target to one of a designated group of secondary designated targets. Secondary designated targets may be prioritized so that failover occurs in a prescribed sequence. Replication of information between the primary designated target and the secondary designated targets allows failover in a manner that maximizes continuity of operation. In addition, user-specified actions may be initiated on failure detection and/or on failover operations and/or on failback operations.

Claims

1. A logical namespace disaster recovery method, comprising:

monitoring shared directories in a logical namespace;

detecting an access failure to one of the monitored shared directories, the failed shared directory having at least two shared directories associated therewith and designated as alternate shared directories, each of the alternate shared directories having replicate of data associated with the failed shared directory;

identifying the alternate shared directories associated with the failed shared directory;

selecting one of the alternate shared directories in accordance with a specified priority scheme; and

swapping the selected alternate shared directory for the failed shared directory.

2. The method of claim 1, wherein the act of monitoring comprises:

issuing a command to a shared directory; and

waiting for an appropriate response from the shared directory in response to the issued command.

3. The method of claim 2, wherein the act of issuing a command comprises issuing a directory listing command.

4. The method of claim 2, wherein the act of detecting comprises determining that an appropriate reply to the issued command was not received.

5. The method of claim 4, wherein the act of determining that an appropriate response to the issued command was not received comprises, not receiving a reply responsive to the issued command within a specified time period.

6. The method of claim 1, wherein the act of monitoring further comprises periodically replicating data associated with one or more of the monitored shared directories to a second storage location in the logical namespace.

7. The method of claim 1, wherein the act of monitoring further comprises periodically replicating data associated with one or more of the monitored shared directories to one or more alternate storage locations.

8. The method of claim 7, wherein the act of replicating data to one or more alternate storage locations comprises replicating data to storage locations identified by the alternate shared directories.

9. The method of claim 1, wherein the act of selecting in accordance with the specified priority scheme comprises selecting in accordance with a user-specified priority scheme.

10. The method of claim 9, wherein the act of selecting in accordance with a user-specified priority scheme comprises selecting the one alternate shared directory having the most recent replicate of data associated with the failed shared directory.

11. The method of claim 9, wherein the act of selecting in accordance with a user-specified priority scheme comprises selecting in accordance with a user-designated order.

12. The method of claim 9, further comprising:

correcting access to the failed shared directory;

synchronizing data of the corrected shared directory with data of the selected alternate shared directory; and

returning access to the corrected shared directory.

13. The method of claim 1, wherein the act of swapping comprises substituting the selected alternate shared directory for the failed shared directory so that a message directed to the failed shared directory is sent to the selected alternate shared directory in a manner transparent to a user issuing the command.

14. The method of claim 1, further comprising invoking a user-specified action after the act of detecting and before the act of swapping.

15. The method of claim 14, wherein the act of invoking a user-specified action comprises invoking a routine to record the detected failure in a log.

16. The method of claim 1, further comprising invoking a user-specified action after the act of swapping.

17. The method of claim 16, wherein the act of invoking a user-specified action comprises invoking a routine to record the act of swapping the selected shared directory for the failed shared directory.

18. The method of claim 1, wherein the logical namespace disaster recovery method comprises a distributed file system (DFS) disaster recovery method.

19. The method of claim 1, wherein the act of identifying comprises identifying a link replica associated with the failed shared directory.

20. A program storage device, readable by a programmable control device, comprising instructions stored on the program storage device for causing the programmable control device to:

monitor shared directories in a logical namespace;

detect an access failure to one of the monitored shared directories, the failed shared directory having at least two shared directories associated therewith and designated as alternate shared directories, each of the alternate shared directories having replicate of data associated with the failed shared directory;

identify the alternate shared directories associated with the failed shared directory;

select one of the alternate shared directories in accordance with a specified priority scheme; and

swap the selected alternate shared directory for the failed shared directory.

21. The program storage device of claim 20, wherein the instructions to monitor comprise instructions to:

issue a command to a shared directory; and

wait for an appropriate response from the shared directory in response to the issued command.

22. The program storage device of claim 21, wherein the instruction to issue a command comprises an instruction to issue a directory listing command.

23. The program storage device of claim 20, wherein the instructions to monitor further comprise instructions to periodically replicate data associated with one or more of the monitored shared directories to a second storage location.

24. The program storage device of claim 23, wherein the instructions to periodically replicate data to a second storage location comprise instructions to replicate data to a location identified by a designated alternate shared directory associated with the monitored shared directory.

25. The program storage device of claim 21, wherein the instructions to detect comprise instructions to determine that an appropriate reply to the issued command was not received.

26. The program storage device of claim 20, wherein the instructions to swap comprise instructions to substitute the selected alternate shared directory for the failed shared directory so that a message directed to the failed shared directory is sent to the selected alternate shared directory in a manner transparent to a user issuing the command.

27. The program storage device of claim 20, further comprising instructions to invoke a user-specified action after the instructions to detect and before the instructions to swap.

28. The program storage device of claim 20, further comprising instructions to invoke a user-specified action after the instructions to swap.

29. The program storage device of claim 20, wherein the instructions to select in accordance with the specified priority scheme comprises instructions to select in accordance with a user-specified priority scheme.

30. The program storage device of claim 29, wherein the instructions to select in accordance with a user-specified priority scheme comprises instructions to select the one alternate shared directory having the most recent replicate of data associated with the failed shared directory.

31. The program storage device of claim 29, wherein the instructions to select in accordance with a user-specified priority scheme comprises instructions to select in accordance with a user-designated order.

32. The program storage device of claim 20, wherein the instructions cause the programmable control device to:

correct access to the failed shared directory;

synchronize data of the corrected shared directory with data of the selected alternate shared directory; and

return access to the corrected shared directory.

33. A logical namespace disaster recovery system, comprising:

a plurality of logical shared directories, each logical shared directory associated with a physical shared directory;

logical namespace means for routing requests directed to a designated logical shared directory to an associated physical shared directory, said logical namespace means operatively coupled to each of the logical shared directories;

monitor means for actively monitoring the physical shared directories; and

recovery means for receiving notification from the monitor means that a physical shared directory has failed and causing the logical namespace means to;

identify at least two alternate physical shared directories associated with the failed physical shared directory, each of the alternate physical shared directories having replicate of data associated with the failed physical shared directory

select one of the alternate physical shared directories in accordance with a specified priority scheme, and

substitute the selected alternate physical shared directory for the failed physical shared directory.

34. The disaster recovery system of claim 33, wherein the logical namespace means comprises a distributed file system (DFS) logical namespace.

35. The disaster recovery system of claim 34, wherein the logical namespace means identifies a secondary physical shared directory through a DFS link-replica means.

36. The disaster recovery system of claim 35, wherein a designated primary physical shared directory is associated with a plurality of secondary physical shared directories.

37. The disaster recovery system of claim 34, wherein the recovery means for causing the logical namespace means to substitute a designated alternate physical shared directory for the failed physical shared directory comprises using DFS application programming interface (API) commands to:

remove the failed physical shared directory from active use in the DFS; and

using a designated alternate physical shared directory for the failed physical shared directory in the DFS.

38. The disaster recovery system of claim 33, wherein the monitor means comprises a means for issuing periodic queries to at least one of the physical shared directories to determine if the logical shared directory is operational.

39. The disaster recovery system of claim 38, wherein the means for issuing periodic queries comprises issuing directory listing queries.

40. The disaster recovery system of claim 33, wherein the recovery means selects the one designated alternate physical shared directory in accordance with a user-specified priority scheme.

41. The disaster recovery system of claim 40, wherein to select in accordance with a user-specified priority scheme, the recovery means causes the logical namespace means to select in accordance with a user-designated order.

42. The disaster recovery system of claim 40, wherein to select in accordance with a user-specified priority scheme, the recovery means causes the logical namespace means to select the one alternate shared directory having the most recent replicate of data associated with the failed shared directory.

43. The disaster recovery system of claim 33, further comprising an execution means for executing a user-specified action after the monitor means detects failure of a physical shared directory.

44. The disaster recovery system of claim 43, further comprising a second execution means for executing a second user-specified action after the recovery means causes the logical namespace means to substitute a designated alternate physical shared directory for the failed physical shared directory.

45. The disaster recovery system of claim 33, further comprising a replication means for replicating information associated with a first physical shared directory to a second physical shared directory.

46. The disaster recovery system of claim 45, wherein the replication means replicates information associated with the first physical shared directory to a plurality of secondary physical shared directories.

47. The disaster recovery system of claim 33, wherein the monitor means comprises a first computer executable module executing on a first computer system and the recovery means comprises a second computer executable module executing on a second, different, computer system.

48. The disaster recovery system of claim 33, wherein the logical namespace means and the disaster recovery means comprise separate computer executable modules executing on a common computer system.

49. The disaster recovery system of claim 33, wherein the recovery means causes the logical namespace means to:

correct access to the failed shared directory,

synchronize data of the corrected shared directory with data of the selected alternate shared directory, and

return access to the corrected shared directory.