System and method for domain failure analysis of a storage area network

• US 7,930,583 B1
• Filed: 09/14/2006
• Issued: 04/19/2011
• Est. Priority Date: 09/14/2006
• Status: Active Grant

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Symantec Operating Corporation (Gen Digital Inc.)
• Inventors: Nandi, Siddhartha, Nagineni, Venkata Sreenivasa Rao, Vemuri, Hari Krishna
• Primary Examiner(s): Wilson; Yolanda L
• Application Number:
  US11/521,039
• Time in Patent Office: 1,678 Days
• Field of Search: 714/4, 714/43, 714/48, 714/39, 710/316, 709/220, 709/221, 709/223, 709/224
• US Class Current: N/A
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: H04L 41/065 involving logical or physical relationship, e.g. grouping and hierarchies
• CPC Class Codes: H04L 41/0677 Localisation of faults
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Systems, methods, apparatus and software can implement a SAN monitoring scheme for determining changes in SAN topology, such as device failure and state changes. These changes are recorded in a SAN topology data structure. Information in the SAN topology data structure is used, for example, to identify a suspect path or set of paths, and to make decisions about communications pathways used by a multipath device driver.

Claims

1. A method comprising:

creating a storage area network (SAN) topology data structure using data comprising SAN device connectivity information, and SAN device status information, wherein the data is gathered via calls to a host bus adapter (HBA);

loading a module into a multipath device driver, wherein

the module comprises device-specific procedures augmenting an existing set of general procedures, and

the module is specific to one of a plurality of devices within a SAN topology;

receiving, at the HBA, a message indicating occurrence of an event related to a change in the SAN topology;

obtaining, from the HBA, information from the received message, wherein the information describes a change in a SAN device;

updating the SAN topology data structure according to the information from the received message;

using information in the updated SAN topology data structure to identify a suspect path for use by the multipath device driver; and

selecting, using the updated SAN topology data structure, a path different from the suspect path, wherein

the selecting is based in part on the module loaded into the multipath device driver.

2. The method of claim 1 wherein the message is generated in response to a failure of a SAN component.

3. The method of claim 1 wherein the message is a registered state change notification (RSCN) message.

4. The method of claim 1 further comprising:

causing the HBA to register with one or more of the SAN device or another SAN device in order to receive the message.

5. The method of claim 1 further comprising:

polling the HBA at one of a periodic interval and an arbitrary interval to determine if the HBA has received the message.

6. The method of claim 1 further comprising:

requesting that the HBA send the information from the received message to a software process in response to the receiving.

7. The method of claim 1 further comprising:

registering with the HBA using an application programming interface (API).

8. The method of claim 1 wherein the event related to a change in topology comprises one or more of:

a device failure, addition of a device, a connection failure, or a device status change.

9. The method of claim 1 wherein the SAN topology data structure further comprises:

a table describing SAN devices and corresponding ports.

10. The method of claim 1 wherein the SAN topology data structure further comprises:

at least one path between a host device and the SAN device, wherein the at least one path describes a plurality of components coupled between the host device and the SAN device.

11. The method of claim 1 wherein the selecting is performed by one of:

the multipath device driver, or

an event monitor.

12. The method of claim 1 wherein the using information in the SAN topology data structure to identify the suspect path for use by the multipath device driver further comprises:

probing the suspect path.

13. The method of claim 11 wherein the using information in the SAN topology data structure to identify the suspect path for use by the multipath device driver further comprises:

flagging the suspect path.

14. The method of claim 1 wherein the multipath device driver performs one or more of:

selecting one of a plurality of communication pathways to at least one storage device;

selecting one or more sub-devices of the at least one storage device which will be affected due to a communication pathway failure;

selecting an alternate communication pathway in case of a failure of one of the plurality of communication pathways;

changing a current communications pathway from a first one of the plurality of communication pathways to a second one of the plurality of communication pathways;

or

selectively transmitting I/O operations along at least two of the plurality of communication pathways to the at least one storage device.

15. The method of claim 1 further comprising:

requesting a list of interconnected devices in the SAN;

receiving the list of interconnected devices in the SAN; and

updating the SAN topology data structure according to the received list of interconnected devices in the SAN.

16. The method of claim 15 further comprising:

querying at least one device from the list of interconnected devices in the SAN for a list of device ports;

receiving the list of device ports; and

updating the SAN topology data structure according to the received list of device ports.

17. The method of claim 16

querying at least one device port from the list of device ports for one or more of an attached port names list or port type information;

receiving the one or more of the attached port names list or port type information; and

updating the SAN topology data structure according to the one or more of the attached port names list or the port type information.

18. A system comprising instructions stored in a memory device, wherein the instructions are configured to:

create a storage area network (SAN) topology data structure using data comprising SAN device connectivity information, and SAN device status information, wherein the data is gathered via calls to a host bus adapter (HBA);

load a module into a multipath device driver, wherein

the module comprises device-specific procedures augmenting an existing set of general procedures, and

the module is specific to one of a plurality of devices within a SAN topology;

obtain, from the HBA coupled to the processor and coupled to a SAN, information about a change in the SAN topology;

update the SAN topology data structure according to the information about the change in SAN topology;

direct input/output (I/O) operations along at least one of a plurality of communication pathways of the processor to at least one storage device in the SAN,

use information in the SAN topology data structure to identify a suspect path of the plurality of communication pathways, and

select a path of the plurality of communication pathways different from the suspect path, based on the updated SAN topology data structure, wherein the path is selected based in part on the module loaded into the multipath device driver.

19. The system of claim 18 further comprising:

the host bus adapter configured to;

receive the information about the change in topology of the SAN from a device in the SAN; and

store the information about the change in topology of the SAN.

20. The system of claim 19 wherein the host bus adapter is further configured to automatically send the information about the change in topology of the SAN to an event monitor.

21. The system of claim 19 wherein the event monitor is further configured to request the information about the change in topology of the SAN from the host bus adapter.

22. The system of claim 19 wherein the host bus adapter is further configured to register with the device in the SAN in order to receive the information about the change in topology of the SAN.

23. The system of claim 18 wherein the information about the change in topology of the SAN is from a registered state change notification (RSCN) message.

24. The system of claim 18 wherein the SAN topology data structure further comprises:

a table describing SAN devices and corresponding ports.

25. The system of claim 18 wherein the SAN topology data structure further comprises:

at least one path between a host device and a SAN device, wherein the at least one path describes a plurality of components coupled between the host device and the SAN device.

26. The system of claim 18 further comprising:

a memory; and

a processor coupled to the memory, wherein one or more of an event monitor or the multipath device driver are encoded as instructions stored in the memory and executable on the processor.

27. The system of claim 18 wherein the multipath device driver is further configured to do one or more of:

probe the suspect path;

or

flag the suspect path.

28. A computer readable storage medium comprising program instructions executable on a processor, the computer readable medium encoding the program instructions, wherein the program instructions are operable to implement each of:

creating a storage area network (SAN) topology data structure using data comprising SAN device connectivity information, and SAN device status information, wherein the data is gathered via calls to a host bus adapter (HBA);

loading a module into a multipath device driver, wherein

the module comprises device-specific procedures augmenting an existing set of general procedures, and

the module is specific to one of a plurality of devices within a SAN topology;

obtaining, from the HBA, information about a change in the SAN topology;

updating the SAN topology data structure according to the information about the change in SAN topology;

using information in the updated SAN topology data structure to identify a suspect path for use by the multipath device driver; and

selecting a path different from the suspect path, wherein

the selecting is based in part on the module loaded into the multipath device driver.

29. The computer readable storage medium of claim 28 further comprising program instructions operable to:

cause the host bus adapter to register with at least one SAN device in order to receive the information about the change in topology of the SAN.

30. The computer readable storage medium of claim 28 further comprising program instructions operable to:

poll the host bus adapter at one of a periodic interval and an arbitrary interval to determine if the host bus adapter has received a message.

31. The computer readable storage medium of claim 28 further comprising program instructions operable to:

request that the host bus adapter send the information about the change in topology of the SAN to a software process when the host bus adapter receives the information about the change in topology of the SAN.

32. The computer readable storage medium of claim 28 wherein the SAN topology data structure further comprises one or more of:

a table describing SAN devices and corresponding ports;

or

at least one path between a host device and a SAN device, wherein the at least one path describes a plurality of components coupled between the host device and the SAN device.

33. The computer readable storage medium of claim 28 further comprising program instructions operable to one or more of:

probe the suspect path;

or

flag the suspect path.