Method and system for end-to-end problem determination and fault isolation for storage area networks

• US 6,636,981 B1
• Filed: 01/06/2000
• Issued: 10/21/2003
• Est. Priority Date: 01/06/2000
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Barnett, Barry Stanley, Bossen, Douglas Craig
• Primary Examiner(s): Beausoliel, Robert
• Assistant Examiner(s): MASKULINSKI, MICHAEL C
• Application Number:
  US09/478,306
• Time in Patent Office: 1,384 Days
• Field of Search: 714/4, 709/224
• US Class Current: 714/4.5
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0781 Error filtering or prioritizing based on a policy defined by the user or on a policy defined by a hardware/software module, e.g. according to a severity level
• CPC Class Codes: H04L 41/022 Multivendor or multi-standard integration
• CPC Class Codes: H04L 41/0609 based on severity or priority
• CPC Class Codes: H04L 41/064 involving time analysis
• CPC Class Codes: H04L 41/065 involving logical or physical relationship, e.g. grouping and hierarchies

Abstract

A method and system for problem determination and fault isolation in a storage area network (SAN) is provided. A complex configuration of multi-vendor host systems, FC switches, and storage peripherals are connected in a SAN via a communications architecture (CA). A communications architecture element (CAE) is a network-connected device that has successfully registered with a communications architecture manager (CAM) on a host computer via a network service protocol, and the CAM contains problem determination (PD) functionality for the SAN and maintains a SAN PD information table (SPDIT). The CA comprises all network-connected elements capable of communicating information stored in the SPDIT. The CAM uses a SAN topology map and the SPDIT are used to create a SAN diagnostic table (SDT). A failing component in a particular device may generate errors that cause devices along the same network connection path to generate errors. As the CAM receives error packets or error messages, the errors are stored in the SDT, and each error is analyzed by temporally and spatially comparing the error with other errors in the SDT. If a CAE is determined to be a candidate for generating the error, then the CAE is reported for replacement if possible.

Claims

1. A method for processing errors within a storage area network (SAN), the method comprising the computer implemented steps of:

generating a SAN topology map comprising a table in which each row of the SAN topology table is uniquely mapped to a communication architecture element (CAE) and each column of the SAN topology table is uniquely mapped to the CAE, wherein the CAE is a network-connected device that has successfully registered with a communications architecture manager (CAM) via a network service protocol, wherein the CAM contains problem determining (PD) functionality for the SAN and maintains the SPDIT;

generating a SAN problem determination information table (SPDIT); and

generating a SAN diagnostic table (SDT) using the SAN topology map and the SPDIT.

2. The method of claim 1 wherein a communication architecture (CA) comprises all network-connected elements capable of communicating information stored in the SPDIT, and wherein the SPDIT comprises at least one data record associated with each product or element on the CA.

3. The method of claim 2 wherein at least one data record associated with each product or element on the CA further comprises one or more data items selected from the group consisting of:

product vendor information;

product identifier information;

information concerning a type of communication link supported by the product or element; and

/or information concerning a type of error information to be reported by the product or element.

4. The method of claim 3 wherein the type of error information indicates whether the product or element supports Extended Link Services (ELS) Registered Link Incident Record (RLIR).

5. The method of claim 3 wherein the SDT stores information from the SAN topology map and errors received by the CAM from CAEs.

6. A method for processing errors within a storage area network (SAN), the method comprising the computer-implemented steps of:

receiving an error message at a communication architecture manager (CAM), wherein the CAM comprises problem determination (PD) functionality for the SAN, wherein a CAM maintains a SAN PD information table (SPDIT), and wherein a communication architecture (CA) managed by the CAM comprises all network-connected elements capable of communicating information stored in the SPDIT;

generating a SAN topology map comprising a table in which each row of the SAN topology tables uniquely mapped to a communication architecture element (CAE) and each column of the SAN topology table is uniquely mapped to the CAE, wherein the CAE is a network-connected device that has successfully registered with the communications architecture manager (CAM) via a network service protocol, wherein the CAM contains problem determination (PD) functionality for the SAN and maintains the SPDIT; and

processing the error message using a real-time diagnostic algorithm (RDA).

7. The method of claim 6 wherein a network supporting the CA comprises in-band Fibre Channel communication links and out-of-band communication links.

8. The method of claim 6 wherein the SAN further comprises:

a plurality of storage devices connected to the network; and

a plurality of host computers connected to the network, wherein at least one of the plurality of host computers comprises a CAM;

wherein at least some of the error messages are generated by at least some of the plurality of storage devices and host computers.

9. The method of claim 6 further comprising:

map;

generating a SAN diagnostic table (SDT) using the SAN topology map and the SPDIT.

10. The method of claim 9 further comprising:

analyzing the received error message using a temporal correlation window (TCW) value to temporally constrain fault isolation determination while searching for temporally-related error messages previously received by the CAM and stored within the SDT; and

analyzing the received error message using a spatial correlation path data structure (SCP) to spatially constrain fault isolation determination while searching for spatially-related error messages previously received by the CAM and stored within the SDI.

11. The method of claim 10 further comprising:

analyzing the received error message using error severity weightings according to a type of error indicated by the received error message.

12. A data processing system for communicating error information in a storage area network (SAN), the data processing system comprising:

a network comprising in-band Fibre Channel communication links and out-of-band communication links, wherein the network supports a communications architecture (CA);

a plurality of storage devices connected to the network;

a plurality of host computers connected to the network, wherein at least one of the plurality of host computers comprises a communications architecture manager (CAM) containing problem determination (PD) functionality, wherein a CAM maintains a SAN PD information table (SPDIT), and wherein the CA comprises all network-connected elements capable of communicating information stored in the SPDIT, and wherein the at least one of the plurality of host computer systems includes a SAN topology map comprising a table in which each row of the SAN topology table is uniquely mapped to a communication architecture element (CAE) and each column of the SAN topology table is uniquely mapped to the CAE, wherein the CAE is a network-connected device that has successfully registered with the communications architecture manager (CAM) via a network service protocol, wherein the CAM contains problem determination (PD) functionality for the SAN and maintains the SPDIT.

13. The data processing system of claim 12 further comprising:

a plurality of CAMs, wherein the CA comprises a primary CAM and one or more secondary CAMs, wherein a secondary CAM operates redundantly for a primary CAM.

14. The data processing system of claim 12 wherein the CA further comprises one or more CA elements (CAEs) and one or more CA non-participants (CANs), wherein a CAE is a network-connected device that has successfully registered with a CAM via a network service protocol, and wherein a CAN is a network-connected device that has not registered with a CAM yet known to be present via a SAN topology discovery process.

15. The data processing system of claim 12 wherein the in-band Fibre Channel communication links and the out-of-band communication links are provided by a single, physical communication link.

16. A data processing system for processing errors within a storage area network (SAN), the data processing system comprising:

first generating means for generating a SAN topology map comprising a table in which each row of the SAN topology table is uniquely mapped to a communication architecture element (CAE) and each column of the SAN topology table is uniquely mapped to the CAE, wherein the CAE is a network-connected device that has successfully registered with a communications architecture manager (CAM) via a network service protocol, wherein the CAM contains problem determination (PD) functionality for the SAN and maintains the SPDIT;

second generating means for generating a SAN problem determination information table (SPDIT); and

third generating means for generating a SAN diagnostic table (SDT) using the SAN topology map and the SPDIT.

17. The data processing system of claim 16 wherein a communication architecture (CA) comprises all network-connected elements capable of communicating information stored in the SPDIT, and wherein the SPDIT comprises at least one data record associated with each product or element on the CA.

18. The data processing system of claim 17 wherein at least one data record associated with each product or element on the CA further comprises one or more data items selected from the group consisting of:

product vendor information;

product identifier information;

information concerning a type of communication link supported by the product or element; and

/or information concerning a type of error information to be reported by the product or element.

19. The data processing system of claim 18 wherein the type of error information indicates whether the product or element supports Extended Link Services (ELS) Registered Link Incident Record (RLIR).

20. The data processing system of claim 18 wherein the SDT stores information from the SAN topology map and errors received by the CAM from CAEs.

21. A data processing system for processing errors within a storage area network (SAN), the data processing system comprising:

receiving means for receiving an error message at a communication architecture manager (CAM), wherein the CAM comprises problem determination (PD) functionality for the SAN, wherein a CAM maintains a SAN PD information table (SPDIT), and wherein a communication architecture (CA) managed by the CAM comprises all network-connected elements capable of communicating information stored in the SPDIT;

generating means for generating a SAN topology map comprising a table in which each row of the SAN topology table is uniquely mapped to a communication architecture element (CAE) and each column of the SAN topology table is uniquely mapped to the CAE, wherein the CAE is a network-connected device that has successfully registered with the communications architecture manager (CAM) via a network service protocol, wherein the CAM contains problem determination (PD) functionality for the SAN and maintains the SPDIT; and

processing means for processing the error message using a real-time diagnostic algorithm (RDA).

22. The data processing system of claim 21 wherein a network supporting the CA comprises in-hand Fibre Channel communication links and out-of-band communication links.

23. The data processing system of claim 21 wherein the SAN further comprises:

a plurality of storage devices connected to the network; and

a plurality of host computers connected to the network, wherein at least one of the plurality of host computers comprises a CAM;

wherein at least some of the error messages are generated by at least some of the plurality of storage devices and host computers.

24. The data processing system of claim 21 further comprising:

generating means for generating a SAN diagnostic table (SDT) using the SAN topology map and the SPDIT.

25. The data processing system of claim 24 further comprising:

first analyzing means for analyzing the received error message using a temporal correlation window (TCW) value to temporally constrain fault isolation determination while searching for temporally-related error messages previously received by the CAM and stored within the SDT; and

second analyzing means for analyzing the received error message using a spatial correlation path data structure (SCP) to spatially constrain fault isolation determination while searching for spatially-related error messages previously received by the CAM and stored within the SDT.

26. The data processing system of claim 25 further comprising:

third analyzing means for analyzing the received error message using error severity weightings according to a type of error indicated by the received error message.

27. A computer program product in a computer-readable medium for use in a data processing system for processing errors within a storage area network (SAN), the computer program product comprising:

first instructions for generating a SAN topology map comprising a table in which each row of the SAN topology table is uniquely mapped to a communication architecture element (CAE) and each column of the SAN topology table is uniquely mapped to the CAE, wherein the CAE is a network-connected device that has, successfully registered with a communications architecture manager (CAM) via a network service protocol, wherein the CAM contains problem determination (PD) functionality for the SAN and maintains the SPDIT;

second instructions for generating a SAN problem determination information table (SPDIT); and

third instructions for generating a SAN diagnostic table (SDT) using the SAN topology map and the SPDIT.

28. A computer program product in a computer-readable medium for use in a data processing system for processing errors within a storage area network (SAN), the computer program product comprising:

first instructions for receiving an error message at a communication architecture manager (CAM), wherein the CAM comprises problem determination (PD) functionality for the SAN, wherein a CAM maintains a SAN PD information table (SPDIT), and wherein a communication architecture (CA) managed by the CAM comprises all network-connected elements capable of communicating information stored in the SPDIT; and

second instructions for generating a SAN topology map comprising a table in which each row of the SAN topology table is uniquely mapped to a communication architecture element (CAE) and each column of the SAN topology table is uniquely mapped to the CAE, wherein the CAE is a network-connected device that has successfully registered with the communications architecture manager (CAM) via a network service protocol wherein the CAM contains problem determination (PD) functionality for the SAN and maintains the SPDIT; and

third instructions for processing the error message using a real-time diagnostic algorithm (RDA).