Fault isolation in a network

• US 7,619,979 B2
• Filed: 06/15/2004
• Issued: 11/17/2009
• Est. Priority Date: 06/15/2004
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Yardley, Brent W., Phelan, Thomas G.
• Primary Examiner(s): Rao; Seema S
• Assistant Examiner(s): DUONG, CHRISTINE T
• Application Number:
  US10/869,390
• Publication Number:
  US 20050276214A1
• Time in Patent Office: 1,981 Days
• Field of Search: 370/216, 370/217, 370/221, 370/242, 370/243, 370/244, 370/245, 714/25, 714/48, 714/49
• US Class Current: 370/242
• CPC Class Codes: H04L 41/0659 by isolating or reconfiguring faulty entities
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

A system to isolate a fault to a particular port from among multiple ports in a network. The network typically has a plurality of devices including hosts, storage units, and switch groups that intercommunicate via transceivers. A fault indication is received from one or more of the devices in the network. The fault indication is then processed with a chain of fault indication rules that have been linked together into a binary decision path based on a set of device rules and a data flow model for the network. This permits determining the particular port responsible for the fault, and reporting that port to a user of the network.

Claims

1. A method to isolate a fault in a network, the method comprising:

receiving multiple correlated fault indications from devices in the network, wherein fault indication is a loss of a portion of transmitted information while maintaining routing of data to said device;

processing said correlated fault indications with a chain of fault indication rules linked together into a binary decision path based on a set of device rules and a data flow model for the network to determine a root cause of said fault indications including using attribute data in said device rules to look up port information selected from the group consisting of;

error classification, error propagation, correlation between said ports, and topology data provided by a device provider embodied in said device rules; and

reporting said root cause to a user of the network, wherein said root cause identifies a faulty link where initial information loss occurred.

2. The method of claim 1, wherein:

said processing includes employing at least one of an aggregate type, a classify type, a connected-port type, an event type, a no-fault type and a secondary-counter type of said fault indication rules.

3. The method of claim 1, wherein:

said fault indication rules are integrated before said chain is present.

4. The method of claim 3, wherein:

said fault indication rules are embodied in an externalized form.

5. The method of claim 3, wherein:

said processing is performed in a separate thread.

6. The method of claim 1, wherein:

said processing includes employing at least one said device rule having an error counter

and a device attribute that is at least one of device class, vendor, model, hardware version, and software version.

7. The method of claim 6, wherein:

said device rule includes a counter listing of multiple said error counters.

8. The method of claim 1, wherein:

said processing includes employing at least one said device rule having an error counter attribute that is at least one of error classification, error watermark, error notify threshold, and error contributing counter.

9. The method of claim 8, wherein:

said device rule includes an error counter attribute that is a contributing counter listing of multiple contributing error counters.

10. The method of claim 1, wherein:

said information includes identifying port-information that is unique within the network.

11. A system to isolate a fault in a network including one or more hosts, comprising:

a processor in one said host to receive multiple correlated fault indications from devices in the network, wherein fault indication includes loss of information while maintaining routing of data to said device;

said processor further to determine a faulty link where initial information loss occurred, by processing instances of said correlated fault indications with a chain of fault indication rules linked together into a binary decision path based on a set of device rules and a data flow model for the network, wherein said data flow model is based upon information about instances of ports selected from the group consisting of;

error classification, error propagation, correlation between said ports, topology data embodied in said device rules, and combination thereof; and

said processor to report said faulty link to a user of the network.

12. The system of claim 11, wherein:

said processor processes said fault indication rules based on at least one of an aggregate

type, a classify type, a connected-port type, an event type, a no-fault type, a fault

type and secondary-counter type.

13. The system of claim 12, wherein:

said processor determines each instance of said particular port by processing performed

in a separate thread.

14. The system of claim 11, wherein:

said processor processes said device rules that are based on an error counter and a device attribute that is at least one of device class, vendor, model, hardware version, and software version.

15. The system of claim 14, wherein:

said processor processes said device rules that include a counter listing of multiple said error counters.

16. The system of claim 11, wherein:

said processor processes at least one said device rule having an error counter attribute that

is at least one of error classification, error watermark, error notify threshold, and

error contributing counter.

17. The system of claim 16, wherein:

said processor processes at least one said device rule that includes an error counter

attribute that is a contributing counter listing of multiple contributing error counters.

18. The system of claim 11, wherein:

said processor processes to determine said instances of said particular port said data flow model by looking up information about instances of said ports using attribute data in said device rules.

19. A method to isolate a fault to a particular link among a plurality of links in a storage area network (SAN), wherein the SAN has a plurality of devices including hosts, storage units, and switch groups that intercommunicate via optical transceivers, the method comprising:

receiving multiple correlated recorded fault indications from at least one said device in the SAN, wherein said fault indications are associated with loss of information while maintaining routing of data to said device in receipt of said fault;

wherein said fault indications are provided only through device port counters and are absent from an error log;

processing said correlated fault indications to determine a faulty link where initial information loss occurred based on a chain of fault indication rules linked together into a binary decision path, wherein said fault indication rules are based on a set of device rules and a data flow model for the SAN, including using attribute data in said rules to look up port information instances selected from the group consisting of;

error classification, error propagation, correlation between said ports, topology data embodied in said device rules, and combinations thereof; and

reporting said faulty link port to a user of the SAN.