Method and apparatus for outage measurement

• US 20040024865A1
• Filed: 07/30/2002
• Published: 02/05/2004
• Est. Priority Date: 07/30/2002
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Huang, Jiandong, Marathe, Madhav, Sejun, Song
• Granted Patent: US 7,149,917 B2
• Time in Patent Office: Days
• US Class Current: 709/224
• CPC Class Codes: H04L 41/0213 Standardised network management protocols, e.g. simple network management protocol [SNMP]
• CPC Class Codes: H04L 41/044 comprising hierarchical management structures
• CPC Class Codes: H04L 41/0604 using filtering, e.g. reduction of information by using priority, element types, position or time
• CPC Class Codes: H04L 41/064 involving time analysis
• CPC Class Codes: H04L 43/0811 by checking connectivity
• CPC Class Codes: H04L 43/10 Active monitoring, e.g. heartbeat, ping or trace-route
• CPC Class Codes: H04L 43/106 using time related information in packets, e.g. by adding timestamps

Abstract

An Outage Measurement System (OMS) monitors and measures outage data at a network processing device. The outage data can be stored in the device and transferred to a Network Management System (NMS) or other correlation tool for deriving outage information. The OMS automates the outage measurement process and is more accurate, efficient and cost effective than pervious outage measurement systems.

Claims

1. A method for detecting outages, comprising:

automatically measuring outages using an outage measurement system located locally in a network processing device.

2. A method according to claim 1 including measuring outages for devices directly attached to the network processing device using the outage measurement system.

3. A method according to claim 1 including measuring outages for local objects in the network processing device.

4. A method according to claim 1 including storing outage data locally in the network processing device.

5. A method according to claim 4 including using a persistent memory device for storing the outage data.

6. A method according to claim 4 including storing the outage data in a Management Information Base (MIB).

7. A method according to claim 6 including using a Simple Network Management Protocol (SNMP) to transfer the outage data in the MIB.

8. A method according to claim 1 including transferring the outage data to a network management system or upper level tool for correlating.

9. A method according to claim 1 including polling for layer-2 outages with the outage measurement system.

10. A method according to claim 1 including polling for layer-3 outages with the outage measurement system.

11. A method according to claim 1 including automatically discovering devices connect to the local network processing device and automatically polling for outages associated with the discovered devices.

12. A method according to claim 1 including:

receiving configuration data from a network management system at the network processing device; and

automatically monitoring for outages with the outage measurement system according to the received configuration data.

13. A method according to claim 12 including retaining the configuration data in a configuration table located in the network processing device.

14. A method according to claim 1 including filtering the local outage data with the outage measurement system.

15. A method for measuring outages in a network, comprising:

identifying single point of failure devices in the network;

selectively locating outage measurement systems in the identified single point of failure devices; and

automatically testing for outages locally in the selected single point of failure devices using the outage measurement systems.

16. A method according to claim 15 including:

storing outage data for the outages in the identified devices;

correlating the outage data with one or more correlation systems; and

identifying outage statistics according to the correlated outage data.

17. A method according to claim 16 wherein one or more of the correlation systems are located on a network management system remote from the identified devices.

18. A method according to claim 15 including polling for Iayer-2 outages using the outage measurement systems and polling for layer-3 outages according to results of the layer-2 polling.

19. A method according to claim 18 including using a combination of the polled layer-2 and layer-3 outages to distinguish between local outages local to the identified devices and remote outages associated with links or devices connected to the identified devices.

20. A method for identifying outages, comprising:

monitoring traffic statistics locally at a network processing device for objects associated with the network processing device; and

using the traffic statistics to identify outages for the objects.

21. A method according to claim 20 including monitoring link status of objects associated with the network processing device and detecting outages according to the monitored traffic statistics and monitored link status.

22. A method according to claim 21 including;

pinging remote devices connected to the network processing device;

monitoring the results of the pinging; and

detecting outages according to the monitored traffic statistics, monitored link status and monitored pinging.

23. A method according to claim 20 including monitoring utilization of a processor in the network processing device and using the monitored utilization to identifying Denial Of Service (DOS) conditions in the network processing device.

24. A method for identifying outages, comprising:

polling for local outages of local objects at the network processing device;

polling for remote outages of remote objects connected to the network processing device; and

comparing the local outages with the remote outages to distinguish between outages caused by the local objects and outages caused by the remote objects.

25. A method according to claim 24 including polling for the remote outages by pinging the remote objects with test packets.

26. A method according to claim 24 including:

deriving an Accumulated Outage Time (AOT) for the local objects;

deriving an Accumulated Outage Time (AOT) for the remote objects; and

distinguishing local object outages from remote object outages by comparing the AOT for the local objects with the AOT for the remote objects.

27. A method according to claim 24 wherein the local objects include local physical and logical interfaces, a local line card, or a local router processor.

28. A method according to claim 24 wherein the remote objects include a remote peer router or remote customer equipment.

29. A method for identifying outages, comprising:

identifying outage events locally at a network processing device; and

providing the outage events to a Network Management Server (NMS) or a correlation tool for outage analysis.

30. A method according to claim 29 including storing outage data for the outage events locally in the network processing device at least until the outage data is provided to the NMS or correlation tool.

31. A method according to claim 30 including storing the outage data in an outage Management Information Base (MIB).

32. A method according to claim 29 including polling for the outage events according to a configuration file sent from the NMS.

33. A method according to claim 32 including modifying which outage events are monitored in the network processing device according to commands sent from the NMS.

34. A method according to claim 29 including storing an object outage table in the network processing device that identifies which objects to poll for outages.

35. A method according to claim 34 including:

automatically discovering remote devices connected to the network processing device; and

automatically updating the object outage table with the discovered remote devices.

36. A method according to claim 29 including:

filtering the outage events identified at the network processing device; and

sending the filtered events to the network management system or correlation tool.

37. A method for network outage monitoring, comprising:

monitoring for outages associated with a configuration of devices connected to a network processing device;

automatically discovering a new configuration of devices connected to the network processing device; and

dynamically updating outage monitoring according to the new configuration of devices.

38. A method according to claim 37 including using a Cisco Discovery Protocol (CDP), Address Resolution Protocol (ARP) protocol, or Internet Control Message Protocol (ICMP) for automatically discovering the new configuration of devices.

39. A method according to claim 37 including:

storing a configuration table identify devices connected to the network processing device; and

automatically updating the configuration table with the new configuration of devices.

40. A network processing device, comprising:

a processor configured to manage outage monitoring for objects associated with the network processing device.

41. A network processing device according to claim 40 including memory for storing outage data for the monitored objects.

42. A network processing device according to claim 41 wherein the memory comprises a persistent storage of outage monitoring data when the network processing device loses power.

43. A network processing device according to claim 40 wherein the outage data is stored in a Management Information Base (MIB).

44. A network processing device according to claim 41 wherein the processor monitors for outages by monitoring local objects within the network processing devices.

45. A network processing device according to claim 44 wherein the local objects are associated with a router processor, line card, or software program within the network processing device.

46. A network processor according to claim 40 wherein the processor initiates pinging of test packets to neighboring devices connected to the network processing device according to outage monitoring results.

47. A network processing device according to claim 40 wherein the processor automatically discovers objects coupled to the network processing device and automatically polls for outages of the discovered objects.

48. A network processing device according to claim 40 wherein the processor determines Accumulated Outage Time (AOT) or Number of Accumulate Failures (NAF) for the monitored objects.

49. A network processing device according to claim 40 including a backup processor and an associated backup memory, the processor storing data from outage monitoring in both the backup memory.

50. A method for measuring outages, comprising:

locally monitoring for outages at a network processing device;

filtering the monitored outages with the local network processing device; and

sending the filtered outages to an outage correlation system.

51. A method according to claim 50 including calculating an Accumulated Outage Time (AOT) outage parameter or a Number of Accumulated Failures (NAF) outage parameter from the monitored outages and sending the AOT or NAF to the correlation system.

52. A method according to claim 51 including deriving a Mean Time Between Failures or a Mean Time To Repair value from the AOT or NAF parameters with the correlation system.

53. A method according to claim 50 including:

identifying different outages associated with a same line card;

filtering the different line card outages into a single line card outage indication; and

sending the single line card outage indication to the outage correlation system.

54. A method according to claim 50 including filtering the monitored outages from a system log file.

55. A method for measuring an outage for a network processor device, comprising:

generating a time stamp for the network processing device;

storing the time stamp locally in the network processing device;

periodically updating the stored time stamp with a most recent time stamp for the network processing device;

retaining the most recent stored time stamp during a network processing device outage; and

using the retained most recent stored time stamp to determine an outage time for the network processing device.

56. A method according to claim 55 including;

identifying a system up time when the network processing device has recovered from the outage; and

comparing the stored time stamp with the system up time to determine the outage time.

57. A method according to claim 55 including periodically storing the most recent time stamp in a persistent memory.

58. A method according to claim 55 wherein the outage comprises a power loss to the network processing device.

59. A method according to claim 55 wherein the time stamp stored in the network processing device is updated about once every minute.

60. An electronic storage medium for storing computer executable code for detecting outages, the computer executable code comprising:

code for automatically measuring outages using an outage measurement system located locally in a network processing device.

61. An electronic storage medium according to claim 60 including code for storing outage data locally in the network processing device.

62. An electronic storage medium according to claim 60 including code for transferring the outage data to a network management system or upper level tool for correlating.

63. An electronic storage medium according to claim 60 including code for automatically discovering devices connect to the local network processing device and automatically polling for outages associated with the discovered devices.

64. An electronic storage medium according to claim 60 including:

code for receiving configuration data from a network management system at the network processing device; and

code for automatically monitoring for outages with the outage measurement system according to the received configuration data.

65. A system for detecting outages, comprising:

means for automatically measuring outages using an outage measurement system located locally in a network processing device.

66. A system according to claim 65 including means for storing outage data locally in the network processing device.

67. A system according to claim 65 including means for transferring the outage data to a network management system or upper level tool for correlating.

68. A system according to claim 65 including means for automatically discovering devices connected to the local network processing device and automatically polling for outages associated with the discovered devices.

69. A system according to claim 65 including:

means for receiving configuration data from a network management system at the network processing device; and

means for automatically monitoring for outages with the outage measurement system according to the received configuration data.