Automated benchmarking with self customization

• US 5,446,874 A
• Filed: 12/23/1993
• Issued: 08/29/1995
• Est. Priority Date: 12/23/1993
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Waclawsky, John G., Hershey, Paul C., Daugherty, Raymond F.
• Primary Examiner(s): Beausoliel, Jr., Robert W.
• Assistant Examiner(s): Palys, Joseph E.
• Application Number:
  US08/173,530
• Time in Patent Office: 614 Days
• Field of Search: 395/575, 371/20.1, 371/68.2, 371/20.4, 371/20.6, 371/68.2
• US Class Current: 714/1
• CPC Class Codes: H04L 43/00 Arrangements for monitoring or testing data switching networks

Abstract

The method enables realtime establishment and maintenance of a standard of operation for a data communications network. The process begins by monitoring the network over some period of time to build benchmark data sets. The benchmark data sets contain a standard of operation for the network, which is historically categorized by either traffic type or activity. This standard of operation is accumulated by the intelligent monitoring facilities. After a period of accumulation, the benchmark is used to determine whether data taken from current monitoring activity indicates normal network behavior. Network monitoring information is analyzed using criteria in modules that have an interface to an expert system. The criteria modules evaluate the current monitored data against the prior benchmark collected data. The criteria module determines if the current network operating characteristics are outside the bounds of normal behavior. If they are, then alerts and logs of information can be sent to the expert system. The expert system can perform network control routing changes, or close down applications, or allocate additional bandwidth as required. In addition, the expert system can modify the characterization of the currently monitored data for historical purposes by supplying information to the benchmark manager regarding traffic types, activity, heuristic accuracy and changes between current and past behavior. In this manner, auto benchmarking can be accomplished with self customization, in an improved manner.

Claims

1. In a system for analyzing, managing and controlling the output of a data communications network which communicates using a stream of data having characteristic patterns, a method for outputting control signals which are customized by classes of traffic on the network, comprising:

storing a first criteria of rules in an expert system in said system to compare information derived from an event vector for a first class of traffic in said network with a standard for said first class of traffic in said network;

said event vector comprising multiple statistical patterns of traffic classes;

storing in a benchmark manager in said system a first benchmark data set as said standard for said first class of traffic, said first benchmark data set being derived from previously monitored events of said first class of traffic in said network;

inputting to said system an event vector from an Event Driven Interface coupled to a data communications network, in response to an event of said first class of traffic in said network;

accessing said first criteria of rules and comparing information derived from said event vector with said standard, using said first criteria of rules;

outputting from said expert system in in said system, an inference signal in response to said comparison, which characterizes said event;

archiving information derived from said event vector in a second benchmark data set in said system, for subsequent use as said standard;

outputting control signals from a monitor to said network in response to said inference signal to manage and control said first class of traffic.

2. The method of claim 1 which further comprises:

accessing a second criteria of rules to compare information from an event vector for a second class of traffic in said network with a standard for a second class of traffic in said network, in response to said inference signal.

3. The method of claim 1 which further comprises the step of:

displaying a representation of said event classified as said first class of traffic.

4. The method of claim 1 wherein said archiving is in a selected location in response to said inference signal.

5. The method of claim 1 wherein said event vector is prepared over a period whose duration is selected in response to said inference signal.

6. The method of claim 1 which further comprises:

substituting said second benchmark data set for said first benchmark data set as said standard to be used by said first criteria of rules, after a predetermined period of time.

7. The method of claim 6 wherein said predetermined period of time can be modified, as determined by a second criteria of rules in said system.

8. In a system for automated benchmarking using an Event Driven Interface for analyzing, managing and controlling the output of a data communications network which communicates using a stream of data containing different classes of traffic on the network, each class having characteristic patterns, a method for outputting control signals which are customized by classes of traffic on the network, comprising:

storing a first criteria of rules in an expert system in said system to compare information derived from an event vector for a first class of traffic in said network with a standard for said first class of traffic in said network;

said event vector comprising multiple statistical patterns of traffic classes;

storing in a benchmark manager in said system a first benchmark data set as said standard for said first class of traffic, said first benchmark data set being derived from previously monitored events of said first class of traffic in said network;

inputting to said system an event vector from an Event Driven Interface coupled to a data communications network, in response to an event of said first class of traffic in said network;

accessing said first criteria of rules in said expert system and comparing information derived from said event vector with said standard, using said first criteria of rules;

outputting from the expert system in said system, an inference signal in response to said comparison, which characterizes said event;

displaying in a monitor in said system a representation of said event classified as said first class of traffic;

archiving information derived from said event vector in a second benchmark data set in the benchmark manager in said system, for subsequent use as said standard; and

outputting control signals from the monitor to said network in response to the inference signal to manage and control said first class of traffic.

9. The method of claim 8 which further comprises:

accessing a second criteria of rules to compare information from an event vector for a second class of traffic in said network with a standard for a second class of traffic in said network, in response to said inference signal.

10. The method of claim 8 which further comprises the step of:

said display step including logging information derived from said event vector.

11. The method of claim 8 wherein said first class of traffic is batch traffic.

12. The method of claim 8 wherein said first class of traffic is video traffic.

13. The method of claim 8 which further comprises:

substituting said second benchmark data set for said first benchmark data set as said standard to be used by said first criteria of rules, after a predetermined period of time.

14. The method of claim 13 wherein said predetermined period of time can be modified, as determined by a second rule in said system.

15. A system for automated benchmarking with self customization using an Event Driven Interface for analyzing, managing and controlling the output of a data communications network which communicates using a stream of data containing different classes of traffic on the network, each class having characteristic patterns, comprising:

an event vector register coupled to the Event Driven Interface, for receiving event vectors characterizing the behavior of the data communications network;

each event vector descriptive of a class of traffic occurring in the network;

an accumulator means coupled to said event vector register, for accumulating information from said event vectors;

a benchmark data set construction register coupled to said accumulator, for storing information accumulated in said accumulator from said event vectors characterizing said data communications network for a current interval;

an expert system, having an input coupled to said even vector register, for receiving information from said event vectors, said expert system having associated therewith one or more criteria modules which include rules for analyzing said behavior of said data communications network as evidenced by said event vectors relative to historical traffic patterns and pre-determined standards for each criteria module;

a benchmark data set comparison register coupled to another input to said expert system, for storing previous benchmark data sets characterizing said data communications network at a prior instant to said current interval;

said expert system performing an analysis of said data communications network behavior during said current interval using said benchmark data set comparison register and said information input from said event vector register;

said expert system updating said benchmark data set comparison register with the contents of said benchmark data construction register; and

means responsive to the expert system for modifying criteria modules in analyzing the network or initiating network control signals or logging traffic patterns or displaying traffic patterns or initiating an alert generator

whereby said data communications network is analyzed, managed and controlled for its behavior in comparison with dynamically updated benchmark information.

16. A system for controlling a data communications network using an Event Driven Interface for analyzing the output of a data communications network which communicates using a stream of data containing different classes of traffic on the network, each class having characteristic patterns, comprising:

an event vector register coupled to the Event Driven Interface, for receiving event vectors characterizing the behavior of the data communications network;

each event vector descriptive of a class of traffic occurring on the network;

an accumulator means coupled to said event vector register, for accumulating information from said event vectors;

a benchmark data set construction register coupled to said accumulator, for storing information accumulated in said accumulator from said event vectors characterizing said data communications network for a current interval;

an expert system, having an input coupled to said event vector register, for receiving information from said event vectors, said expert system having associated therewith one or more criteria modules which include rules for analyzing said behavior of said data communications network as evidenced by said event vectors, each criteria module applicable to a different class of traffic;

a benchmark data set comparison register coupled to another input to said expert system, for storing previous benchmark data sets characterizing said data communications network at a prior instant to said current interval;

said expert system performing an analysis of said data communications network behavior during said current interval using said benchmark data set comparison register and said information input from said event vector register;

said expert system updating said benchmark data set comparison register with the contents of said benchmark data construction register; and

said expert system outputting control signals to said network through monitor means in response to said analysis.

17. In a system for analyzing the output of a data communications network which communicates using a stream of data containing different classes of traffic on the network, each class having characteristic patterns, a method for outputting control signals which are customized by classes of traffic on the network, comprising:

storing a first criteria of rules in said system to compare information derived from an event vector for a first class of traffic in said network with a first standard for said first class of traffic in said network;

each event vector descriptive of a class of traffic occuring in the network, the combination of vectors definitive of all classes of traffic on the network;

storing a second criteria of rules in said system to compare information derived from an event vector for a second class of traffic in said network with a second standard for said second class of traffic in said network;

storing in said system a first benchmark data set as said first standard for said first class of traffic, said first benchmark data set being derived from previously monitored events of said first class of traffic in said network;

storing in said system a second benchmark data set as said second standard for said second class of traffic, said second benchmark data set being derived from previously monitored events of said second class of traffic in said network;

inputting to said system an event vector from an Event Driven Interface coupled to a data communications network, in response to an event of a class of traffic in said network;

accessing said first criteria of rules and comparing information derived from said event vector with said first standard, using said first criteria of rules, for said first class of traffic;

outputting in said system, a plurality of inference signals in response to said comparison, which characterizes said event;

accessing said second criteria of rules and comparing information from said event vector with said second standard, in response to a first inference signal, for said second class of traffic;

archiving information derived from said event vector corresponding to said second benchmark data set in said system, for subsequent use as said second standard; and

managing and controlling the network using the remaining inference signals.

18. In a system for analyzing the output of a data communications network which communicates using a stream of data containing different classes of traffic on the network, each class having characteristic patterns, a method for outputting control signals which are customized by classes of traffic on the network, comprising:

storing a first criteria of rules in said system to compare information derived from an event vector for a first class of traffic in said network with a first standard for said first class of traffic in said network;

storing a second criteria of rules in said system to compare information derived from an event vector for a second class of traffic in said network with a second standard for said second class of traffic in said network;

storing in said system a first benchmark data set as said first standard for said first class of traffic, said first benchmark data set being derived from previously monitored events of said first class of traffic in said network;

storing in said system a second benchmark data set as said second standard for said second class of traffic, said second benchmark data set being derived from previously monitored events of said second class of traffic in said network;

inputting to said system an event vector from an Event Driven Interface coupled to a data communications network, in response to an event of a class of traffic in said network;

accessing said first criteria of rules and comparing information derived from said event vector with said first standard, using said first criteria of rules, for said first class of traffic;

outputting in said system, a first inference signal in response to said comparison, which characterizes said event;

accessing said second criteria of rules and comparing information from said event vector with said second standard in lieu of said first criteria of rules to customize analysis of the network, in response to said first inference signal, for said second class of traffic; and

outputting control signals to said network to control said second class of traffic.

19. In a system for analyzing the output of a data communications network which communicates using a stream of data containing different classes of traffic on the network, each class having characteristic patterns, a method for outputting control signals which are customized by classes of traffic on the network, comprising:

storing a first criteria of rules in said system to compare information derived from an event vector for a first class of traffic in said network with a standard for said first class of traffic in said network;

each event vector descriptive of a class of traffic occuring in the network, the combination of vectors definitive of all classes of traffic on the network;

storing in said system a first benchmark data set as said standard for said first class of traffic, said first benchmark data set being derived from previously monitored events of said first class of traffic in said network;

inputting to said system an event vector from an Event Driven Interface coupled to a data communications network, in response to an event of said first class of traffic in said network;

accessing said first criteria of rules and comparing information derived from said event vector with said standard, using said first criteria of rules;

outputting in said system, an inference signal in response to said comparison, which characterizes said event;

accessing a second criteria of rules to compare information from an event vector for a second class of traffic in said network with a standard for a second class of traffic in said network, in response to said inference signal; and

outputting control signals to said network to control said first class of traffic and providing modifying signals to customize the first and second criteria of rules according to the traffic patterns.

20. In a system for analyzing the output of a data communications network which communicates using a stream of data containing different classes of traffic on the network, each class having characteristic patterns, a method for outputting control signals which are customized by classes of traffic on the network, comprising:

storing a first criteria of rules in an expert system in said system to compare information derived from an event vector for a first class of traffic in said network with a standard for said first class of traffic in said network;

storing in a benchmark manager in said system a first benchmark data set as said standard for said first class of traffic, said first benchmark data set being derived from previously monitored events of said first class of traffic in said network;

inputting to said system an event vector from an Event Driven Interface coupled to a data communications network, in response to an event of said first class of traffic in said network;

each event vector descriptive of a class of traffic occuring in the network, the combination of vectors definitive of all classes of traffic on the network;

accessing said first criteria of rules and comparing information derived from said event vector with said standard, using said first criteria of rules;

outputting in said system from said expert system, an inference signal in response to said comparison, which characterizes said event;

modifying a duration over which said event vector represents said first class of traffic in said network, in response to said inference signal; and

outputting control signals from a monitor to said network in response to the inference signal to control said first class of traffic.

21. A system for analyzing the output of a data communications network which communicates using a stream of data containing different classes of traffic on the network, each class having characteristic patterns, said system for outputting control signals which are customized by classes of traffic on the network, comprising:

means for storing a first criteria of rules in an expert system in said system to compare information derived from an event vector for a first class of traffic in said network with a standard for said first class of traffic in said network;

each event vector descriptive of a class of traffic occurring in the network, the combination of vectors definitive of all classes of traffic on the network;

means for storing in a benchmark manger in said system a first benchmark data set as said standard for said first class of traffic, said first benchmark data set being derived from previously monitored events of said first class of traffic in said network;

means for inputting to said system an event vector from an Event Driven Interface coupled to a data communications network, in response to an event of said first class of traffic in said network;

means for accessing said first criteria of rules and comparing information derived from said event vector with said standard, using said first criteria of rules;

means for outputting from the expert system in said system, an inference signal in response to said comparison, which characterizes said event;

means for archiving information derived from said event vector in a second benchmark data set in said system, for subsequent use as said standard; and

means for outputting control signals to said network to control said first class of traffic and providing customizing signals to the expert system to modify the first criteria of rules according to the network traffic class.

22. A system for analyzing the output of a data communications network which communicates using a stream of data containing different classes of traffic on the network, each class having characteristic patterns, said system for outputting control signals which are customized by classes of traffic on the network, comprising:

means for storing a first criteria of rules in said system to compare information derived from an event vector for a first class of traffic in said network with a standard for said first class of traffic in said network;

each event vector descriptive of a class of traffic occurring in the network, the combination of vectors definitive of all classes of traffic on the network;

means for storing in said system a first benchmark data set as said standard for said first class of traffic, said first benchmark data set being derived from previously monitored events of said first class of traffic in said network;

means for inputting to said system an event vector from an Event Driven Interface coupled to a data communications network, in response to an event of said first class of traffic in said network;

means for accessing said first criteria of rules and comparing information derived from said event vector with said standard, using said first criteria of rules;

means for outputting in said system, an inference signal in response to said comparison, which characterizes said event, said inference signal (a) modifying criteria sequencing and selection in the expert system or changing network operating parameters and activities or changing the archiving of classes of traffic data;

means for modifying a duration over which said event vector represents said first class of traffic in said network, in response to said inference signal; and

means for outputting control signals to said network to control said first class of traffic and providing signals to modify the first criteria of rules.

23. A system for analyzing the output of a data communications network which communicates using a stream of data containing different classes of traffic on the network, each class having characteristic patterns, said system for outputting control signals which are customized by classes of traffic on the network, comprising:

means for storing a first criteria of rules in an expert system in said system to compare information derived from an event vector for a first class of traffic in said network with a standard for said first class of traffic in said network;

each event vector descriptive of a class of traffic occurring in the network, the combination of vectors definitive of all classes of traffic on the network;

means for storing in a benchmark manager in said system a first benchmark data set as said standard for said first class of traffic, said first benchmark data set being derived from previously monitored events of said first class of traffic in said network;

means for inputting to said system an event vector from an Event Driven Interface coupled to a data communications network, in response to an event of said first class of traffic in said network;

means for accessing said first criteria of rules in said expert system and comparing information derived from said event vector with said standard, using said first criteria of rules;

means for outputting from said expert system in said system, an inference signal in response to said comparison, which characterizes said event;

means for accessing a second criteria of rules in said expert system to compare information from an event vector for a second class of traffic in said network with a standard for a second class of traffic in said network, in response to said inference signal; and

means for outputting control signals to said network in response to the inference signal to control said first class of traffic or providing modifying signals to modify the first and second criteria of rules according to the traffic classes.