Methods, systems and computer program products for scheduled network performance testing

• US 6,397,359 B1
• Filed: 01/19/1999
• Issued: 05/28/2002
• Est. Priority Date: 01/19/1999
• Status: Expired due to Term

• Current Assignee: NetIQ Corporation (Open Text Corporation)
• Original Assignee: NetIQ Corporation (Open Text Corporation)
• Inventors: McCorry, Mark Eric, Quan, David Vi Hien, Chandra, Vikas, Wood, John Lee, Selvaggi, Christopher David, Schwaller, Peter James
• Primary Examiner(s): Ray, Gopal C.
• Application Number:
  US09/234,276
• Time in Patent Office: 1,225 Days
• Field of Search: 714/712,4,38 710/105 703/13,21 702/108,122,182 709/224,200,223,238 370/231,251,254
• US Class Current: 714/712
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5029 Service quality level-based billing, e.g. dependent on measured service level customer is charged more or less
• CPC Class Codes: H04L 41/5032 Generating service level reports
• CPC Class Codes: H04L 43/50 Testing arrangements

Abstract

Methods, systems and computer program products are provided which test network performance by defining test schedules including test protocols to be implemented and when the protocols should be executed for a plurality of defined connections on a network. A connection may be defined between two endpoint nodes on the network. At times specified in the test schedule, the endpoint node pair executes the test protocol and measures the performance of the network connection between the two nodes without requiring any involvement of application software which may or may not be installed on the computer hardware supporting the endpoint node. The test protocol may define the type of network layer protocol to utilize (for example, TCP), and the test script or scripts to be communicated using the appropriate stack on the computer hardware supporting the endpoint node. The schedule may be provided with an expiration date and a console node is provided for distribution of test schedules, monitoring of availability of endpoint nodes and receipt of measured performance measurements for reporting to a network manager. In further aspects of the present invention, auto-thresholding and coordination of interrelated but asynchronous tasks executing at the console node are provided.

Claims

1. A method for testing the performance of a communications network including a plurality of endpoint nodes comprising the steps of:

adding a console node residing on a computer to the network;

defining a connection associated with at least one of the endpoint nodes;

defining a test schedule for the connection;

communicating the test schedule to the associated endpoint node from the console node;

receiving network performance test results from the endpoint node a the console node based on the test schedules;

wherein the step of defining a connection comprises the step of defining an endpoint node specific test protocol between the at least one endpoint node and an associated second endpoint node to simulate communications traffic therebetween to be tested; and

wherein the step of defining a connection comprises the step of defining at least one performance threshold for the connection.

2. A method according to claim 1, wherein the step of defining a connection comprises the step of defining an action to be taken responsive to a detected violation of the at least one performance threshold.

3. A method according to claim 1 wherein the step of defining a connection comprises the step of defining a plurality of connections associated with the plurality of endpoint nodes wherein each of the plurality of connections is associated with a pair of the plurality of endpoint nodes and defines an endpoint node specific test protocol between the pair of the plurality of endpoint nodes to simulate communications traffic therebetween to be tested.

4. A method according to claim 1 further comprising the following steps that are performed by one of the at least one endpoint node and the associated second endpoint node:

receiving the test schedule;

executing the endpoint node specific test protocol over the network between the a least one endpoint node and the associated second endpoint node at a start time for initiating execution specified in the test schedule;

monitoring the performance of the network while the endpoint node specific test protocol is executed to obtain the network performance test results; and

communicating the network performance test results to the console node over the network.

5. A method for testing the performance of a communications network including a plurality of endpoint nodes comprising the steps of:

adding a console node residing on a computer to the network;

defining a connection associated with at least one of the endpoint nodes;

defining a test schedule for the connection;

communicating the test schedule to the associated endpoint node from the console node;

receiving network performance test results from the endpoint node at the console node based on the test schedules;

wherein the step of defining a connection comprises the step of defining an endpoint node specific test protocol between the at least one endpoint node and an associated second endpoint node to simulate communications traffic therebetween to be tested;

wherein the step of defining, a connection comprises the step of defining a plurality of connections associated with the plurality of endpoint nodes wherein each of the plurality of connections is associated with a pair of the plurality of endpoint nodes and defines an endpoint node specific test protocol between the pair of the plurality of endpoint nodes to simulate communications traffic therebetween to be tested; and

wherein the step of defining a test schedule comprises the step of defining a test schedule for each of tie plurality of connections, each test schedule including a start time for initiating execution of the associated endpoint node specific test protocol and a repeat schedule for re-initiating execution of the associated endpoint node specific test protocol.

6. A method according to claim 5 wherein the step of defining a test schedule comprises the step of defining a test schedule, the test schedule including an expiration time for terminating re-initiating execution of the associated endpoint node specific test protocol.

7. A method according to claim 5 wherein the step of communicating the test schedule comprises the step of transmitting each of the test schedules, including transmitting the endpoint node specific test protocols associated with each of the test schedules, to at least one of the associated pair of the plurality of endpoint nodes for each test schedule.

8. A method according to claim 7 wherein the step of receiving comprises the step of receiving over the network, at the console node, network performance results from at least one of the associated pair of the plurality of endpoint nodes based on measurements from execution of the associated endpoint node specific test protocol.

9. A method according to claim 8 wherein the step of receiving comprises the step of receiving network performance results from at least one of the associated pair of the plurality of endpoint nodes at time intervals based on the associated test schedule of the connection associated with the at least one of the associated pair of the plurality of endpoint nodes.

10. A method according to claim 9 wherein the step of communicating the test schedule is followed by the step of re-connecting with the at least one of the associated pair of the plurality of endpoint nodes if network performance results are not received from the at least one of the associated pair of the plurality of endpoint nodes within a criteria time interval after expiration of one of the time intervals at which results were expected.

11. A method according to claim 10 wherein the step of re-connecting comprises the step of indicating an error if unable to re-connect with the at least one of the associated pair of the plurality of endpoint nodes after a selected number of attempts to re-connect.

12. A method for testing the performance of a communications network including a plurality of endpoint nodes comprising the steps of:

adding a console nod residing on a computer to the network;

defining a connection associated with at least one of the endpoint nodes;

defining a test schedule for the connection;

communicating the test schedule to the associated endpoint node from the console node;

receiving network performance test results from the endpoint node at the console

node based on the test schedules;

wherein the step of defining a connection comprises the step of defining an endpoint node specific test protocol between the at least one endpoint node and an associated second endpoint node to simulate communications traffic therebetween to be tested;

further comprising the following steps that are performed by one of the at least one endpoint node and the associated second endpoint node;

receiving the test schedule;

executing tie endpoint node specific test protocol over the network between the at least one endpoint node and the associated second endpoint node a start time for initiating execution specified in the test schedule;

monitoring the performance of the network while the endpoint node specific test protocol is executed to obtain the new work performance test results; and

communicating the network performance test results to the console node over the network; and

repeating the executing, monitoring and communicating steps at a repeat time for re-initiating execution specified in the test schedule.

13. A method for testing the performance of a communications network including a plurality of endpoint nodes comprising the steps of:

adding a console node residing on a computer to the network;

defining a connection associated with at least one of the endpoint nodes;

defining a test schedule for the connection;

communicating the test schedule to the associated endpoint node from the console node;

receiving network performance test results from the endpoint node at the console node based on the test schedules;

wherein the step of defining a connection comprises the step of defining an endpoint node specific test protocol between the at least one endpoint node and an associated second endpoint node to simulate communications traffic therebetween to be tested;

further comprising the following steps that are performed by or of the at least one endpoint node and the associated second endpoint node;

receiving the test schedule;

executing the endpoint node specific test protocol over the network between the at least one endpoint node and the associated second endpoint node at a star time for initiating execution specified in the test schedule;

monitoring the performance of the network while the endpoint node specific test protocol is executed to obtain the network performance test results;

communicating the network performance test results to the console node over the network; and

wherein the step of communicating the network performance test results is initiated at a selected time following completion of the monitoring step.

14. A method according to claim 13 wherein the step of communicating the test schedule is followed by the step of re-connecting with the one of the at least one endpoint node and the associated second endpoint node if the network. performance results are not received at the console node at a criteria time based on the selected time and the start time.

15. A method according to claim 14 wherein the step of re-connecting comprises the step of indicating an error if unable to re-connect with the one of the at least one endpoint node and the associated second endpoint node after a selected number of attempts to re-connect.

16. A method for testing the performance of a communications network including a plurality of endpoint nodes comprising the steps of:

adding a console node residing on a computer to the network;

defining a connection associated with at least one of the endpoint nodes;

defining a test schedule for the connection;

communicating the test schedule to the associated endpoint node from the console node;

receiving network performance test results from the endpoint node at the console node based on the test schedules; and

wherein the test schedule includes an associated test protocol and wherein the step of defining a test schedule comprises the step of defining a test schedule including a start time for initiating execution of the associated test protocol and a repeat schedule for re-initiating execution of the associated test protocol.

17. A method according to claim 16 wherein the step of defining a test schedule comprises the step of defining a test schedule including an expiration time for terminating re-initiating execution of the associated test protocol.

18. A method according to claim 16 wherein the step of communicating the test schedule comprises the step of transmitting the test schedule to the at least one of the endpoint nodes over the network.

19. A method according to claim 18 wherein the step of receiving comprises the step of receiving over the network, at the console node, network performance results from the at least one of the endpoint nodes based on measurements from execution of the associated test protocol.

20. A method according to claim 19 wherein the step of receiving comprises the step of receiving network performance results from the at least one of the endpoint nodes at time intervals based on the test schedule.

21. A method according to claim 20 wherein the step of communicating the test schedule is followed by the step of re-connecting with the at least one of the endpoint nodes if network performance results are not received from the at least one of the endpoint nodes within a criteria time interval after expiration of a time interval at which network performance results were expected.

22. A method according to claim 21 wherein the step of re-connecting comprises the step of indicating an error if unable to re-connect with the at least one of the endpoint nodes after a selected number of attempts to re-connect.

23. A method for testing the performance of a communications network including an endpoint node comprising the steps of:

adding a console node residing on a computer to the network;

wherein the following steps are performed by the console node;

defining a connection associated with the endpoint node, the connection having an associated endpoint node specific test protocol between the endpoint node and an associated second endpoint node to simulate communications traffic therebetween to be tested and an associated performance threshold;

defining a test schedule including a start time for initiating execution of the associated endpoint node specific test protocol and a repeat schedule for re-initiating execution of the associated endpoint node specific test protocol for the connection;

communicating the test schedule to at least one of the endpoint node and the second endpoint node from the console node over the network;

receiving an indication of a violation of the performance threshold if the threshold has been violated; and

receiving network performance test results over the network from the endpoint node at the console node based on the test schedule; and

wherein the following steps are performed by at least one of the endpoint node and the second endpoint node;

monitoring the performance of the network while the endpoint node specific test protocol is executed over the network to obtain network performance test results;

determining if the performance threshold has been violated;

reporting an indication of a violation of the performance threshold to the console if the threshold has been violated; and

reporting network performance results over the network to the console node based on the test schedules.

24. A method according to claim 23 wherein the performance threshold is an auto-threshold wherein the performance threshold is determined based on determinations of a selected performance characteristic from obtained network performance test results.

25. A method according to claim 24 wherein the auto-threshold provides a first detection criteria establishing the performance threshold for the connection if the connection has a first standard deviation for the selected performance characteristic and a second detection criteria establishing the performance threshold for the connection if the connection has a second standard deviation for the selected performance characteristic, the first standard deviation exceeding a selected standard deviation and the second standard deviation being less than the selected standard deviation.

26. A system for testing the performance of a communications network comprising:

a console node residing on a computer connected to the network;

an endpoint node residing on a computer connected to the network;

wherein the console node comprises;

means for defining a connection associated with the endpoint node, the connection having an associated endpoint node specific test protocol between the endpoint node and an associated second endpoint node to simulate communications traffic therebetween to be tested and an associated performance threshold;

means for defining a test schedule including a start time for initiating execution of the associated endpoint node specific test protocol and a repeat schedule for re-initiating execution of the associated endpoint node specific test protocol for the connection;

means for communicating the test schedule to at least one of the endpoint node and the second endpoint node from the console node over the network;

means for receiving an indication of a violation of the performance threshold if the threshold has been violated; and

means for receiving network performance test results over the network from the endpoint node at the console node based on the test schedule; and

wherein at least one of the endpoint node and the second endpoint node comprise;

means for monitoring the performance of the network while the endpoint node specific test protocol is executed over the network to obtain network performance test results;

means for determining if the performance threshold has been violated;

means for reporting an indication of a violation of the performance threshold to the console node if the threshold has been violated; and

means for reporting network performance results over the network to the console node based on the test schedules.

27. A computer program product for testing the performance of a communications network, comprising:

a computer readable storage medium having computer readable program code means embodied in said medium, said computer-readable program code means comprising;

computer-readable program code means for defining a connection associated with an endpoint node, the connection having an associated endpoint node specific test protocol between the endpoint node and an associated second endpoint node to simulate communications traffic therebetween to be tested and an associated performance threshold;

computer-readable program code means for defining a test schedule including a start time for initiating execution of the associated endpoint node specific test protocol and a repeat schedule for re-initiating execution of the associated endpoint node specific test protocol for the connection;

computer-readable program code means for communicating the test schedule to at least one of the endpoint node and the second endpoint node from the console node over the network;

computer-readable program code means for receiving an indication of a violation of the performance threshold if the threshold has been violated; and

computer-readable program code means for receiving network performance test results over the network from the endpoint node at the console node based on the test schedule.

28. A computer program product for testing the performance of a communications network, comprising:

a computer readable storage medium having computer readable program code means embodied in said medium, said computer-readable program code means comprising;

computer-readable program code means for receiving a test schedule from a console node, the test schedule including an identification of an associated second endpoint node defining a connection and a start time for initiating execution over the network of an associated endpoint node specific test protocol and a repeat schedule for re-initiating execution of the associated endpoint node specific test protocol for the connection;

computer-readable program code means for monitoring the performance of the network while the endpoint node specific test protocol is executed over the network to obtain network performance test results;

computer-readable program code means for determining if the performance threshold has been violated;

computer-readable program code means for reporting an indication of a violation of the performance threshold to the console node if the threshold has been violated; and

computer-readable program code means for reporting network performance results over the network to the console node based on the test schedules.

29. A console system for testing the performance of a communications network including a plurality of endpoint nodes, the system comprising:

means for defining a connection associated with at least one of the endpoint nodes;

means for defining a test schedule for the connection;

means for communicating the test schedule to the at least one of the endpoint nodes over the network;

means for receiving network performance test results over the network from the at least one of the endpoint node based on the test schedules;

wherein the means for defining a connection comprises means for defining an endpoint node specific test protocol between the at least one endpoint node and an associated second endpoint node to simulate communications traffic therebetween to be tested; and

wherein the means for defining a connection comprises means for defining at least one performance threshold for the connection.

30. A console system for testing the performance of a communications network including a plurality of endpoint nodes, the system comprising:

means for defining a connection associated with at least one of the endpoint nodes;

means for defining a test schedule for the connection;

means for communicating the test schedule to at the at least one endpoint node over the network;

means for receiving network performance test results over the network from the at least one endpoint node based on the test schedules;

wherein the means for defining a connection comprises means for defining an endpoint node specific test protocol between the at least one endpoint node and an associated second endpoint node to simulate communications traffic therebetween to be tested;

wherein at least one of the at least one endpoint node and the associated second endpoint node further comprise;

means for receiving the test schedule;

means for executing the endpoint node specific test protocol over the network between the at least one endpoint node and the associated second endpoint node at a start time for initiating execution specified in the test schedule;

means for monitoring the performance of the network while the endpoint node specific test protocol is executed to obtain the network performance test results;

means for communicating the network performance test results to the console system over the network; and

further comprising means for repeating initiation of operations of the executing, monitoring and communicating means at a repeat time for re-initiating execution specified in the test schedule.

31. A console system for testing the performance of a communications network including a plurality of endpoint nodes, the system comprising:

means for defining a connection associated with at least one of the endpoint nodes;

means for defining a test schedule for the connection;

means for communicating the test schedule to at the at least one endpoint node over the network;

means for receiving network performance test results over the network from the at least one endpoint node based on the test schedules;

wherein the means for defining a connection comprises means for defining an endpoint node specific test protocol between the at least one endpoint node and an associated second endpoint node to simulate communications traffic therebetween to be tested;

wherein at least one of the at least one endpoint node and the associated second endpoint node further comprise;

means for receiving the test schedule;

means for executing the endpoint node specific test protocol over the network between the at least one endpoint node and the associated second endpoint node at a start time for initiating execution specified in the test schedule;

means for monitoring the performance of the network while the endpoint node specific test protocol is executed to obtain the network performance test results;

means for communicating the network performance test results to the console system over the network; and

wherein the means for communicating the network performance test results further comprises means for initiating communication of the network performance test results at a selected time after monitoring of the performance of the network is completed.

32. A console system for testing the performance of a communications network including a plurality of endpoint nodes, the system comprising:

means for defining a connection associated with at least one of the endpoint nodes;

means for defining a test schedule for the connection;

means for communicating the test schedule to at the at least one endpoint node over the network;

means for receiving network performance test results over the network from the at least one endpoint node based on the test schedules; and

wherein the test schedule includes an associated test protocol and wherein the means for defining a test schedule comprises means for defining a test schedule including a start time for initiating execution of the associated test protocol and a repeat schedule for initiating execution of the associated test protocol.

33. A computer program product testing the performance of a communications network including a plurality of endpoint notes, comprising:

a computer readable storage medium having computer readable program code means embodied in said medium, said computer-readable program code means comprising;

computer-readable program code means for defining a connection associated with at least one of the endpoint nodes;

computer-readable program code means for defining a test schedule for the connection;

computer-readable program code means for communicating the test schedule to the associated endpoint node over the network; and

computer-readable program code means for receiving network performance test results over the network from the endpoint node based on the test schedule;

wherein the computer-readable program code means for defining a connection comprises computer-readable program code means for defining an endpoint node specific test protocol between the at least one endpoint node and an associated second endpoint node stimulate communications traffic therebetween to be tested; and

wherein the computer-readable program code means for defining a connection comprises computer-readable program code means for defining at least one performance threshold for the connection.

34. A computer program product testing the performance of a communications network including a plurality of endpoint nodes, comprising:

a computer readable storage medium having computer readable program code means embodied in said medium, said computer readable program code means comprising;

computer readable program code means for defining a connection associated with at least one of the endpoint nodes;

computer readable program code means for defining a test schedule for the connection;

computer readable program code means for communicating the test schedule to the associated test schedule node over the network;

computer readable program code means for receiving network performance test results over the network from the endpoint node based on the test schedules; and

wherein the test schedule includes an associated test protocol and wherein the computer-readable program code means for defining a test schedule comprises computer-readable program code means for defining a test schedule including a start time for initiating execution of the associated test protocol and a repeat schedule for re-initiating execution of the associated test protocol.