Methods, system and computer program products for dynamic filtering of network performance test results

• US 6,901,442 B1
• Filed: 01/07/2000
• Issued: 05/31/2005
• Est. Priority Date: 01/07/2000
• Status: Expired due to Fees

• Current Assignee: NetIQ Corporation (Open Text Corporation)
• Original Assignee: NetIQ Corporation (Open Text Corporation)
• Inventors: Quan, David V. H., Southard, Scott Alan, Lingle, Roger Allen, Bellinghausen, Joan Marie, Borger, Dana Scott, Christensen, John C., Joyce, Steven Thomas, Schwaller, Peter James
• Primary Examiner(s): Dharia, Rupal
• Assistant Examiner(s): Nguyen, Quang
• Application Number:
  US09/479,568
• Time in Patent Office: 1,971 Days
• Field of Search: 709/223, 709/224, 709/226, 709/238
• US Class Current: 709/224
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 43/028 by filtering
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/067 using time frame reporting
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/0852 Delays
• CPC Class Codes: H04L 43/0888 Throughput
• CPC Class Codes: H04L 43/16 Threshold monitoring

Abstract

Dynamic filtering methods, systems and computer program products are provided for network performance test results which may apply network troubleshooting expertise and knowledge of network topology to select and display test results in a manner which may facilitate analysis of those results by IT staffs. In a further aspect of the present invention, a severity index is provided which may be generated based on exception events from a plurality of network performance measurements, for example, response time, throughput and availability, which measurements are generated from test results obtained from agents located on various devices on a computer network. The test results may be obtained from either passive application monitor agents or active network test agents. In another aspect of the present invention, the exception events may be detected based on automatically generated threshold criteria which may be provided user selectable sensitivity and may be based on a specified percentage criteria relative to baseline performance results. In a further aspect of the present invention, the test results may be stored using data buckets with the number of data buckets and/or the range of each of the data buckets being selected to provide a desired overall range and granularity for each of the plurality of network performance measurement types. The range (width) of some of the data buckets may be narrower than others to provide greater resolution in a region of interest.

Claims

1. A method for dynamically filtering network performance test results comprising the steps of:

displaying a performance index for each of one of a plurality of applications that execute on a network or a first plurality of devices coupled to the network and an identification of each of the one of a plurality of applications that execute on the network or a first plurality of devices coupled to the network;

filtering the network performance test results based on a first user selection of one of the identifications to provide network performance test results associated with the selected one of the one of a plurality of applications that execute on the network or a first plurality of devices coupled to the network as first filtered results;

obtaining performance indexes for the other of a plurality of applications that execute on the network or a first plurality of devices coupled to the network based on the first filtered results responsive to the first user selection; and

displaying the obtained performance indexes for the other of a plurality of applications that execute on the network or a first plurality of devices coupled to the network responsive to the first user selection.

2. A method according to claim 1 wherein the plurality of applications are application programs executing on the first plurality of devices and the network performance test results are based on passive monitoring of the application programs executing on the first plurality of devices.

3. A method according to claim 2 wherein the network performance test results are associated with a client-server communications network and wherein the first plurality of devices are client devices on the communications network.

4. A method according to claim 1 wherein the plurality of applications are application scripts and the network performance test results are based on active network testing of a communications network by the first plurality of devices using the application scripts.

5. A method according to claim 4 wherein the network performance test results are based on endpoint pair based active network testing using an endpoint specific test protocol between a designated first endpoint and an associated second endpoint and wherein the first plurality of devices are designated first endpoint devices.

6. A method according to claim 1 wherein the step of displaying the obtained performance indexes for the other of a plurality of applications or a first plurality of devices further comprises the step of displaying identifications of the other of a plurality of applications or a first plurality of devices and wherein the step of displaying the obtained performance indexes for the other of a plurality of applications or a first plurality of devices is followed by the steps of:

filtering the first filtered results based on a second user selection of one of the identifications of the other of a plurality of applications or a first plurality of devices to provide second filtered results;

obtaining performance indexes for a second plurality of devices coupled to the network based on the second filtered results responsive to the second user selection; and

displaying the obtained performance indexes for the second plurality of devices coupled to the network responsive to the second user selection.

7. A method according to claim 6 wherein the network performance test results are associated with a client-server communications network and wherein the first plurality of devices are client devices on the communications network and wherein the second plurality of devices are server devices on the communications network.

8. A method according to claim 6 wherein the network performance test results are associated with a client-server communications network and wherein the first plurality of devices are server devices on the communications network and wherein the second plurality of devices are client devices on the communications network.

9. A method according to claim 1 wherein the step of obtaining performance indexes further comprises the step of generating a severity index for each of the other of a plurality of applications or a first plurality of devices characterizing network performance test results based on a number of exception events for each of a plurality of types of network performance tests and wherein the step of displaying the obtained performance indexes comprises the step of displaying the generated severity indexes.

10. A method according to claim 9 wherein the plurality of types of network performance tests include at least one of throughput, response time, application availability and transaction rate.

11. A method according to claim 9 wherein the step of generating a severity index further comprises the step of identifying exception events for each of the plurality of types of network performance tests using threshold criteria for each of the plurality of types of network performance tests which are based on a specified percentage of baseline results for the respective types of network performance tests.

12. A method according to claim 11 wherein the step of identifying exception events further comprises the step obtaining the specified percentages from a user to provide threshold criteria having a user selected sensitivity.

13. A method according to claim 11 wherein the step of identifying exception events further comprises the following steps performed for each of the plurality of types of network performance tests:

designating a number of data buckets for the respective type of network performance test and an associated range for each of the data buckets, at least one of the number of data buckets or the associated range for each of the data buckets being selected based on a desired range for the specified percentages, the data buckets comprising the baseline results;

generating a network performance test data record based on a transaction that is performed over the network;

incrementing one of the designated data buckets having an associated range including the generated network performance test data record to record the network performance test data record;

determining ones of the data buckets satisfying the specified percentage for the respective type of network performance test based on numbers of records recorded in the data buckets; and

identifying an exception event if the generated network performance test data record corresponds to a data bucket not satisfying the specified percentage.

14. A method according to claim 13 wherein the specified percentage falls within the range of one of the data buckets and wherein the step of determining ones of the data buckets satisfying the specified percentage further comprises the step of designating a first portion of the one of the data buckets as satisfying the specified percentage and a second portion of the one of the data buckets as not satisfying the specified percentage based on the specified percentage and treating the data records within the one of the data buckets as being uniformly distributed.

15. A method according to claim 9 further comprising the steps of:

obtaining information related to the number of exception events for each of a plurality of types of network performance tests associated with a displayed one of the generated severity indexes responsive to a user selection of the displayed one of the generated severity indexes; and

displaying the obtained information responsive to the user selection of the displayed one of the generated severity indexes.

16. A method according to claim 1 further comprising the steps of:

displaying a trend associated with the performance index for each of one of a plurality of applications or a first plurality of devices;

obtaining trends associated with each of the obtained performance indexes for the other of a plurality of applications or a first plurality of devices responsive to the first user selection; and

displaying the obtained trends associated with each of the obtained performance indexes responsive to the first user selection.

17. A method according to claim 16 wherein the steps of displaying a performance index for each of one of a plurality of applications or a first plurality of devices and displaying a trend associated with the performance index for each of the one of a plurality of applications or a first plurality of devices are executed concurrently and wherein the steps of displaying the obtained performance indexes and displaying the obtained trends are executed concurrently and wherein the trends are displayed associated with their respective performance indexes.

18. A method according to claim 17 further comprising the steps of:

obtaining information related to performance trends for each of a plurality of types of network performance tests associated with a displayed one of the obtained trends responsive to a user selection of the displayed one of the obtained trends; and

displaying the obtained information responsive to the user selection of the displayed one of the obtained trends.

19. A method for identifying a network performance problem based on network performance test results comprising the steps of:

generating a performance index for each of one of a plurality of applications that execute on the network or a plurality of first devices coupled to the network and an identification of each of the one of a plurality of applications that execute on the network or a plurality of first devices coupled to the network based on the network performance test results;

selecting one of the one of a plurality of applications that execute on the network or a plurality of first devices coupled to the network associated with a worst one of the generated performance indexes to provide a first selection;

filtering the network performance test results based on the first selection to provide network performance test results associated with the selected one of the one of a plurality of applications that execute on the network or a first plurality of devices coupled to the network as first filtered results;

obtaining performance indexes for the other of a plurality of applications that execute on the network or a first plurality of devices coupled to the network based on the first filtered results responsive to the first selection;

selecting one of the obtained performance indexes associated with a worst one of the obtained performance indexes to provide a second selection;

obtaining information related to a number of exception events for each of a plurality of types of network performance tests, the exception events being associated with the selected one of the obtained performance indexes, responsive to the second selection;

selecting one of the plurality of types of network performance tests associated with a worst one of the number of exception events to provide a third selection; and

obtaining information related to the worst one of the number of exception events responsive to the third selection.

20. A method for recording network Performance measurements for a communications network comprising the steps of:

designating a number of data buckets for the network performance measurements and an associated range for each of the data buckets, at least one of the number of data buckets or the associated range for each of the data buckets being selected based on a user interest criterion;

obtaining one of the network performance measurements based on a communication transaction over the communications network;

incrementing one of the designated data buckets having an associated range including the obtained one of the network performance measurements to record the generated one of the network performance measurements; and

wherein designating a number of data buckets comprises;

determining a maximum and a minimum expected value for the network performance measurements;

determining a desired granularity for the network performance measurements; and

determining the number of data buckets based on the maximum and minimum expected values and the desired granularity.

21. A method according to claim 20 wherein the step of designating a number of data buckets further comprises the steps of:

identifying a first range of values between the maximum and the minimum expected value as a preferred range; and

associating a portion of the data buckets to the preferred range which exceeds a ratio of the preferred range to a difference between the maximum and the minimum expected value to provide an increased granularity in the preferred range.

22. A method according to claim 21 wherein the step of identifying a first range further comprises the step of identifying the first range to cover an expected range of performance threshold criteria for the network performance measurements.

23. A method for characterizing performance of a communications network comprising the step of generating a severity index associated with at least one of an application executing on the network or a device coupled to the network based on a number of exception events for each of a plurality of types of network performance tests, wherein the step of generating a severity index comprises the step of generating the severity index based on a function of a ratio of the severity points to a total transaction count over which the severity points were obtained.

24. A method according to claim 23 wherein the plurality of types of network performance tests include at least one of throughput, response time, application availability and transaction rate.

25. A method according to claim 24 wherein the step of generating a severity index further comprises the step of providing each of the plurality of types of network performance tests a selected weighting in the severity index.

26. A method according to claim 25 wherein the step of generating a severity index further comprises the step of pre-scaling each of the exception events based on a degree of deviation of the exception event wherein an exception based on a larger degree of deviation is provided a greater magnitude of severity points.

27. A method according to claim 26 wherein the step of generating a severity index comprises the step of generating the severity index based on a logarithmic function of the ratio of the severity points to the total transaction count over which the severity points were obtained.

28. A system for dynamically filtering network performance test results comprising:

a display that displays a performance index for each of one of a plurality of applications that execute on a network or a first plurality of devices coupled to the network and an identification of each of the one of a plurality of applications that execute on the network or a first plurality of devices coupled to the network;

a filter module that filters the network performance test results based on a first user selection of one of the identifications to provide network performance test results associated with the selected one of the one of a plurality of applications that execute on the network or a first plurality of devices coupled to the network as first filtered results;

a performance index generation module that obtains performance indexes for the other of a plurality of applications that execute on the network or a first plurality of devices coupled to the network based on the first filtered results responsive to the first user selection; and

wherein the display is configured to display the obtained performance indexes for the other of a plurality of applications that execute on the network or a first plurality of devices coupled to the network responsive to the first user selection.

29. A system for dynamically filtering network performance test results comprising:

means for displaying a performance index for each of one of a plurality of applications that execute on a network or a first plurality of devices coupled to the network and an identification of each of the one of a plurality of applications that execute on the network or a first plurality of devices coupled to the network;

means for filtering the network performance test results based on a first user selection of one of the identifications to provide network performance test results associated with the selected one of the one of a plurality of applications that execute on the network or a first plurality of devices coupled to the network as first filtered results;

means for obtaining performance indexes for the other of a plurality of applications that execute on the network or a first plurality of devices coupled to the network based on the first filtered results responsive to the first user selection; and

means for displaying the obtained performance indexes for the other of a plurality of applications that execute on the network or a first plurality of devices coupled to the network responsive to the first user selection.

30. A system for recording network performance measurements for a communications network comprising:

means for designating a number of data buckets for the network performance measurements and an associated range for each of the data buckets, at least one of the number of data buckets or the associated range for each of the data buckets being selected based on a user interest criterion;

means for obtaining one of the network performance measurements based on a communication transaction over the communications network;

means for incrementing one of the designated data buckets having an associated range including the obtained one of the network performance measurements to record the generated one of the network performance measurements; and

wherein the means for designating a number of data buckets comprises;

means for determining a maximum and a minimum expected value for the network performance measurements;

means for determining a desired granularity for the network performance measurements; and

means for determining the number of data buckets based on the maximum and minimum expected values and the desired granularity.

31. A system for characterizing performance of a communications network comprising:

means for detecting exception events for a plurality of types of network performance tests; and

means for generating a severity index associated with at least one of an application executing on the network or a device coupled to the network based on a number of exception events for each of a plurality of types of network performance tests, wherein the step of generating a severity index comprises the step of generating severity index based on a function of a ratio of the severity points to a total transaction count over which the severity points were obtained.

32. A computer program product for dynamically filtering network performance test results, the computer program product comprising:

a computer readable storage medium having computer readable program code embodied in said medium, said computer-readable program code comprising;

computer-readable program code that displays a performance index for each of one of a plurality of applications that execute on a network or a first plurality of devices coupled to the network and an identification of each of the one of a plurality of applications that execute on the network or a first plurality of devices coupled to the network;

computer-readable program code that filters the network performance test results based on a first user selection of one of the identifications to provide network performance test results associated with the selected one of the one of a plurality of applications or a first plurality of devices as first filtered results;

computer-readable program code that obtains performance indexes for the other of a plurality of applications or a first plurality of devices based on the first filtered results responsive to the first user selection; and

computer-readable program code that displays the obtained performance indexes for the other of a plurality of applications or a first plurality of devices responsive to the first user selection.

33. A computer program product for recording network performance measurements for a communications network, the computer program product comprising:

a computer-readable storage medium having computer readable program code embodied in said medium, said computer-readable program code comprising;

computer-readable program code that designates a number of data buckets for the network performance measurements and an associated range for each of the data buckets, at least one of the number of data buckets or the associated range for each of the data buckets being selected based on a user interest criterion;

computer-readable program code that obtains one of the network performance measurements based on a communication transaction over the communications network;

computer-readable program code that increments one of the designated data buckets having an associated range including the obtained one of the network performance measurements to record the generated one of the network performance measurements; and

wherein the computer-readable program code that designates a number of data buckets comprises;

computer-readable program code that determines a maximum and a minimum expected value for the network performance measurements;

computer-readable program code that determines a desired granularity for the network performance measurements; and

computer-readable program code that determines the number of data buckets based on the maximum and minimum expected values and the desired granularity.

34. A computer program product for characterizing performance of a communications network, the computer program product comprising:

a computer-readable storage medium having computer readable program code embodied in said medium, said computer-readable program code comprising;

computer-readable program code that detects exception events related to a communication transaction on a communications network for a plurality of types of network performance tests; and

computer-readable program code that generates a severity index associated with at least one of an application executing on the network or a device connected to the network based on a number of exception events for each of a plurality of types of network performance tests, wherein the step of generating a severity index comprises the step of generating severity index based on a function of a ratio of the severity points to a total transaction count over which the severity points were obtained.