System and method for statistical performance monitoring

• US 7,076,397 B2
• Filed: 10/10/2003
• Issued: 07/11/2006
• Est. Priority Date: 10/17/2002
• Status: Active Grant

• Current Assignee: BMC Software Incorporated (KKR & Co., Inc.)
• Original Assignee: BMC Software Incorporated (KKR & Co., Inc.)
• Inventors: Ding, Yiping, Newman, Kenneth W.
• Primary Examiner(s): Bui, Bryan
• Assistant Examiner(s): Bhat, Aditya S.
• Application Number:
  US10/684,132
• Publication Number:
  US 20040133395A1
• Time in Patent Office: 1,005 Days
• Field of Search: 702/146, 702/182, 702/183, 700/32, 700/108, 700/174, 714/1, 714/4, 714/74, 714/722, 340/439, 709/224
• US Class Current: 702/182
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

A method using statistical parameters (e.g. mean, standard deviation, exceptional values) of performance monitoring metrics to substantially reduce the quantity of performance monitoring data collected and reported, make system performance monitoring scalable and enhance the readability of the system performance display. The number of metrics monitored may be reduced by monitoring only one of any two metrics that are closely correlated.

Claims

1. A method for reducing the amount of data of system metrics collected or reported from agent nodes to a system performance monitor for system performance monitoring and analysis, the method comprising the steps of:

obtaining a sampled value of a first system metric;

reporting the sampled value of the first system metric if the sampled value is not between a first parameter and a second parameter, wherein the first parameter and the second parameter are any real numbers;

not reporting the sampled value if the sampled value is between the first and second parameters;

calculating a weighted running average, wherein

{overscore (d)}_n(w)=d_nw+{overscore (d)}<sub>n−

1</sub>(1−

w),

{overscore (d)}_n and {overscore (d)}<sub>n−

1</sub> are the weighted running average after n'"'"'th or (n−

1)'"'"'th sampling,

w is the weighing factor for the sampling,

S_n=S<sub>n−

1</sub>+(n−

1)(d_n−

{overscore (d)}<sub>n−

1</sub>)²/n,

σ

_n²=S_n/n,

S_n and S<sub>n−

1</sub> are the sum of the differences squared,

σ

_n is the standard deviation,

calculating the first parameter to be ({overscore (d)}_n−

aσ

_n); and

calculating the second parameter to be ({overscore (d)}_n+bσ

_n), wherein a and b are two constant real numbers.

2. The method in claim 1, wherein the first parameter and the second parameter are derived from sampled values of the first system metric.

3. The method in claim 1, wherein the first parameter and the second parameter are derived from at least one statistical parameter of the sampled values of the first system metric.

4. The method in claim 3, wherein the at least one statistical parameter of the sampled values of the first system metric includes the first moment of the sampled values.

5. The method in claim 4, wherein the at least one statistical parameters of the sampled values of the first system metric further includes the second moment of the sampled values.

6. The method in claim 1, further comprising assuming the sampled value of the first system metric that is not reported with an average, wherein the average is an average of previously sampled data of the first system metric.

7. The method in claim 6, wherein the avenge is a running average.

8. The method in claim 1, further comprising assuming the sampled value of the first system metric that is not reported with an average, wherein the first parameter is zero and the second parameter is a positive number.

9. The method in claim 1, wherein a and b are any real numbers between 0.5 and 3.1.

10. The method in claim 9, wherein a and b are 1.

11. The method in claim 1, wherein continuing sampling is repeated for N times, wherein N is an integer.

12. The method in claim 11, wherein w is between 1/N and 2/N.

13. The method in claim 11, wherein N is determined by a confidence interval cl and a tolerable variance error e_v, wherein

$e_v=\frac{100}{}$

⁢

f ⁢

⁢

( cl ) 2 N ,

and wherein f(cl) is the (1+cl/100)/2-quantile of the unit normal distribution.

14. The method in claim 1, further comprising:

reporting the weighed running average {overscore (d)}_iN, wherein iN is a multiple of N, wherein i is an integer; and

reporting {overscore (d)}_n and replacing {overscore (d)}_iN with {overscore (d)}_n when the |{overscore (d)}_n−

{overscore (d)}_iN| is greater than dd, wherein dd is a real number.

15. The method in claim 14, wherein dd is σ

_n.

16. The method in claim 1, wherein w=c/n, wherein c is a real number;

and wherein n is the n'"'"'th sampling.

17. The method in claim 16, wherein c is between 0.5 and 2.

18. The method in claim 1, further comprising:

obtaining a sampled value of a second system metric;

calculating the correlation coefficient cc between the sampled value of the first system metric and the second system metric after M sampling;

stopping sampling and stopping reporting the sampled value of the second system metric if |cc| is not less than a threshold; and

continuing sampling and reporting the sampled value of the second system metric if |cc| is less than a threshold, wherein |cc| is the absolute value of correlation coefficient cc.

19. The method in claim 1, further comprising:

receiving, at the system performance monitor, the reported sampled value of the first system metric; and

assuming, at the system performance monitor, the sampled value of the first system metric as an average for the sampled value not reported.

20. The method in claim 19, further comprising displaying the received and assumed values of the first system metric.

21. A computer system module for system performance monitoring, reporting and analysis, the module comprising:

a controller module operative to control the system performance monitoring; and

a sampling module coupled to the controller module, operative to

sample at least a first system metric, and

obtain a sampled value of the first system metric,

wherein each sampled value of the first system metric is reported if the sampled value is not between a first parameter and a second parameter, and not reported if the sampled value is between the first and second parameters,

wherein the first parameter and the second parameter are any real numbers, and

wherein the controller module is operative to

calculate a weighted running avenge, wherein

{overscore (d)}_n(w)=d_nw+{overscore (d)}<sub>n−

1</sub>(1−

w),

{overscore (d)}_n and {overscore (d)}<sub>n−

1</sub> are the weighted running average after n'"'"'th or (n−

1)'"'"'th sampling,

w is the weighing factor for the sampling,

S_n=S<sub>n−

1</sub>+(n−

1)(d_n−

{overscore (d)}<sub>n−

1</sub>)²/n,

σ

_n²=S_n/n,

S_n and S<sub>n−

1</sub> are the sum of the differences squared, and

σ

_n is the standard deviation; and

calculate the first parameter to be ({overscore (d)}_n−

aσ

_n) and the second parameter to be ({overscore (d)}_n+bσ

_n), wherein a and b are two constant real numbers.

22. The computer system module as in claim 21, wherein the first parameter and the second parameter are derived from sampled values of the first system metric.

23. The computer system module as in claim 21, wherein the first parameter and the second parameter are derived from at least one statistical parameter of the sampled values of the first system metric.

24. The computer system module as in claim 23, wherein the at least one statistical parameter of the sampled values of the first system metric includes the first moment of the sampled values.

25. The computer system module as in claim 24, wherein the at least one statistical parameters of the sampled values of the first system metric further includes the second moment of the sampled values.

26. The computer system module as in claim 21, wherein the controller module is operative to calculate an average, wherein the average is an average of previously sampled data of the first system metric.

27. The computer system module as in claim 21, wherein the controller module is operative to calculate an average, wherein the avenge is a running average.

28. The computer system module as in claim 21, wherein the first parameter is zero and the second parameter is a positive number.

29. The computer system module in claim 21, wherein the controller module is operative to stop sampling after N times, wherein N is an integer.

30. The computer system module in claim 29, wherein N is determined by a confidence interval cl and a tolerable variance error e_v, wherein

$e_v=\frac{100}{}$

⁢

f ⁢

⁢

( cl ) 2 N ,

and wherein f(cl) is the (1+cl/100)/2-quantile of the unit normal distribution.

31. The computer system module in claim 29, wherein the controller module is operative to

report the weighted running average {overscore (d)}_iN, wherein iN is a multiple of N, wherein i is an integer;

and

report {overscore (d)}_n when the |{overscore (d)}_n−

{overscore (d)}_iN| is greater than dd, wherein dd is a real number.

32. The computer system module in claim 31, wherein dd is σ

_n.

33. The computer system module in claim 21, wherein the w=c/n, wherein c is a real number, and wherein n is the n'"'"'th sampling.

34. The computer system module in claim 21, wherein the controller module is operative to

sample a second system metric;

obtain a sampled value of the second system metric;

calculate the correlation coefficient cc between the sampled value of the first system metric and the second system metric after M sampling;

stop sampling and not report the sampled value of the second system metric if |cc| is not less than a threshold; and

continue sampling and report the sampled value of the second system metric if |cc| is less than a threshold, wherein |cc| is the absolute value of correlation coefficient cc.

35. The computer system module in claim 34, wherein the threshold is 0.7.

36. The computer system module in claim 35, further comprising a display module operative to display the received and assumed values of the first system metric.

37. The computer system module in claim 21, further comprising a monitoring module operative to receive the reported sampled value of the first system metric and to assume the sampled value of the first system metric as an average for the sampled value not reported.

38. A computer network system comprising:

a plurality of network nodes having

a CPU;

a memory module coupled to CPU, operative to contain computer executable programs; and

a network interface operative to interconnect different nodes of the network,

wherein one computer executable program is loaded in the memory module in one node, wherein the computer executable program is operative to perform a method for reducing the amount of data of system metrics collected or reported from agent nodes to a system performance monitor for system performance monitoring and analysis, the method comprising the steps of;

obtaining a sampled value of a first system metric;

reporting the sampled value of the first system metric if the sampled value is not between a first parameter and a second parameter, wherein the first parameter and the second parameter are any real numbers;

not reporting the sampled value if the sampled value is between the first and second parameters;

calculating a weighted running average, wherein

{overscore (d)}_n(w)=d_nw+{overscore (d)}<sub>n−

1</sub>(1−

w),

{overscore (d)}_n and {overscore (d)}<sub>n−

1</sub> are the weighted running average after n'"'"'th or (n−

1)'"'"'th sampling,

w is the weighing factor for the sampling,

S_n=S<sub>n−

1</sub>+(n−

1)(d_n−

{overscore (d)}<sub>n−

1</sub>)²/n,

σ

_n²=S_n/n,

S_n and S<sub>n−

1</sub> are the sum of the differences squared,

σ

_n is the standard deviation,

calculating the first parameter to be ({overscore (d)}_n−

aσ

_n); and

calculating the second parameter to be ({overscore (d)}_n+bσ

_n), wherein a and b are two constant real numbers.

39. The system in claim 38, wherein a and b are any real numbers between 0.5 and 3.1.

40. The system in claim 39, wherein a and b are 1.

41. The system in claim 38, wherein continuing sampling is repeated for N times, wherein N is an integer.

42. The system in claim 41, wherein w is between 1/N and 2/N.

43. The system in claim 41, wherein N is determined by a confidence interval cl and a tolerable variance error e_v, wherein

$e_v=\frac{100}{}$

⁢

f ⁢

⁢

( cl ) 2 N ,

and wherein f(cl) is the (1+cl/100)/2-quantile of the unit normal distribution.

44. The system in claim 38, wherein the method further comprises:

reporting the weighted running average {overscore (d)}_iN, wherein iN is a multiple of N, wherein i is an integer; and

reporting {overscore (d)}_n and replacing {overscore (d)}_iN with {overscore (d)}_n when the |{overscore (d)}_n−

{overscore (d)}_iN| is greater than dd, wherein dd is a real number.

45. The system in claim 44, wherein dd is σ

_n.

46. The system in claim 38, wherein w=c/n, wherein c is a real number, and wherein n is the n'"'"'th sampling.

47. The system in claim 46, wherein c is between 0.5 and 2.

48. A machine readable medium comprising a machine executable program, wherein the machine executable program is operative to perform a method for reducing the amount of data of system metrics collected or reported from agent nodes to a system performance monitor for system performance monitoring and analysis, the method comprising the steps of:

obtaining a sampled value of a first system metric;

reporting the sampled value of the first system metric if the sampled value is not between a first parameter and a second parameter, wherein the first parameter and the second parameter are any real numbers;

not reporting the sampled value if the sampled value is between the first and second parameters;

calculating a weighted running average, wherein

{overscore (d)}_n(w)=d_nw+{overscore (d)}<sub>n−

1</sub>(1−

w),

{overscore (d)}_n and {overscore (d)}<sub>n−

1</sub> are the weighted running average after n'"'"'th or (n−

1)'"'"'th sampling,

w is the weighing factor for the sampling,

S_n=S<sub>n−

1</sub>+(n−

1)(d_n−

{overscore (d)}<sub>n−

1</sub>)²/n,

σ

_n²=S_n/n,

S_n and S<sub>n−

1</sub> are the sum of the differences squared,

σ

_n is the standard deviation,

calculating the first parameter to be ({overscore (d)}_n−

aσ

_n); and

calculating the second parameter to be ({overscore (d)}_n+bσ

_n), wherein a and b are two constant real numbers.

49. The system in claim 48, wherein a and b are any real numbers between 0.5 and 3.1.

50. The system in claim 49, wherein a and b are 1.

51. The system in claim 48, wherein continuing sampling is repeated for N times, wherein N is an integer.

52. The system in claim 51, wherein w is between 1/N and 2/N.

53. The system in claim 51, wherein N is determined by a confidence interval cl and a tolerable variance error e_v, wherein

$e_v=\frac{100}{}$

⁢

f ⁢

⁢

( cl ) 2 N ,

and wherein f(cl) is the (1+cl/100)/2-quantile of the unit normal distribution.

54. The system in claim 48, wherein the method further comprises:

reporting the weighted running average {overscore (d)}_iN, wherein iN is a multiple of N, wherein i is an integer; and

reporting {overscore (d)}_n and replacing {overscore (d)}_iN with {overscore (d)}_n when the |{overscore (d)}_n−

{overscore (d)}_iN| is greater than dd, wherein dd is a real number.

55. The system in claim 54, wherein dd is σ

_n.

56. The system in claim 48, wherein w=c/n, wherein c is a real number, and wherein n is the n'"'"'th sampling.

57. The system in claim 56, wherein c is between 0.5 and 2.