Confidence Intervals for Key Performance Indicators in Communication Networks
First Claim
1. A method, comprising:
- performing, using one or more computer systems,identifying vectors representing network events observed by a network monitoring system, each vector including;
a dimension, an indication of a sampling ratio with which a respective network event was observed, and a value associated with the dimension;
calculating a Key Performance Indicator (KPI) corresponding to the observed network events for the dimension based, at least in part, upon the values; and
calculating a confidence associated with the KPI, based, at least in part, upon the sampling ratios.
5 Assignments
0 Petitions
Accused Products
Abstract
Systems and methods for calculating and presenting confidence interval(s) for key performance indicator(s) (KPIs) are described. For example, in some embodiments, a method may include identifying vectors representing network events observed by a network monitoring system, each vector including: a dimension, an indication of a sampling ratio with which a respective event was observed, and a value associated with the dimension. The method may also include calculating a KPI corresponding to the observed events for the dimension based, at least in part, upon the values. The method may further include calculating a confidence associated with the KPI, based, at least in part, upon the sampling ratios. In some cases, events may be observed with different sampling ratios. Additionally or alternatively, sampling ratios may include adaptive sampling ratios controlled by the network monitoring system in response to network or resource loading (e.g., subject varying over time), whitelist differentiated sampling ratios, etc.
-
Citations
20 Claims
-
1. A method, comprising:
performing, using one or more computer systems, identifying vectors representing network events observed by a network monitoring system, each vector including;
a dimension, an indication of a sampling ratio with which a respective network event was observed, and a value associated with the dimension;calculating a Key Performance Indicator (KPI) corresponding to the observed network events for the dimension based, at least in part, upon the values; and calculating a confidence associated with the KPI, based, at least in part, upon the sampling ratios. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
13. A network monitoring system, comprising:
-
a processor; and a memory coupled to the processor, the memory configured to store program instructions executable by the processor to cause the network monitoring system to; identify a plurality of vectors representing sampled network events, each vector including a plurality of dimensions, each of the plurality of dimensions associated with an indication of a sampling ratio with which the respective dimension was sampled, and a value associated with each respective dimension, the sampling ratios being subject to modification over time as a function of network or resource loading; calculate a Key Performance Indicator (KPI) associated with a selected one of the plurality of dimensions based, at least in part, upon an operation performed with respect to respective values, the operation including at least one of a;
mean, average, minimum, or maximum;estimate a number of network events (KN) that would have been observed in the absence of sampling; determine a number of sampled network events (Kn); calculate a standard deviation (Kσ
) of the values; andcalculate a confidence value associated with the KPI, based, at least in part, upon KN, Kn, and Kσ
. - View Dependent Claims (14, 15, 16, 17)
-
-
18. A tangible electronic storage medium having program instructions stored thereon that, upon execution by a processor within a computer system, cause the computer system to:
-
determine a Key Performance Indicator (KPI) based, at least in part, upon values associated with a dimension in a plurality of vectors representing detected network events, the detected network events having been adaptively sampled by a network monitoring system using a sampling ratio; calculate a confidence value associated with the KPI, based, at least in part, upon the sampling ratio; and calculate a second sampling ratio with which subsequent network events are detected based, at least in part, upon a user'"'"'s adjustment to the confidence value. - View Dependent Claims (19, 20)
-
Specification