Methods and computer program products for correlation analysis of network traffic in a network device

• US 10,404,556 B2
• Filed: 06/21/2013
• Issued: 09/03/2019
• Est. Priority Date: 06/22/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Reynolds, Patrick Alexander, Irwin, David William
• Primary Examiner(s): Cho, Un C
• Assistant Examiner(s): Ahmed, Atique
• Application Number:
  US13/923,473
• Publication Number:
  US 20130343213A1
• Time in Patent Office: 2,265 Days
• Field of Search: 370252, 370235, 370328, 709224, 455423, 455436, 375148
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters

Abstract

Provided are methods and computer program products for monitoring the contents of network traffic in a network device and performing correlation analysis of collected performance metrics to help identify reasons for network performance issues. Methods for correlation analysis include selecting scope and network metric types to include in the correlation analysis to generate an educated candidate set for correlation analysis. The correlation analysis methods result in a hypothesis set that assist the operator in identifying transactions and infrastructure problems resulting in network performance degradation.

Claims

1. A method of analyzing network traffic comprising:

collecting data related to network traffic within a computing network;

generating a plurality of metrics associated with the data, the plurality of metrics including network element metrics and node-to-node contexts;

selecting a primary metric from the plurality of metrics associated with the network element metrics and the node-to-node contexts;

identifying a candidate set corresponding to a portion of the plurality of metrics;

generating correlation coefficients between the primary metric and the portion of the plurality of metrics that are members of the candidate set;

generating a hypothesis set based on the correlation coefficients, wherein the hypothesis set includes the portion of the plurality of metrics that are members of the candidate set that are causally related to the primary metric;

identifying other metrics of the plurality of metrics with a high correlation to the primary metric, wherein a positive correlation suggests that a higher load on one network element or context is indicative of a causal link to increased utilization of another network element or resource and a negative correlation suggests that one process or transaction is indicative of a causal link to inhibiting another process or transaction from obtaining adequate resources;

adding the other metrics of the plurality of metrics with the high correlation to the primary metric to the hypothesis set; and

terminating a process associated with at least one metric of the plurality of metrics indicated in the hypothesis set;

wherein at least one of selecting the primary metric, generating the correlation coefficients, and generating the hypothesis set is performed using at least one processor.

2. The method of analyzing network traffic of claim 1, wherein identifying the candidate set corresponding to the portion of the plurality of metrics is based on a defined setting.

3. The method of analyzing network traffic of claim 2, wherein identifying the candidate set comprises selecting a scope of network elements to include in the candidate set.

4. The method of analyzing network traffic of claim 3, wherein the scope includes selecting network elements within N network hops of a network element associated with the primary metric, wherein N>

=1.

5. The method of analyzing network traffic of claim 3, wherein the scope includes selecting all network elements that are associated with the selected primary metric.

6. The method of analyzing network traffic of claim 2, wherein identifying the candidate set comprises selecting a network metric type that identifies a type of network metrics to include in the candidate set.

7. The method of analyzing network traffic of claim 6, wherein identifying the candidate set further comprises:

applying a filtering function to the plurality of metrics associated with network element metrics and/or node-to-node contexts.

8. The method of analyzing network traffic of claim 7, wherein applying the filter function further comprises:

receiving an input associated with the network metric type; and

identifying members of the candidate set from the plurality of metrics based on the received input.

9. The method of analyzing network traffic of claim 8, wherein receiving the input associated with the network metric type comprises:

receiving input from a user via a user input interface; and

generating the network metric type based on the input from the user.

10. The method of analyzing network traffic of claim 8, wherein receiving the input associated with the network metric type comprises:

reading a data file that includes configuration information; and

generating the network metric type based on the configuration information.

11. The method of analyzing network traffic of claim 6, wherein selecting the network metric type comprises selecting transactions as the network metric type to include in the candidate set.

12. The method of analyzing network traffic of claim 6, wherein selecting the network metric type comprises selecting virtual machines as the network metric type to include in the candidate set.

13. The method of analyzing network traffic of claim 6, wherein selecting the network metric type comprises selecting network infrastructure metrics as the network metric type to include in the candidate set.

14. The method of analyzing network traffic of claim 6, wherein selecting the network metric type comprises selecting links between nodes in the network as the network metric type to include in the candidate set.

15. The method of analyzing network traffic of claim 6, wherein selecting the network metric type comprises selecting processes running on nodes in the network as the network metric type to include in the candidate set.

16. The method of analyzing network traffic of claim 6, wherein selecting the network metric type comprises selecting server stacks as the network metric type to include in the candidate set.

17. The method of analyzing network traffic of claim 2, further comprising:

displaying, on a display, information that represents the primary metric; and

displaying in tabular form, on the display, for one or more members of the candidate set, respective information associated with a metric source, context type, network element identification, and/or corresponding correlation coefficient.

18. The method of analyzing network traffic of claim 1, wherein the correlation coefficient is calculated using Pearson'"'"'s correlation coefficient.

19. The method of analyzing network traffic of claim 1, wherein generating the hypothesis set comprises:

identifying the primary metric and the other metrics for which a respective correlation coefficient is greater than a first threshold or less than a second threshold; and

adding to the hypothesis set, the identified primary metric and the other metrics;

wherein the correlation coefficient being greater than the first threshold corresponds to a positive correlation coefficient and the correlation coefficient being less than the second threshold corresponds to a negative correlation coefficient.

20. The method of analyzing network traffic of claim 1, further comprising:

displaying, on a display, information representing the primary metric, at least one of the portion of the plurality of metrics, and the correlation coefficients corresponding to the at least one of the portion of the plurality of metrics.

21. The method of analyzing network traffic of claim 1, further comprising:

displaying, on a display, data graphs representative of the primary metric and at least one of the portion of the plurality of metrics.

22. The method of analyzing network traffic of claim 1, further comprising:

displaying, on a display, information associated with one or more members of the hypothesis set in a visually distinctive manner relative to other ones of the portion of the plurality of metrics.

23. The method of analyzing network traffic of claim 1, further comprising:

generating heat maps based on the correlation coefficients, wherein color-coded variations are used to distinguish varying levels of correlation between the primary metric and the portion of the plurality of metrics.

24. The method of analyzing network traffic of claim 1,

wherein the portion of the plurality of metrics includes all of the plurality of the metrics except the primary metric, and

wherein generating the correlation coefficients comprises generating the correlation coefficients between the primary metric and each of the portion of the plurality of the metrics.

25. The method of analyzing network traffic of claim 1,

wherein data corresponding to the primary metric corresponds to a first time and data corresponding to the portion of the plurality of the metrics corresponds to a second time that is different from the first time by a time interval;

wherein generating the hypothesis set comprises generating a temporally shifted hypothesis set.

26. The method of analyzing network traffic of claim 25, wherein the time interval is a multiple of a data collection interval corresponding to the plurality of metrics.

27. The method of analyzing network traffic of claim 25, further comprising:

receiving an increment and/or decrement input via a user interface; and

adjusting the time interval based on the received increment and/or decrement input.

28. The method of analyzing network traffic of claim 25, further comprising:

receiving a temporal shift value from a user interface,

wherein the time interval is adjusted based on the received temporal shift value.

29. A method of analyzing network traffic comprising:

collecting data related to network traffic within a computing network;

generating a plurality of metrics associated with the data, the plurality of metrics including network element metrics and node-to-node contexts;

selecting a primary metric from the plurality of metrics associated with the network element metrics and the node-to-node contexts;

generating correlation coefficients between the primary metric and a portion of a plurality of collected metrics associated with the network element metrics and the node-to-node contexts that are causally related to the primary metric; and

generating a hypothesis set based on the correlation coefficients, wherein the hypothesis set includes the portion of the plurality of metrics that are causally related to the primary metric;

identifying other metrics of the plurality of metrics with a high correlation to the primary metric, wherein the high correlation indicates that processes associated with the other metrics are consuming more resources than metrics of the plurality of metrics without the high correlation to the primary metric;

adding the other metrics of the plurality of metrics with the high correlation to the primary metric to the hypothesis set; and

terminating at least one process indicated by a metric of the plurality of metrics in the hypothesis set to thereby free resources for remaining metrics of the plurality of metrics; and

wherein at least one of generating the correlation coefficients and generating the hypothesis set is performed using at least one processor.

30. The method of analyzing network traffic of claim 29, wherein data corresponding to the primary metric of the plurality of collected metrics corresponds to a first time, and data corresponding to the portion of the plurality of the metrics corresponds to a second time that is different from the first time by a time interval;

and

wherein generating the hypothesis set comprises generating a temporally shifted hypothesis set.

31. The method of analyzing network traffic of claim 30, wherein the time interval is a multiple of a data collection interval corresponding to the plurality of metrics.

32. The method of analyzing network traffic of claim 30, further comprising:

receiving an increment or decrement input via a user interface; and

adjusting the time interval based on the received increment or decrement input.

33. The method of analyzing network traffic of claim 30, further comprising:

receiving a temporal shift value from a user interface;

wherein the time interval is adjusted based on the received temporal shift value.

34. A computer readable storage hardware including computer-readable program code that when executed by a processor enable the processor to:

collect data related to network traffic within a computing network;

generate a plurality of metrics associated with the data, the plurality of metrics including network element metrics and node-to-node contexts;

select a primary metric from the plurality of metrics associated with the network element metrics and the node-to-node contexts;

identify a candidate set corresponding to a portion of the plurality of metrics;

generate correlation coefficients between the primary metric and the portion of the plurality of metrics that are members of the candidate set;

generate a hypothesis set based on the correlation coefficients, wherein the hypothesis set includes the portion of the plurality of metrics that are members of the candidate set that are causally related to the primary metric;

identify other metrics of the plurality of metrics with a high correlation to the primary metric, wherein a positive correlation suggests that a higher load on one network element or context is indicative of a causal link to increased utilization of another network element and a negative correlation suggests that one process is indicative of a causal link to inhibiting another process from obtaining adequate resources;

add the other metrics of the plurality of metrics with the high correlation to the primary metric to the hypothesis set; and

terminate a process associated with at least one metric of the plurality of metrics indicated in the hypothesis set.