Monitoring and detecting causes of failures of network paths

• US 9,001,667 B1
• Filed: 03/31/2011
• Issued: 04/07/2015
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Khanna, Richendra, Wallace, Edward L., Brar, Jagwinder Singh, Marr, Michael David, McKelvie, Samuel J., DeSantis, Peter N., Nowland, Ian P., Klein, Matthew D., Mason, Justin, Gabrielson, Jacob
• Primary Examiner(s): Kizou, Hassan
• Assistant Examiner(s): Belur, Deepa
• Application Number:
  US13/077,323
• Time in Patent Office: 1,468 Days
• Field of Search: 370/241, 370/254
• US Class Current: 370/241
• CPC Class Codes: H04L 41/0631 using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters

Abstract

Generally described, systems and methods are provided for monitoring and detecting causes of failures of network paths. The system collects performance information from a plurality of nodes and links in a network, aggregates the collected performance information across paths in the network, processes the aggregated performance information for detecting failures on the paths, analyzes each of the detected failures to determine at least one root cause, and initiates a remedial workflow for the at least one root cause determined. In some aspects, processing the aggregated information may include performing a statistical regression analysis or otherwise solving a set of equations for the performance indications on each of a plurality of paths. In another aspect, the system may also include an interface which makes available for display one or more of the network topology, the collected and aggregated performance information, and indications of the detected failures in the topology.

Claims

1. A computer system configured to execute root cause analysis of failures across a network, the system comprising:

a topology component, implemented on a computing device, the topology component configured to;

determine a topology of a plurality of network paths, the paths comprising a plurality of nodes and links interconnecting pairs of nodes;

compare the determined topology to an expected topology; and

reconcile the determined topology to the expected topology;

a collection and aggregation component, implemented on a computing device, the collection and aggregation component configured to collect and aggregate information from the network paths; and

a root cause analysis component, implemented on a computing device, the root cause analysis component configured to process the aggregated information to determine at least one of a faulty node and faulty link in the reconciled topology.

2. The computer system of claim 1, wherein the information collected from the paths comprises performance indications of the paths.

3. The computer system of claim 2, wherein the performance indications comprise at least one of packet latency, packet loss, bandwidth and jitter.

4. The computer system of claim 2, wherein the root cause analysis component is configured to process the aggregated information by performing statistical analysis for the performance indications across each of the plurality of paths and solving a set of equations comprising an equation for each of the different paths in order to find the performance indication associated with the nodes and links on the paths.

5. The computer system of claim 1, further comprising an interface component, implemented on a computing device, the interface component configured to display the topology.

6. The computer system of claim 5, wherein the interface component is further configured to display the topology with color indications representative of the at least one of a faulty node and faulty link in the topology.

7. The computer system of claim 1, wherein the root cause analysis component is further configured to use the topology of the plurality of network paths to process the aggregated information.

8. The computer system of claim 1, wherein the topology component is further configured to determine a topology of power to the nodes in the network.

9. The computer system of claim 1, wherein the topology component is further configured to determine a topology for at least one of:

a physical location within a thermal zone, a room and a data center.

10. A computer-implemented method of isolating the root cause of performance issues in a computer network, the method comprising:

generating a topology of the network;

comparing the generated topology to an expected topology;

reconciling the generated topology to the expected topology;

collecting performance indications for packets traveling on each of a plurality of paths between two endpoints in the network, wherein each path between the two endpoints comprises at least two nodes and one link;

aggregating the performance indications for the plurality of paths;

processing the aggregated performance indications to statistically analyze for the performance indications across each of the plurality of paths; and

solving a set of equations comprising an equation for each of the different paths in the reconciled topology in order to find the performance indication associated with the at least two nodes and one link on each of the paths.

11. The computer-implemented method of claim 10, wherein the performance indications comprise at least one of packet latency, packet loss, bandwidth and jitter.

12. The computer-implemented method of claim 10, further comprising determining at least one of a faulty node and faulty link in the topology based on the performance indication associated with the at least two nodes and one link on each of the paths.

13. The computer method of claim 12 further comprising causing the topology to be displayed with color indications representative of the at least one of a faulty node and faulty link in the topology.

14. The computer-implemented method of claim 10, wherein the processing of the aggregated performance indications comprises using the topology of the network to determine whether to collect performance indications for additional paths in the network.

15. The computer-implemented method of claim 10, further comprising generating a topology of power to the nodes in the network.

16. The computer-implemented method of claim 10, further comprising generating a topology for at least one of:

a physical location within a thermal zone, a room and a data center.

17. A non-transitory, computer-readable medium having a computer-executable component, the computer-executable component comprising:

a network topology generation component for;

generating a topology of network paths;

comparing the generated topology to an expected topology; and

reconciling the generated topology to the expected topology;

a measuring component for;

measuring an indication of performance along each of a plurality of paths between two endpoints, wherein each of the plurality of paths includes at least one link and at least two devices;

equating the indication of performance across each path to a sum of a performance indication associated with the at least one link on the path and a performance indication associated with each of the at least two devices; and

developing a set of equations for the performance indications across each of the plurality of paths; and

a root cause finding component for;

solving the set of equations in order to calculate failure modes indicated by the measured performance associated with each of the at least two nodes and one link on each of the paths of the reconciled topology.

18. The non-transitory, computer-readable medium of claim 17, further comprising a topology component for determining a topology of the plurality of paths.

19. The non-transitory, computer-readable medium of claim 18, further comprising an interface component for displaying the topology.

20. The non-transitory, computer-readable medium of claim 18, wherein the computer-executable component further comprises the measuring component for determining at least one of a faulty node and faulty link in the topology based on the delay associated with each of the at least two nodes and one link on each of the paths.

21. The non-transitory, computer-readable medium of claim 20, wherein the computer-executable component further comprises the interface component for displaying the topology with color indications representative of the at least one of a faulty node and faulty link in the topology.

22. The non-transitory, computer-readable medium of claim 17, wherein the root cause finding component uses the topology of the network to determine whether to collect performance indications for additional paths in the network.

23. The non-transitory, computer-readable medium of claim 17, further comprising a topology component for determining a topology of power to the nodes in the network.

24. The non-transitory, computer-readable medium of claim 17, further comprising a topology component for determining a topology of at least one of a physical location within a thermal zone, a room and a data center.