Report generation and visualization systems and methods and their use in testing frameworks for determining suitability of a network for target applications

• US 20030091165A1
• Filed: 10/15/2002
• Published: 05/15/2003
• Est. Priority Date: 10/15/2001
• Status: Active Grant

• Current Assignee: TP-Link USA Corporation
• Original Assignee: Volli Polymer GmbH LLC (Intellectual Ventures LLC)
• Inventors: Denby, Lorraine, Meloche, Jean, Stott, David Thomas, Bearden, Mark J., Karacali, Bengi
• Granted Patent: US 8,868,715 B2
• Time in Patent Office: Days
• US Class Current: 379/88.08
• CPC Class Codes: H04L 41/0213 Standardised network management protocols, e.g. simple network management protocol [SNMP]
• CPC Class Codes: H04L 41/0631 using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
• CPC Class Codes: H04L 41/0681 Configuration of triggering conditions
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 43/50 Testing arrangements
• CPC Class Codes: H04M 3/2236 Quality of speech transmission monitoring
• CPC Class Codes: H04M 3/2254 in networks
• CPC Class Codes: H04M 7/006 Networks other than PSTN/ISDN providing telephone service, e.g. Voice over Internet Protocol (VoIP) , including next generation networks with a packet-switched transport layer H04L65/00 takes precedence; aspects not specific to the type of network H04M3/00; special services in those networks H04M3/42

Abstract

A system visualizes and generates reports from stored data, concerning various measured parameters related to a network carrying call traffic. The network is displayed in an easily recognized arrangement, with various parameters being indicated by different shapes and colors for each device on the network. The parameters are changed to indicate the status of the network at different times. More detailed information about a parameter, a device or a path between devices may be obtained simply using a mouse or keystrokes. A variety of different types of detailed displays allows the operator to troubleshoot problems in the network.

Claims

1. A method of analyzing and displaying data about a network of devices carrying calls, comprising:

retrieving stored data about said network from a data store;

assigning specific shapes to different types of devices and interconnections in said network;

analyzing said data to determine which calls traveling through said network are bad calls;

determining which devices and links are traversed by said bad calls and good calls;

assigning colors to devices and links in said network based on a proportion of bad calls traversing each device;

assigning colors to devices and links in said network reflecting load, utilization or error data associated with said device and link;

displaying said network with devices and interconnections represented by said assigned shapes and colors; and

providing a user interaction mechanism for manual intervention and requests for diagnostic visualizations.

2. The method according to claim 1, further comprising:

selecting two devices on said displayed network;

detecting a path for calls between said two devices; and

displaying said path by changing one of color and shading.

3. The method according to claim 1, wherein said display reflects a selected time interval.

4. The method according to claim 1, further providing a mechanism to reflect a time interval selected.

5. The method according to claim 1, wherein a metric used to determine bad calls is end-to-end one-way delay.

6. The method according to claim 1, wherein a metric used to determine bad calls is jitter.

7. The method according to claim 1, wherein a metric used to determine bad calls is packet loss.

8. The method according to claim 1, wherein a metric used to determine bad calls is packet loss burst.

9. The method according to claim 1, wherein said user interaction mechanism specifies the placement of a link or device with mouse clicks or key strokes.

10. The method according to claim 1, wherein said user interaction mechanism selects diagnostic visualizations with a mouse click or key stroke.

11. A method of analyzing and displaying data about a network of devices comprising:

retrieving stored data from a data store;

determining percentages of utilization for links during specified measurement intervals; and

displaying a graph of inbound and outbound utilization on each interface on each device in said network separately for each measurement interval.

12. A method of analyzing and displaying data about a network of devices comprising:

retrieving stored data from a data store;

determining total counts of each device variable during a specified measurement interval; and

displaying a graph of counts on each device in said network separately.

13. A method of analyzing and displaying data about a network of devices comprising:

retrieving stored data from a data store;

determining load, utilization or error data during a measurement interval for one device; and

displaying a graph of inbound and outbound load, utilization or error on each interface on said device in said network separately.

14. A method of analyzing and displaying data about a network of devices comprising:

retrieving stored data from a data store;

determining total counts of a load or error variable on each interface for one device during a measurement interval; and

displaying a graph of counts on each interface on said device in said network separately.

15. A method of analyzing and displaying data about a network of devices comprising:

retrieving stored data from a data store;

determining load, utilization or error data over time during a measurement interval; and

displaying a graph of load, utilization or error data versus time for an interface on a device in said network separately for packets entering and packets leaving said device.

16. The method according to claim 15, wherein the measurement interval is a series of days including weekdays and weekends.

17. The method according to claim 15 wherein the distribution of the load, utilization or error data over an interval is expressed plotted versus time.

18. A method of analyzing and displaying network data comprising:

retrieving stored data from a data store;

summarizing QoS metric values for each call between two endpoints and for each QoS metric;

displaying graphs of call metrics over a measured time period versus time of the call for both directions of the call; and

displaying a shaded area that indicates a threshold of acceptable calls.

19. A method of analyzing and displaying network data comprising:

retrieving stored data from a data store;

determining percentages of bad calls in a given path in said network over time;

displaying a graph of percentages of bad calls over a measured time period versus time; and

displaying the path in reference to the network.

20. The method according to claim 19, wherein said bad calls are determined in regard to end-to-end on-way delay.

21. The method according to claim 19, wherein said bad calls are determined in regard to jitter.

22. The method according to claim 19, wherein said bad calls are determined in regard to packet loss.

23. The method according to claim 19, wherein said bad calls are determined in regard to packet loss burst.

24. A method of analyzing and displaying data about a network of devices comprising:

retrieving stored data from a data store;

determining summary statistics for the distribution of each QoS metric for specified time intervals in each direction for all calls that took a given path in said network over time;

displaying said summary statistics versus time;

displaying a shaded area that indicates a threshold of acceptable calls; and

displaying the path in reference to the network.

25. A method according to claim 24, wherein the displayed acceptable call threshold is determined in regard to delay.

26. A method according to claim 24, wherein the displayed acceptable call threshold is determined in regard to jitter.

27. A method according to claim 24, wherein the displayed acceptable call threshold is determined in regard to packet loss.

28. A method according to claim 24, wherein the displayed acceptable call threshold is determined in regard to packet loss burst.

29. A method of analyzing and displaying data about a network of devices comprising:

retrieving stored data from a data store;

determining the exact path that a given call took through the network, including devices and interfaces;

determining device or link load, utilization or error data over time during a measurement interval;

determining device or link load, utilization or error data over time during a measurement interval corresponding to some period of time when the given call was in progress;

displaying a graph of device or link load, utilization or error data over time for each interface on each device in said path separately for both directions of the path; and

displaying the path in reference to the network.

30. A method of analyzing and displaying data about a network of devices comprising:

retrieving stored data from a data store;

determining an exact path that a given call took through the network, including devices and interfaces;

determining error counts over time during a measurement interval corresponding to some period of time when the given call was in progress;

displaying a graph of error counts over time for each interface on each device in said path separately for both directions of the path; and

displaying the path in reference to the network.

31. A method of analyzing and displaying a data about a network of devices comprising:

retrieving stored data from a data store;

analyzing said data to determine numbers of bad calls and levels of call metrics between different sections of said network; and

displaying a graph of said numbers of bad calls and levels of call metrics for different sections of said network.

32. A method according to claim 31, wherein said graph is divided into section representing selected time intervals.

33. The method according to claim 31 where the selected time intervals are regular work hours, hours during the week outside of regular work hours, and weekends.

34. The method according to claim 31, wherein the graph uses a box and whisker graph.

35. The method according to claim 31, wherein a shaded area indicates a threshold of acceptable calls.

36. A method of analyzing and displaying data about a network of devices comprising:

an apparatus that displays the network;

an apparatus for interacting with the network display;

an apparatus for obtaining visualizations of device, link, path and call data;

an apparatus for specifying the subset of calls used to produce the visualizations of device, link, and path data;

an apparatus for specifying the subset of the network devices and links used to produce the visualization of the call and path data;

an apparatus for specifying the periods of time for which the visualizations show the call, device, link, and path data;

an apparatus for manually modifying devices and links; and

an apparatus for accessing and modifying collected discovery and topology data.

37. A method for analyzing and reporting integrated network topology, network traffic statistics, and end-to-end quality of service (QoS) measurements comprising:

analyzing traffic statistics;

analyzing end-to-end QoS measurements;

relating network traffic statistics with end-to-end QoS measurements on a common timescale and across network paths;

displaying network topology;

displaying traffic measurements on network devices and links;

displaying end-to-end QoS measurements; and

navigating across relevant displays.

38. The method according to claim 37 for analyzing traffic statistics comprising:

retrieving stored data about said network from a data store; and

determining load, utilization, and error statistics of devices and links in said network at specified times.

39. The method according to claim 37 for analyzing end-to-end QoS comprising:

retrieving stored data about said network from a data store;

determining the source, destination, duration, and the time of end-to-end traffic associated with measurements of poor quality of service metrics;

identifying said poor quality traffic in terms of calls;

determining the source, destination, duration and the time of end-to-end traffic associated with measurements of acceptable quality of service metrics; and

identifying said acceptable quality traffic in terms of calls.

40. The method according to claim 39, wherein the metric used to determine poor quality calls is end-to-end one-way delay.

41. The method according to claim 39, wherein the metric used to determine poor quality calls is jitter.

42. The method according to claim 39, wherein the metric used to determine poor quality calls is packet loss.

43. The method according to claim 39, wherein the metric used to determine poor quality calls is packet loss burst.

44. The method according to claim 37 for relating traffic statistics with end-to-end QoS measurements on a common timescale and across network paths comprising:

retrieving stored data about said network from a data store;

determining simultaneous traffic and call measurements;

determining network devices, links, and paths traversed by each call;

determining which devices and links are traversed by poor quality calls; and

determining calls traversing each network device, link and path.

45. The method according to claim 37 for displaying network topology comprising:

retrieving stored data about said network from a data store;

assigning specific shapes to different types of devices and links in said network; and

displaying said network with devices and links represented by said assigned shapes.

46. The method according to claim 45 for displaying network topology further comprising:

assigning colors to devices and links in said network based on a proportion of poor quality calls traversing each over a given time interval;

assigning colors to devices and links in said network reflecting load, utilization, and error statistics associated with the said device and link; and

displaying said network with devices and links represented by said colors.

47. The method according to claim 46 further comprising:

providing a mechanism to reflect the time interval selected; and

providing a mechanism to change the given time interval.

48. The method according to claim 45, further comprising:

selecting two devices in said displayed network;

detecting a path for calls between said two devices; and

displaying said path by changing one of color and shading.

49. The method according to claim 37 for displaying traffic statistics comprising:

retrieving stored data from a data store;

determining percentages of utilization for links during specified measurement intervals; and

displaying a graph of inbound and outbound utilization on each interface on each device in said network separately for each time measurement interval.

50. The method according to claim 49, further comprising:

determining total counts of each device variable during a specified measurement interval; and

displaying a graph of said counts on each device in said network separately.

51. The method according to claim 50, further comprising:

determining load, utilization or error data during a measurement interval for one device; and

displaying a graph of inbound and outbound load, utilization or error on each interface on said device in said network separately.

52. The method according to claim 51, further comprising:

determining total counts of a load or error variable on each interface for one device during a measurement interval; and

displaying a graph of said counts on each interface on said device in said network separately.

53. The method according to claim 52, further comprising:

retrieving stored data from a data store;

determining load, utilization or error data over time during a measurement interval; and

displaying a graph of load, utilization or error data versus time for an interface on a device in said network separately for packets entering and packets leaving said device.

54. The method according to claim 52, wherein the measurement interval is a series of days including weekdays and weekends.

55. The method according to claim 52, wherein the distribution of the load, utilization or error data over an interval is plotted versus time.

56. A method according to claim 37 for displaying end-to-end QoS comprising:

retrieving stored data from a data store;

summarizing QoS metric values for each call between two endpoints and for each QoS metric;

displaying graphs of call metrics over a measured time period versus time of the call for both directions of the call; and

displaying a shaded area that indicates a threshold of acceptable calls.

57. The method according to claim 56 further comprising:

determining percentages of poor quality calls in a given path in said network over time;

displaying a graph of percentages of poor quality calls over a measured time period versus time; and

displaying the path in reference to the network.

58. The method according to claim 57, wherein said poor quality calls are determined in regard to end-to-end one-way delay.

59. The method according to claim 57, wherein said poor quality calls are determined in regard to jitter.

60. The method according to claim 57, wherein said poor quality calls are determined in regard to packet loss.

61. The method according to claim 57, wherein said poor quality calls are determined in regard to packet loss burst.

62. The method according to claim 57, further comprising:

determining summary statistics for the distribution of each QoS metric for specified time intervals in each direction for all calls that took a given path in said network over time;

displaying said summary statistics versus time;

displaying a shaded area that indicates a threshold of acceptable calls; and

displaying the path in reference to the network.

63. A method according to claim 62, wherein the displayed acceptable call threshold is determined in regard to end-to-end one-way delay.

64. A method according to claim 62, wherein the displayed acceptable call threshold is determined in regard to jitter.

65. A method according to claim 62, wherein the displayed acceptable call threshold is determined in regard to packet loss.

66. A method according to claim 62, wherein the displayed acceptable call threshold is determined in regard to packet loss burst.

67. The method according to claim 62, further comprising:

determining the exact path that a given call took through the network, including devices and interfaces;

determining device or link load, utilization or error data over time during a measurement interval;

determining device or link load, utilization or error data over time during a measurement interval corresponding to some period of time when the given call was in progress;

displaying a graph of device or link load, utilization or error data over time for each interface on each device in said path separately for both directions of the path; and

displaying the path in reference to the network.

68. The method according to claim 67 further comprising:

determining an exact path that a given call took through the network, including devices and interfaces;

determining error counts over time during a measurement interval corresponding to some period of time when the given call was in progress;

displaying a graph of error counts over time for each interface on each device in said path separately for both directions of the path; and

displaying the path in reference to the network.

69. The method according to claim 68 further comprising:

retrieving stored data from a data store;

analyzing said data to determine numbers of poor quality calls and levels of call metrics between different sections of said network; and

displaying a graph of said numbers of poor quality calls and levels of call metrics for different sections of said network.

70. A method according to claim 69, wherein said graph is divided into sections representing selected time intervals.

71. The method according to claim 69, wherein the selected time intervals are adjustable including regular work hours, hours during the week outside of regular work hours, and weekends.

72. The method according to claim 69, wherein the graph uses a box and whisker graph.

73. The method according to claim 69, wherein a shaded area indicates a threshold of acceptable calls.

74. The method according to claim 37, navigating across relevant displays comprising:

selecting a visualized item by a mouse click or a key stroke; and

retrieving displays containing data pertaining to the selected item.

75. The method according to claim 74, wherein the selected item is a network device.

76. The method according to claim 74, wherein the selected item is a network link.

77. The method according to claim 74, wherein the selected item is an endpoint pair.

78. The method according to claim 74, wherein the selected item is a network path.

79. A system for analyzing and reporting integrated network topology, network traffic statistics, and end-to-end quality of service (QoS) measurements comprising:

an apparatus that analyzes network traffic statistics;

an apparatus that analyzes end-to-end QoS measurements;

an apparatus that relates network traffic statistics with end-to-end QoS measurements on a common timescale and across network paths;

an apparatus that displays the network;

an apparatus for interacting with the network display;

an apparatus for obtaining visualizations of device, link, path and call data;

an apparatus for specifying the subset of calls used to produce the visualizations of device, link, and path data;

an apparatus for specifying the subset of the network devices and links used to produce the visualization of the call and path data;

an apparatus for specifying the periods of time for which the visualizations show the call, device, link, and path data; and

an apparatus for accessing topological discovery data.