Methods and apparatus for monitoring end-user experience in a distributed network

• US 6,970,924 B1
• Filed: 02/23/1999
• Issued: 11/29/2005
• Est. Priority Date: 02/23/1999
• Status: Expired due to Fees

• Current Assignee: Visual Networks Incorporated Corporation of The State of California
• Original Assignee: Visual Networks, Inc. (Fortive Corp.)
• Inventors: Davis, Chris, Chu, Gigi, Verma, Sudanshu, Leong, John, Dickson, Craig, Burdick, Matt, Bhide, Shekhar
• Primary Examiner(s): Dinh, Dung C.
• Application Number:
  US09/256,647
• Time in Patent Office: 2,471 Days
• Field of Search: 709/224, 709/239, 709/245, 709/223, 709/203, 709/225, 709/226, 709/227, 709/228, 370/248, 370/252, 717/177, 717/178, 717/172, 717/173, 714/47
• US Class Current: 709/224
• CPC Class Codes: H04L 41/5067 Customer-centric QoS measurements
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/06 Generation of reports
• CPC Class Codes: H04L 43/0811 by checking connectivity
• CPC Class Codes: H04L 43/0817 by checking functioning

Abstract

A network performance monitoring system uses a plurality of user modules, each user operating on a unique user machine coupled to one or more provider servers and an experience test server for collecting data from the plurality of user modules. The collected data includes at least one performance datum relating to user experience with a link from the user machine to the provider server. The system also cleanses the collected data to account for variable user configurations, user errors and the like.

Claims

1. A network performance monitoring system comprising:

a plurality of user modules, wherein each user module operates on a unique user machine coupled to one or more provider servers;

an experience test server for collecting data from the plurality of user modules, wherein the collected data includes at least one performance datum relating to user experience with a link from the user machine to the provider server; and

means for cleansing the collected data to account for variable user configurations, wherein the means for cleansing identifies when given data samples with disparate values represent similar samplings due to user configuration variations.

2. The apparatus of claim 1, wherein the user machines are couple to the provider servers over dial-up connections.

3. The apparatus of claim 2, further comprising a service level report generator, wherein a service level report generated by the service level report generator is a report indicating the level of service provided to the unique user machines and the level of service is based, at least in part, on the cleansed collected data.

4. The apparatus of claim 1, wherein the data samples represent point-of-presence IDs and the disparate values result from user variations in representations of point-of-presence IDs.

5. The apparatus of claim 1, wherein the data samples are clock times and the disparate values result from user variations in local clocks.

6. The apparatus of claim 1, wherein the experience test server further comprises logic to allocate tests among the plurality of user modules.

7. The apparatus of claim 6, wherein the logic to allocate the tests among the plurality of user modules is logic that operates without requiring prior knowledge of the number of user modules available for running tests.

8. The apparatus of claim 6, wherein the logic is logic programmed to allocate tests based on one or more criterion, wherein the one or more criterion are selected from a test type, matching test parameters, maximum number of tests, test durations and conditions under which tests can be allocated.

9. A method of monitoring an end-user experience of attempting to connect a client system to a distributed network, comprising:

detecting that a user is attempting to connect the client system to the distributed network;

monitoring a connection code to obtain user experience data about the connection process, wherein the connection code indicates whether the client system has successfully established a connection to the distributed network, and wherein the user experience data is data relating to the user'"'"'s experience, prior to establishing a connection to the distributed network, of attempting to connect to the distributed network; and

transmitting the data obtained from the connection process to a collector of user connection experience.

10. The method of claim 9, wherein monitoring is done as a background process.

11. The method of claim 9, further comprising:

executing tests on the distributed network from a test point;

querying routers to determine router statistics; and

adjusting results of the executed tests based on the router statistics.

12. In a network monitoring system for monitoring network-based services over a distributed network accessible by user devices capable of collecting data about end-user experience and communicating network performance data to an experience test server, a method of monitoring network-based services, comprising:

configuring the user devices to notify the experience test server of an availability to perform network performance tests in response to being connected to the distributed network;

distributing instructions from the experience test server to the user devices that are available to perform network performance tests, in accordance with notifications from the user devices, wherein at least some of the instructions direct the user devices to perform network performance tests; and

collecting, at the experience test server, network performance data generated by the user devices that perform the network tests;

wherein upon receiving an availability notification from a user device, the experience test server sends an instruction to the user device irrespective of whether the experience test server instructs the user device to perform a network performance test, and wherein instructions that lack test requests direct the user device to contact the experience test server in response to occurrence of a trigger event.

13. The method of claim 12, wherein the instructions distribute the network performance tests over time to available user devices.

14. The method of claim 12, further comprising:

checking test quota limits associated with a user device before instructing the user device to run a test.

15. The method of claim 12, further comprising:

dynamically controlling a rate of test allocation to distribute tests over a test period based on a current test rate.

16. The method of claim 12, further comprising:

dynamically changing test allocation among user devices without prior knowledge of number of user devices available for testing.

17. The method of claim 12, wherein performance of the network performance tests is transparent to the users operating the user devices.

18. The method of claim 12, wherein transmission of the network performance data from the user devices to the experience test server is transparent to users operating the user devices.

19. The method of claim 12, wherein the user devices are connected to the distributed network under control of users who are customers of a provider of the network-based services.

20. The method of claim 12, wherein the user devices maintain information relating to their own testing state, and the experience test server does not maintain testing state information for individual user devices.

21. A network monitoring system for monitoring network-based services over a distributed network, comprising:

a plurality of user devices capable of performing network performance tests and collecting data about end-user experience, wherein the user devices are configured to notify an experience test server of an availability to perform network performance tests in response to being connected to the distributed network; and

the experience test server, configured to receive availability notifications from the user devices and to distribute instructions to the user devices that are available to perform network performance tests, wherein at least some of the instructions direct the user devices to perform network performance tests, and wherein the experience test server collects network performance data generated by the user devices that perform the network performance tests;

wherein, upon receiving an availability notification from a user device, the experience test server sends an instruction to the user device irrespective of whether the experience test server instructs the user device to perform a network performance test and wherein instructions that lack test requests direct the user device to contact the experience test server in response to occurrence of a trigger event.

22. The system of claim 21, wherein the user devices perform network performance tests that are transparent to the users operating the user devices.

23. The system of claim 21, wherein the user devices send network performance data to the experience test server in a manner transparent to users operating the user devices.

24. The system of claim 21, wherein the experience test server distributes instructions to available user devices such that performance of the network performance tests is distributed over time.

25. The system of claim 21, wherein the experience test server checks test quota limits associated with a user device before instructing the user device to run a test.

26. The system of claim 21, wherein the experience test server dynamically controls a rate of test allocation to distribute tests over a test period based on a current test rate.

27. The system of claim 21, wherein the experience test server dynamically changes test allocation among user devices without prior knowledge of number of user devices available for testing.

28. The system of claim 21, wherein the user devices are connected to the distributed network under control of users who are customers of a provider of the network-based services.

29. The system of claim 21, wherein the user devices maintain information relating to their own testing state, and the experience test server does not maintain testing state information for individual user devices.