System and method for wireless data performance monitoring

• US 6,757,543 B2
• Filed: 03/20/2001
• Issued: 06/29/2004
• Est. Priority Date: 03/20/2001
• Status: Expired due to Term

• Current Assignee: Dynatrace LLC (Dynatrace, Inc.)
• Original Assignee: Keynote Systems (Dynatrace, Inc.)
• Inventors: Moran, Brian, Mount, Charles, Feinleib, David A.
• Primary Examiner(s): Tran, CongVan
• Application Number:
  US09/813,431
• Publication Number:
  US 20020177448A1
• Time in Patent Office: 1,197 Days
• Field of Search: 455/456.1, 455/67.1, 455/115.1, 455/466, 455/423, 455/426.1, 455/426.2, 455/428, 455/456.2, 455/456.3, 455/412.1, 455/412.2, 455/445, 455/550.1, 455/527, 455/73, 342/357.07, 342/357.1, 342/357.12, 342/357.13, 342/357.14
• US Class Current: 455/456.1
• CPC Class Codes: H04L 41/00 Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/06 Generation of reports
• CPC Class Codes: H04L 43/0852 Delays
• CPC Class Codes: H04L 43/10 Active monitoring, e.g. heartbeat, ping or trace-route
• CPC Class Codes: H04L 43/106 using time related information in packets, e.g. by adding timestamps
• CPC Class Codes: H04L 43/12 Network monitoring probes
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 43/50 Testing arrangements
• CPC Class Codes: H04W 24/00 Supervisory, monitoring or testing arrangements
• CPC Class Codes: H04W 24/06 Testing, supervising or monitoring using simulated traffic
• CPC Class Codes: H04W 24/10 Scheduling measurement reports ; Arrangements for measurement reports
• CPC Class Codes: H04W 4/14 Short messaging services, e.g. short message services [SMS] or unstructured supplementary service data [USSD]
• CPC Class Codes: H04W 88/18 Service support devices; Network management devices
• CPC Class Codes: H04W 88/184 Messaging devices, e.g. message centre

Abstract

A method and system for monitoring wireless data service performance of wireless service providers. Data messages are sent from a plurality of remote and/or local monitoring probes to other monitoring probes. In general, the remote monitoring probes will be distributed throughout a service area(s) from the service provider and include both fixed probes and mobile probes, wherein each probe will include one or more wireless devices, including cellular phones, PSC phones, PDA devices, Blackberry devices, and the like. Various performance data are then measured, including message deliver latency and network accessibility. This data is then sent from the remote monitoring probes to one or more central monitoring stations, where it is stored and aggregated to generate performance reports and alerts, which are delivered to the service providers or may be accessed through a web user interface. In addition, wireless Internet access service performance is monitored using the remote and local monitoring probes, wherein various access functions are performed using WAP-enabled wireless devices.

Claims

1. A method for monitoring wireless data service performance of a service provider, comprising:

generating a data message to be sent by a sending wireless device;

sending the data message to a destination wireless device via data messaging services provided by the service provider;

measuring a delivery latency corresponding to an amount of time that passes between when the data message was sent from the sending wireless device to when the data message is received by the destination wireless device; and

storing data message delivery performance data corresponding to a delivery of the data message, said data identifying a first location in proximity to where the sending wireless device was when the data message was sent, a second location in proximity to where the destination wireless device was when the data message was received, a time proximate to when the data message was sent, and the delivery latency that was measured for the data message.

2. The method of claim 1, wherein the text message is generated by a computer linked in communication with the sending wireless device.

3. The method of claim 1, wherein the text message comprises a standardized message and the method further comprises:

repeatedly sending the standardized message from at least one of sending device to a plurality of destination wireless devices located at discreet locations that are serviced by the service provider;

measuring a delivery latency for each standardized message that is sent; and

storing message delivery performance data corresponding to each standardized message that was sent.

4. The method of claim 3, further comprising generating performance reports comprising aggregated data corresponding to the delivery latencies that are measured.

5. The method of claim 3, further comprising:

performing the method functions of claim 3 for a plurality of service providers; and

generating performance reports comprising aggregated data corresponding to the delivery latencies that are measured, wherein said aggregated data are grouped by each service provider such that performance of the plurality of service providers can be compared.

6. The method of claim 3, wherein said at least sending device is operated from a vehicle, further comprising sending out a plurality of data messages from different locations as the vehicle is being driven.

7. The method of claim 6, wherein the vehicle is driven along a substantially predetermined route.

8. The method of claim 6, wherein the vehicle comprises one of a taxi or bus.

9. The method of claim 3, wherein the method functions are performed by automated equipment.

10. The method of claim 3, further comprising sending an alert message to the service provider in response to a delivery latency performance level that does not meet predetermined criteria.

11. The method of claim 3, further comprising:

determining if a service problem may exist; and

performing one or more diagnostic tests to determine a cause of the service problem.

12. The method of claim 11, wherein determining if a service problem exists comprises:

aggregating message delivery performance data to establish a baseline performance pattern for a given service provider; and

comparing new message delivery performance data with the baseline performance pattern to identify any significant deviations between them, said any significant deviations being indicative that a service problem exist.

13. The method of claim 1, wherein the sending device comprises one of a wireless phone, a pager, a personal digital assistant (PDA), and a pocket PC.

14. The method of claim 1, further comprising forwarding the data message delivery data to a central monitoring station.

15. The method of claim 1, wherein measuring the delivery latency comprises:

recording a sent timestamp corresponding to a time the data message is sent by the sending wireless device;

recording a received timestamp corresponding to a time the data message is received by the destination wireless device;

storing the sent timestamp and the received timestamp in a database; and

executing a query on data stored in the database corresponding to the sent timestamp and received timestamp, said query returning the delivery latency.

16. The method of claim 1, further comprising:

recording a start receive timestamp corresponding to a time the data message begins to be received by the destination wireless device;

recording an end receive timestamp corresponding to a time the data message has been completely received by the destination wireless device;

determining a size of the data message; and

calculating a data delivery rate based on the size of the data message and a time difference between the end receive timestamp and the start receive timestamp.

17. The method of claim 1, wherein the data message is delivered to the destination wireless device via a short messaging service center (SMSC), further comprising determining an SMSC processing latency based on a time the SMCS receives the data message and a time the SMSC dispatches the data message to be delivered to the destination wireless device.

18. The method of claim 1, wherein the data message is delivered to the destination wireless device via a short messaging service center (SMSC), further comprising:

recording a sent timestamp corresponding to a time the data message is sent from the sending wireless device;

recording a dispatch timestamp corresponding to a time the data message is dispatched from the SMSC; and

determining a time-to-SMSC dispatch processing latency based on a difference between the dispatch timestamp and the sent timestamp.

19. The method of claim 1, wherein the sending wireless device and destination wireless device are located in different metropolitan areas.

20. A method for monitoring wireless data service performance of a plurality of service providers that provide cross-connectivity for delivery of wireless data, comprising:

generating a data message to be sent by a sending wireless device;

sending the data message to a destination wireless device via data messaging services provided a first and second service provider;

measuring a first delivery latency corresponding to an amount of time that passes between when the data message was sent from the sending wireless device to when the data message is passed from the first service provider to the second service provider;

measuring a second delivery latency corresponding to an amount of time that passes between when the data message is passed from the first service provider to the second service provider and when the data message is received by the destination wireless device; and

storing data message delivery performance data corresponding to a delivery of the data message, said data identifying a first location in proximity to where the sending wireless device was when the data message was sent, a second location in proximity to where the destination wireless device was when the data message was received, a time proximate to when the data message was sent, an identity of the first and second service providers and the first and second delivery latencies that were measured for the data message.

21. The method of claim 20, wherein the data message is delivered to the destination wireless device via a short messaging service center (SMSC) corresponding to the first service provider, and wherein determining the first and second delivery latencies comprise:

recording a sent timestamp corresponding to a time the data message is sent by the sending wireless device;

recording a received timestamp corresponding to a time the data message is received by the destination wireless device;

recording a dispatched timestamp corresponding to a time the data message is passed from the SMSC for the first service provider to the second service provider,

wherein the first delivery latency comprises a time difference between the dispatched timestamp and the sent timestamp and the second delivery latency comprises a time difference between the received timestamp and the dispatched timestamp.

22. The method of claim 20 further comprising:

sending a plurality of data messages between sending and destination wireless devices that are serviced by at least three service provider, wherein each data message that is send is handled by two service providers, and the data messages include at least one message that is sent from a sending wireless device that is serviced by each of the service providers that is sent to a destination wireless service serviced by each of the other service providers;

determining a total delivery latency for each of the data messages comprising a time difference between when that message is sent from the sending wireless device to when the message is received by the destination wireless device;

formulating a set of equations corresponding to respective data messages that are sent, each equation containing a first variable corresponding to the service provider for the sending wireless device used to send that message and a second variable corresponding to the service provider for the destination wireless device used to receive that message; and

simultaneously solving the set of equations to resolve the values of the variables, thereby determining proportional latencies corresponding to respective portions of the total delivery latency due to each of the service providers in delivering the data messages.

23. A method for monitoring wireless Internet access service performance of a service provider, comprising:

requesting to access an Internet site using a WAP-enabled wireless device serviced by the service provider;

recording a request timestamp corresponding to a time that the request is made;

recording a delivery start timestamp corresponding to a time when data begins to be delivered to the WAP-enabled wireless device from the Internet site via the service provider;

determining a delivery latency comprising a difference between the delivery start timestamp and the request timestamp; and

storing wireless Internet access performance data corresponding to the access of data from the Internet web site that includes a location in proximity to where the WAP-enabled wireless device was when the request to access the Internet site was made, a time proximate to when the request to access the Internet site was made, and the delivery latency that was measured.

24. The method of claim 23, wherein the request to access the Internet site is generated by a computer linked in communication with the WAP-enabled wireless device.

25. The method of claim 23, further comprising:

requesting access to one or more Internet sites using a plurality of WAP-enabled wireless devices located at discreet locations;

measuring a delivery latency for each request to access said one or more Internet sites; and

storing wireless Internet access performance data corresponding to each request to access said one or more Internet sites.

26. The method of claim 25, further comprising generating performance reports comprising aggregated data corresponding to the delivery latencies that are measured.

27. The method of claim 25, further comprising:

performing the method functions of claim 25 for a plurality of service providers; and

generating performance reports comprising aggregated data corresponding to the delivery latencies that are measured, wherein said aggregated data are grouped by each service provider such that performance of the plurality of service providers can be compared.

28. The method of claim 25, wherein at least one of said plurality of WAP-enabled wireless devices is operated from a vehicle, further comprising accessing said one or more Internet sites with the WAP-enabled wireless device different locations as the vehicle is being driven.

29. The method of claim 25, further comprising sending an alert message to the service provider in response to a delivery latency performance level that does not meet a predetermined criteria.

30. The method of claim 25, further comprising:

determining if a service problem may exist; and

performing one or more diagnostic tests to determine a cause of the service problem.

31. The method of claim 30, wherein determining if a service problem exists comprises:

aggregating message delivery performance data to establish a baseline performance pattern for a given service provider; and

comparing new message delivery performance data with the baseline performance pattern to identify any significant deviations between them, said any significant deviations being indicative that a service problem exist.

32. The method of claim 23, further comprising forwarding the data message delivery data to a central monitoring station.

33. The method of claim 23, further comprising:

recording an completely delivered timestamp corresponding to a time when content from the Internet site has been completely delivered to the WAP-enabled wireless device;

determining an amount of content that is delivered; and

calculating a data delivery rate based on the amount of content that is delivered and a time difference between the completely delivered timestamp and the delivery start timestamp.

34. The method of claim 23, wherein the method functions are performed by automated equipment.

35. The method of claim 23, further comprising:

performing a scripted set of interactions with controls on the Internet site to enable a predetermined function to be performed via the WAP-enabled wireless device;

determining an execution latency corresponding to an amount of time it takes to perform the predetermined function.

36. The method of claim 35, wherein the predetermined function comprises logging on to the Internet site.

37. The method of claim 35, wherein the predetermined function comprises executing a stock trade through stock-trading services provided by the Internet site.

38. A system for monitoring wireless data service performance of one or more service providers, comprising:

a central monitoring station including a means for storing message delivery performance data; and

a plurality of remote monitoring probes located at discreet locations, each remote monitoring probe linked in communication with the central monitoring station and including;

at least one wireless messaging device that sends data messages to other remote monitoring probes and/or the central monitoring station and/or receives data messages from other remote monitoring probes and/or the central monitoring station;

means for capturing messaging delivery performance data corresponding to data messages sent by and/or received by the remote monitoring probe, said message delivery performance data including timestamps corresponding to when data messages are sent and/or received by that remote monitoring probe and an identity of the wireless messaging device that sends or receives a data message; and

means for sending message delivery performance data captured by the remote monitoring probe to the central monitoring station.

39. The system of claim 38, wherein at least one of the plurality of remote monitoring probes comprises a mobile remote monitoring probe that is attached to a vehicle that is driven to different locations while that probes sends and/or receives data messages.

40. The system of claim 39, wherein said at least one remote monitoring probe further includes a position locating device that provides data identifying a location of said at least one remote monitoring probe.

41. The system of claim 38, wherein said means for capturing and sending data comprises a computer linked in communication with said at least one wireless messaging device.

42. The system of claim 41, wherein the computer runs software that automatically generates data messages that are sent from the remote monitoring probe.

43. The system of claim 41, wherein the computer runs software that automatically records message delivery performance data corresponding to data messages received by that probe.

44. The system of claim 38, wherein said at least one wireless messaging device comprises at least two different devices from among the group of a wireless cellular phone, a wireless PCS phone, a wireless pager, a personal digital assistant (PDA) and a pocket PC.

45. The system of claim 38, wherein the central monitoring station includes:

a computer running a database in which the message delivery performance data is stored; and

report generation software that generates performance reports comprising aggregated data corresponding to message delivery latencies derived from message delivery performance data stored in the database.

46. The system of claim 45, wherein the computer at the central monitoring station further includes software that generates an alert to a service provider when message delivery performance data sent to the central monitoring station indicates a message delivery performance level for the service provider has fallen below a predetermined threshold.

47. The system of claim 38, wherein the means for sending data captured by a remote monitoring probe sends data to the central monitoring station on a periodic basis.

48. The system of claim 38, wherein the means for sending message delivery performance data captured by a remote monitoring probe sends the message delivery performance data to the central monitoring station as that data is captured by the remote monitoring probe.

49. A system for monitoring wireless Internet access performance of one or more service providers, comprising:

a central monitoring station including a means for storing Internet access performance data; and

a plurality of remote monitoring probes located at discreet locations, each remote monitoring probe linked in communication with the central monitoring station and including;

at least one WAP-enabled wireless device that requests data from and provides data to one or more Internet sites;

means for capturing wireless Internet access performance data corresponding to wireless Internet site accesses performed by said at least one WAP-enabled wireless device, said wireless Internet access performance data including timestamps corresponding to when data is requested and received by said at least one WAP-enabled wireless device; and

means for sending wireless Internet access performance data captured by the remote monitoring probe to the central monitoring station.

50. The system of claim 49, wherein at least one of the plurality of remote monitoring probes comprises a mobile remote monitoring probe that is attached to a vehicle that is driven to different locations while said at least one WAP-enabled wireless device interacts with one or more Internet sites.

51. The system of claim 50, wherein said at least one remote monitoring probe further includes a position locating device that provides data identifying a location of said at least one remote monitoring probe.

52. The system of claim 49, wherein said means for capturing and sending data comprises a computer linked in communication with said at least one WAP-enabled wireless device.

53. The system of claim 49, wherein the computer runs software that automatically controls a sequence of interactions between a WAP-enabled wireless device and an Internet site.

54. The system of claim 53, wherein the sequence of interactions comprise executing a stock trade on an Internet site that provides online stock trading services for wireless devices.

55. The system of claim 49, wherein said at least one wireless messaging device comprises at least two different devices from among the group of a wireless cellular phone, a wireless PCS phone, a wireless pager, a personal digital assistant (PDA) and a pocket PC.

56. The system of claim 49, wherein the central monitoring station includes:

a computer running a database in which the wireless Internet access performance data is stored; and

report generation software that generates performance reports comprising aggregated data corresponding to wireless Internet access delivery latencies derived from wireless Internet access performance data stored in the database.

57. The system of claim 49, wherein the computer at the central monitoring station further includes software that generates an alert to a service provider when wireless Internet access performance data sent to the central monitoring station indicates a wireless Internet access performance level for the service provider has fallen below a predetermined threshold.

58. The system of claim 49, wherein the means for sending data captured by a remote monitoring probe sends data to the central monitoring station on a periodic basis.

59. The system of claim 49, wherein the means for sending message delivery performance data captured by a remote monitoring probe sends the message delivery performance data to the central monitoring station as that data is captured by the remote monitoring probe.