Systems and methods for characterizing the performance of a wireless network

• US 7,865,194 B2
• Filed: 04/13/2006
• Issued: 01/04/2011
• Est. Priority Date: 04/13/2006
• Status: Expired due to Fees

• Current Assignee: AT&T Mobility LLC (AT&T, Inc.)
• Original Assignee: Carrier Iq
• Inventors: Roskowski, Steve
• Primary Examiner(s): Pérez-Gutiérrez; Rafael
• Assistant Examiner(s): Batista; Marcos
• Application Number:
  US11/403,263
• Publication Number:
  US 20070243881A1
• Time in Patent Office: 1,727 Days
• Field of Search: 455/456.5, 455/456.6
• US Class Current: 455/456.5
• CPC Class Codes: H04W 24/00 Supervisory, monitoring or testing arrangements
• CPC Class Codes: H04W 24/08 Testing, supervising or monitoring using real traffic
• CPC Class Codes: H04W 88/18 Service support devices; Network management devices

Abstract

Systems and methods for characterizing a performance of a wireless network. Data or metrics are collected from a device. The metrics include timing information associated with certain sectors. The timing information can be used to determine an approximate location of the device when the call or other use of the device occurred. A device rating is used to contribute the metrics of the device to bins of a grid that describes the wireless network. Only the bins associated with the approximate location of the device receive the contribution of the metrics. The metrics from other devices are similarly contributed to specific bins. As more usage of the devices occurs, the bins reflect the performance of the wireless network. The device ratings of the devices can also be updated based on the bin ratings and other data associated with the bins.

Claims

1. A method for characterizing a performance of a wireless network, the method comprising:

receiving data from multiple end user devices operating in the network, wherein the data corresponds to real world usage of the devices while operating in the network, wherein the data received from each device includes timing data and wherein the network is represented by a grid having bins;

determining a location of each device based on the timing data, wherein the location of each device is associated with one or more bins of the grid;

identifying a device rating associated with each device, the device rating indicating a previously determined device accuracy and confidence level attributed to the device; and

binning the data received from the multiple devices in the one or more bins according to the corresponding device ratings, wherein the data in the one or more bins provides a holistic view of the performance of the wireless network over time as the multiple devices contribute data to the one or more bins.

2. A method as defined in claim 1, wherein determining a location of each device based on the timing data further comprises determining an approximate location of each device.

3. A method as defined in claim 2, wherein determining an approximate location of each device further comprises identifying one or more sectors communicating with each device.

4. A method as defined in claim 1, wherein receiving data from multiple device further comprises collecting data regarding usage of each device.

5. A method as defined in claim 4, wherein collecting data regarding usage of each device includes, for each device, one or more of:

collecting data regarding a call;

collecting data regarding a dropped call;

collecting data regarding a signal to noise ratio for the device;

collecting data regarding an RSSI of the device;

collecting data regarding clock data from one or more sectors;

collecting data regarding the device, including ROM version, PRL version, and device settings; and

collecting data outside of a call collecting data relating to a signature associated with usage of the device.

6. A method as defined in claim 1, wherein identifying a device rating associated with each device further comprises identifying a device accuracy and a device confidence, wherein at least one device is one of an end-user device, a network probe, or network monitoring equipment.

7. A method as defined in claim 1, wherein binning the data received from each device in the one or more bins further comprises binning the data in each of the one or more bins based on an overlap between each of the one or more bins and the location of each device.

8. A method as defined in claim 1, wherein binning the data received from the device in the one or more bins further comprises comparing the data received from the device with a confidence rating and bin data of each of the one or more bins.

9. A method as defined in claim 1, further comprising adjusting at least one of the device rating of the devices for each of the multiple devices and adjusting a bin rating of each of the one or more bins.

10. A method as defined in claim 1, further comprising identifying one or more data points for the data received from each device, wherein the one or more data points are identified using match filters.

11. A method for characterizing a performance of a wireless network using data collected from devices operating in the wireless network, the method comprising:

collecting a data set from a set of devices operating in a wireless network, wherein the data set that includes both an approximate location of each device operating in a wireless network and metrics related to each device, wherein the data set corresponds to real world conditions;

determining a device rating for each device in the set of devices, the device rating indicating a device accuracy and confidence attributed to the devices;

contributing the metrics from each device to at least one bin of a grid that represents the wireless network based on at least the device rating of each device and a bin rating of the at least one bin, wherein the metrics are weighted according to the device rating and the bin rating; and

characterizing a performance of the wireless network based on the metrics contributed to bins in the grid from the set of devices, wherein the metrics in the bins of the grid provide a holistic view of the performance of the wireless network over time as multiple devices contribute metrics to the at least one bin.

12. A method as defined in claim 11, wherein collecting a data set that includes both an approximate location of a device operating in a wireless network and metrics related to the device further comprises receiving the metrics from each device in the set of devices, wherein the metrics from each device includes timing data from one or more sectors that communicated with each device.

13. A method as defined in claim 11, wherein the metrics of each device include one or more of RF metrics and transaction metrics.

14. A method as defined in claim 13, wherein the RF metrics include one or more of RSSI, signal to noise ratio, noise margin, signature, and sector traffic.

15. A method as defined in claim 13, wherein the transaction metrics include one or more of:

sectors that communicated with the device;

whether a call of the device was dropped;

a type of dropped call, call duration; and

timing data from each of the sectors.

16. A method as defined in claim 11, further comprising generating an ellipse that describes the approximate location of the device.

17. A method as defined in claim 11, wherein the approximate location of each device in the set of devices is associated with one or more bins of the grid.

18. A method as defined in claim 11, wherein contributing the metrics from each device to at least one bin further comprises comparing the metrics of the device with metrics already contributed to the at least one bin of the grid from other devices.

19. A method as defined in claim 18, wherein contributing the metrics from each device to at least one bin further comprises adjusting the contribution based on at least one of the device rating, which includes a device rating a confidence level, and a confidence rating of the at least one bin.

20. A method as defined in claim 11, further comprising updating the device rating based on one or more uses of the device including calls.

21. A method as defined in claim 20, further comprising:

updating each device rating based on a comparison of the contribution of the device to the at least one bin with data existing in the bin;

updating a confidence level of each device; and

updating a rating and a confidence level of the at least one bin.

22. A method as defined in claim 11, wherein each device is one of an end-user device, a network probe, or network monitoring equipment.

23. A method as defined in claim 11, wherein collecting a data set that includes both an approximate location of a device operating in a wireless network and metrics related to the device from a set of devices operating in the wireless network further comprises collecting a second data set from a second data source, wherein devices in the second data source are calibrated and have a higher initial rating than the set of devices.

24. A method for characterizing a performance of end use devices operating in a wireless network, the method comprising:

representing a wireless network as a grid, the grid having one or more bins each having a bin rating;

identifying one or more bins associated with each location of multiple devices operating in the wireless network;

determining a device rating of each device, the device rating indicating a device accuracy and confidence attributed to each device;

contributing metrics from each device to the one or more bins associated with the location of each device, wherein the metrics are weighted according to each device rating and wherein the metrics define real usage of each device operating the wireless network; and

characterizing each device based on the metrics contributed by each device and metrics existing in the one or more bins receiving the contributed metrics, wherein the one or more bins provide a holistic view of the performance of the wireless network as each device contributes metrics at least some of to the one or more bins.

25. A method as defined in claim 24, wherein identifying one or more bins associated with each location of multiple devices operating in the wireless network further comprises identifying an approximate location of a use of each device using timing data from one or more sectors used by each device.

26. A method as defined in claim 24, wherein the metrics of each device characterize an RF performance of each device for the use of the device.

27. A method as defined in claim 24, further comprising contributing metrics to the one or more bins from a second data source, the second data source including at least one of network probes or network monitoring equipment.

28. A method as defined in claim 26, wherein the metrics of each device include one or more of:

RSSI;

sectors used for the call;

type of call drop;

ROM version, PRL version, device settings;

signal to noise ratio;

signature data; and

timing data from the sectors.

29. A method as defined in claim 24, wherein determining a device rating of each device further comprises accessing a database to access the device rating of each device using an identifier of each device.

30. A method as defined in claim 24, wherein contributing metrics from the device to the one or more bins further comprises comparing the metrics with the existing metrics associated with the one or more bins.

31. A method as defined in claim 29, further comprising adjusting each device rating based on the comparison between the metrics of the device and the existing metrics associated with the one or more bins.

32. A method as defined in claim 30, further comprising:

updating a confidence level of each device based on how the metrics contributed with each device correlates with the existing metrics;

updating a confidence level of the one or more bins after receiving the metrics of each device; and

updating a rating of the one or more bins based on the existing metrics and the metrics contributed from each device.

33. A method as defined in claim 24, further comprising including device configuration in the metrics such that a view of the wireless network from a perspective of device configuration is include in the one or more bins.

34. A method as defined in claim 24, further comprising:

establishing the device rating system for the devices, the device rating system including the device rating and a confidence level of each device;

establishing a confidence rating for each of the one or more bins of the grid;

updating, each time each device contributes metrics to the grid;

the device rating of each device based on how the contributed metrics match the existing metric in the one or more bins of the grid;

the confidence level of each device based on how the contributed metrics match the existing metric in the one or more bins grid, wherein the confidence level of each device and a confidence level of the one or more bins receiving the contributed metrics are decreased if the contributed metrics do not match the existing metrics in the one or more bins of the grid;

the rating of the one or more bins, wherein the rating of the one or more bins reflects a number of devices that have contributed metrics to the one more bins; and

a confidence level of the one or more bins of the grid.