Real-time analytics of web performance using actual user measurements

• US 9,491,248 B2
• Filed: 04/10/2015
• Issued: 11/08/2016
• Est. Priority Date: 07/19/2010
• Status: Active Grant

• Current Assignee: Akamai Technologies, Inc.
• Original Assignee: Soasta Incorporated (Akamai Technologies, Inc.)
• Inventors: Hemmert, Michael, Broda, Tal, Solnit, Matthew, Tellis, Philip Santosh
• Primary Examiner(s): Nguyen, Maikhanh
• Application Number:
  US14/683,805
• Publication Number:
  US 20150222720A1
• Time in Patent Office: 578 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3457 Performance evaluation by simulation
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/06 Generation of reports
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 43/10 Active monitoring, e.g. heartbeat, ping or trace-route
• CPC Class Codes: H04L 43/50 Testing arrangements
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

A method for real-time capture of actual user experiences on a website, web application or mobile app includes receiving, in first servers, one or more beacons, each containing data items. Each beacon is generated in real-time from a user session. The data items are aggregated in one or more first sets of N data buckets associated with each of the first servers. Each of the first servers transmits the data items of each of the N data buckets to a second server over a network, which further aggregates the data items received in one or more sets of N data buckets corresponding to and identically configured as the data buckets of the first servers. The corresponding data contents of the one or more second sets of N data buckets is aggregated into one or more final sets of N data buckets used in generating a real-time analytic dashboard.

Claims

1. A method for real-time capture of actual user experiences on a website, web application or mobile app, comprising:

receiving, in each of a plurality of first servers, one or more beacons, each beacon being generated in real-time from a user session on the website, web application or mobile app, each beacon comprising one or more data items which includes a performance metric representative of an experience of a user on the website, web application or mobile app;

aggregating, in each of the first servers, the one or more data items in one or more first sets of N, where N is an integer greater than 1, data buckets, each of the one or more data items being aggregated in accordance with a value of the performance metric, each data bucket being defined in a memory for data values falling within a predetermined range, each of the one or more first sets of N data buckets being arranged in an order of data value ranges;

transmitting, by each of the first servers, the one or more data items of each of the N data buckets in each of the one or more first sets to an associated one of a plurality of second servers;

aggregating, by each of the second servers, the one or more data items of the one or more first sets of N data buckets received from each of the first servers in one or more second sets of N data buckets;

transmitting, by each of the second servers, the one or more data items of each of the one or more second sets of N data buckets to a computing instance;

aggregating, by the computing instance, the corresponding data contents of the one or more second sets of N data buckets in one or more final sets of N data buckets; and

generating, on a real-time analytic dashboard, one or more visual indicators representative of at least one aggregated data result computed from the data contents of the one or more final sets of N data buckets.

2. The method of claim 1 wherein each second server is associated with a different subset of the first servers.

3. The computer-implemented method of claim 1 wherein each visualization has a size that corresponds to the total number of users at the distinct one of the geographical locations.

4. The method of claim 3 wherein the predefined subset of metrics include operating system (O/S) type.

5. The method of claim 3 wherein the predefined subset of metrics include browser type.

6. The computer-implemented method of claim 1 wherein the geographical locations include one or more countries.

7. The computer-implemented method of claim 1 wherein each visualization has one of a plurality of colors, each color corresponding to a distinct range of the statistical result.

8. The method of claim 1 further comprising storing the one or more final sets of N data buckets in a database accessible to a computing device that generates the real-time analytic dashboard.

9. The method of claim 1 wherein the one or more visual indicators includes a median page load time.

10. The method of claim 1 wherein the one or more visual indicators includes a histogram based on a total number of beacons received by the first servers, where each bar of the histogram corresponds to a count in a unique one of the data buckets in the one or more final sets of N data buckets.

11. A non-transitory computer-readable storage medium encoded with computer instructions, which, when executed by one or more processors, operable to:

receive, in each of a plurality of first servers, one or more beacons, each beacon being generated in real-time from a user session on the website, web application or mobile app, each beacon comprising one or more data items which includes a performance metric representative of an experience of a user on the website, web application or mobile app;

aggregate, in each of the first servers, the one or more data items in one or more first sets of N, where N is an integer greater than 1, data buckets, each of the one or more data items being aggregated in accordance with a value of the performance metric, each data bucket being defined in a memory for data values falling within a predetermined range, each of the one or more first sets of N data buckets being arranged in an order of data value ranges;

transmit, by each of the first servers, the one or more data items of each of the N data buckets in each of the one or more first sets to an associated one of a plurality of second servers;

aggregate, by each of the second servers, the one or more data items of the one or more first sets of N data buckets received from each of the first servers in one or more second sets of N data buckets;

transmit, by each of the second servers, the one or more data items of each of the one or more second sets of N data buckets to a computing instance;

aggregate, by the computing instance, the corresponding data contents of the one or more second sets of N data buckets in one or more final sets of N data buckets; and

generate, on a real-time analytic dashboard, one or more visual indicators representative of at least one aggregated data result computed from the data contents of the one or more final sets of N data buckets.

12. The non-transitory computer-readable storage medium of claim 11 wherein each second server being associated with a different subset of the first servers.

13. The non-transitory computer-readable storage medium of claim 11 wherein execution of the computer instructions is further operable to expand, according to a template, data contents of the one or more final sets of N data buckets to a larger set of data, the template being based on a predefined subset of metrics and combinations thereof.

14. The non-transitory computer-readable storage medium of claim 13 wherein the predefined subset of metrics include a country.

15. The non-transitory computer-readable storage medium of claim 14 wherein the predefined subset of metrics further include a region of the country.

16. The non-transitory computer-readable storage medium of claim 13 wherein the predefined subset of metrics include operating system (O/S).

17. The non-transitory computer-readable storage medium of claim 11 wherein execution of the computer instructions is further operable to store the one or more final sets of N data buckets in a database accessible to a computing device that generates the real-time analytic dashboard.

18. The non-transitory computer-readable storage medium of claim 11 wherein the one or more visual indicators includes a median page load time.

19. The non-transitory computer-readable storage medium of claim 11 wherein the one or more visual indicators includes a histogram based on a total number of beacons received by the first servers, where each bar of the histogram corresponds to a count in a unique one of the data buckets in the one or more final sets of N data buckets.

20. A system comprising:

a plurality of first servers configured to receive one or more beacons, each beacon being generated in real-time from a user session on the website, web application or mobile app, each beacon comprising one or more data items which includes a performance metric representative of an experience of a user on the website, web application or mobile app, each of the first servers being further configured to aggregate the one or more data items in one or more first sets of N, where N is an integer greater than 1, data buckets, each of the one or more data items being aggregated in accordance with a value of the performance metric received with each of the one or more beacons, each data bucket being defined in a memory for data values falling within a predetermined range, each of the one or more first sets of N data buckets being arranged in an order of data value ranges, each of the first servers being further configured to periodically transmit the one or more data items of each of the N data buckets over a first network;

a plurality of second servers, each second server being associated with a different subset of the first servers, each second server being configured to aggregate the one or more data items of the one or more first sets of N data buckets received from each of the first servers in one or more second sets of N data buckets, each of the second servers being further configured to transmit the one or more data items of each of the second sets of N data buckets over a second network; and

a computing instance configured to aggregate the corresponding data contents of the one or more second sets of N data buckets in one or more final sets of N data buckets, the computing instance being further configured to execute a program that generates, on a real-time analytic dashboard, one or more visual indicators representative of at least one aggregated data result computed from the data contents of the one or more final sets of N data buckets.

21. The system of claim 20 wherein each second server is associated with a different subset of the first servers.

22. The system of claim 20 wherein the computing instance being further configured to expand, according to a template, data contents of the one or more final sets of N data buckets to a larger set of data, the template being based on a predefined subset of metrics and combinations thereof.

23. The system of claim 22 wherein the predefined subset of metrics include a country.

24. The system of claim 22 wherein the predefined subset of metrics further include a region of the country.

25. The system of claim 22 wherein the predefined subset of metrics include operating system (O/S).

26. The system of claim 20 wherein the one or more visual indicators includes a median page load time.

27. The system of claim 20 wherein the one or more visual indicators includes a histogram based on a total number of beacons received by the first servers, where each bar of the histogram corresponds to a count in a unique one of the data buckets in the one or more final sets of N data buckets.