Fast OLAP for real user measurement of website performance

• US 9,772,923 B2
• Filed: 03/14/2013
• Issued: 09/26/2017
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Akamai Technologies, Inc.
• Original Assignee: Soasta Incorporated (Akamai Technologies, Inc.)
• Inventors: Tellis, Philip Santosh, Brewer, III, Vernon Archie
• Primary Examiner(s): Burgess, Glenton B
• Assistant Examiner(s): Doan, Tan
• Application Number:
  US13/830,946
• Publication Number:
  US 20140280880A1
• Time in Patent Office: 1,657 Days
• Field of Search: 709224
• CPC Class Codes: G06F 11/3082 the data filtering being achieved by aggregating or compressing the monitored data
• CPC Class Codes: G06F 11/32 with visual or acoustical indication of the functioning of the machine
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3438 monitoring of user actions tracking the activity of the user H04L67/535
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: G06F 2201/88 Monitoring involving counting
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5083 wherein the managed service relates to web hosting

Abstract

A method for real-time capture and analysis of website performance metrics includes receiving, in each of a plurality of result servers, data values transmitted from a plurality of client devices, each data value representing a measurement of a performance metric associated with an experience of a user on a web page or using a web application, the measurement being made and the data value produced being transmitted in real-time while the user is on the web page or using the web application. For each data value received, a count in one of a set of N data buckets defined in a memory is incremented. All of the counts in each of the corresponding data buckets is aggregated to produce a histogram of the performance metric is rendered on an analytic dashboard.

Claims

1. A computer-implemented method for real-time capture and analysis of data of real user experiences either accessing a website, or using a web-based application or a mobile app, via a plurality of client devices, each of the client devices being associated with a user, the computer-implemented method comprising:

receiving, in each of a plurality of collector servers, web page load time data values transmitted from the plurality of client devices, each web page load time data value representing a real user measurement (RUM) associated with an experience of the user on a web page of the website, or using the web-based application or the mobile app, the web page load time data value being transmitted while the user is on the web page, or using the web-based application or the mobile app, each of the collector servers being located in a network cloud remote to each of the client devices;

for each web page load time data value received, incrementing a count in a corresponding one of a set of N, where N is an integer greater than 1, data buckets, each data bucket being defined in a memory resource for web page load time data values falling within a predetermined range of values, the set of N data buckets being arranged in an order of web page load time data value ranges;

periodically transmitting, by each collector server, the count of each of the N data buckets to a unit over a network;

aggregating, by the unit in a final set of N data buckets, all of the counts received from each collector server for each corresponding data bucket such that an ith ,data bucket, where i is an integer ranging from 1 to N, of the final set contains a sum of all of the ith data bucket counts received from all the collector servers; and

rendering, on an analytic dashboard, a visual presentation in the form of a histogram of the aggregated counts of the web page load time data values, wherein a height of each bar of the histogram corresponds to an aggregated count of one or more of the N data buckets of the final set, the rendering being performed while the client devices are either accessing the website, or using the web-based application or the mobile app,

wherein the rendering of the visual representation is performed in real-time concurrently with the on-going collection of the web page load time data values, within seconds or milliseconds from a time that the web page load time data values are fiat received in the plurality of collector servers, such that the histogram changes nearly instantaneously in response to new web page load time data values being received in the plurality of collector servers.

2. The computer-implemented method of claim 1 further comprising the steps of:

determining a total count equal to the sum of the counts of all N data buckets in the final set;

calculating a percentile count as the product of the total count and a percentage M;

identifying on the analytic dashboard a particular data bucket of the final set as having a Mth percentile data value based on the percentile count;

approximating the Mth percentile data value as;

V_B+((R/NUM)×

I)

where V_B is a value at a bottom of the predetermined range of the particular data bucket, R is the predetermined range of the particular data bucket, NUM is the count of the particular data bucket and I is an incremental count within the particular data bucket that corresponds to the percentile count.

3. The computer-implemented method of claim 1 wherein the set of N data buckets and the final set of N data buckets are identically configured.

4. The computer-implemented method of claim 1 wherein the predetermined range is identical for all of the N data buckets.

5. The computer-implemented method of claim 1 wherein at least one of the, data buckets has a different predetermined range than that of another data bucket.

6. The computer-implemented method of claim 1 further comprising:

selecting a minimum bar width for the histogram; and

in the event that a given bar of the histogram has a width that is an integer multiple of the minimum bar width, scaling a height of the given bar to a reduced height equal to the aggregated count represented by the given bar divided by the integer multiple.

7. The computer-implemented method of claim 1 further comprising storing the final set of N data buckets in a database.

8. The computer-implemented method of claim 7 wherein the database is accessible to a computer running a program that generates the analytic dashboard.

9. The computer-implemented method of claim 1 wherein the order is an ascending order.

10. A non-transitory computer-readable storage medium encoded with computer instructions, which, when executed by one or more processors operable to:

receive, in each of a plurality of collect servers, web page load time data values transmitted from a plurality of client devices, each web page load time data value representing a real user measurement (RUM) associated with an experience of a user either browsing a website, or using a web-based application or a mobile app, the user beacon data value being transmitted in real-time while the user is on a web page, or using the web-based application or the mobile app, each of the collector servers being located in a network cloud remote to each of the client devices;

increment, for each web page load time data value received, a count in corresponding one of a set of N, where N is an integer greater than 1, data buckets, each data bucket being defined in a memory resource for web page load, time data values falling within a predetermined range of values, the set of N data buckets being arranged in an order of web page load time data value ranges;

transmit, by each collector server, the count of each of the N data buckets to a unit over a network;

aggregate, by the unit in a final set of N data buckets, all of the counts received from each collector server for each corresponding data bucket such that an ith data bucket, where i is an integer ranging from 1 to N. of the final set contains a sum of all of the ith data buckets counts received from all of the collector servers; and

render, on an analytic dashboard, a visual presentation in the form of a histogram of the aggregated counts of the web page load time data values, wherein a height of each bar of the histogram corresponds to an aggregated count of one or more of the N data buckets of the final set, the rendering being performed in real-time while the client devices are either accessing the website, or using the web- based application or the mobile app,

wherein the visual representation is rendered in real-time concurrently with the on-going collection of the web page load time data values, within seconds or milliseconds from a time that the web page load time data values are first received in the plurality of collector servers, such that the histogram changes nearly instantaneously in response to new web page load time data values being received in the plurality of collector servers.

11. The non-transitory computer-readable storage medium of claim 10 wherein execution of the computer instructions is further operable to:

determine a total count equal to the sum of the counts of all N data buckets in the final set;

calculate a percentile count as the product of the total count and a percentage M;

visually identify on the analytic dashboard a particular data bucket of the final set as having a Mth percentile data value based on the percentile count;

approximate the Mth percentile data value as;

V_B+((R/NUM)×

I)

where V_B is a value at a bottom of the predetermined range of the particular data bucket, R is the predetermined range of the particular data bucket, NUM is the count of the particular data bucket and I is an incremental count within the particular data bucket that corresponds to the percentile count.

12. The non-transitory computer-readable storage medium of claim 10 wherein the set of N data buckets and the final set of N data buckets are identically configured.

13. The non-transitory computer-readable storage medium of claim 10 wherein the predetermined range is identical for all of the N data buckets.

14. The non-transitory computer-readable storage medium of claim 10 wherein at least one of the data buckets has a different predetermined range than that of another data bucket.

15. The non-transitory computer-readable storage medium of claim 10 wherein execution of the computer instructions is further operable to store the final set of N data buckets in a database accessible to a computer running a program that generates the analytic dashboard.