Real time web usage reporter using RAM

• US 8,396,834 B2
• Filed: 10/11/2006
• Issued: 03/12/2013
• Est. Priority Date: 10/10/2006
• Status: Active Grant

• Current Assignee: Acoustic, L.P.
• Original Assignee: International Business Machines Corporation
• Inventors: Bahadori, Hamid, Puttaswamy, Hemanth
• Primary Examiner(s): Ng, Amy
• Application Number:
  US11/546,923
• Publication Number:
  US 20080086454A1
• Time in Patent Office: 2,344 Days
• Field of Search: 707/1, 707/200, 707/201, 707/202, 707/204, 707/640, 707/674, 711/126
• US Class Current: 707/640
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

A system that improves the real time presentation of web analytics data to clients. Real time data is improved in one embodiment by using a messaging system that directs the data stream to RAM memory, before it is sent to the database. A separate path is provided for presenting the real time data, which does not require going through the main database. The data is serialized and streamed in batches at an interval which can be set from the order of seconds to minutes. Additionally, the data is aggregated in RAM according to the desired groupings of data, so that the aggregation is pre-computed.

Claims

1. A method for tracking web usage data, comprising:

collecting, in a server system, data corresponding to monitored actions on at least one web site for a predetermined period of time, wherein each of the at least one web sites is identified by a client ID;

pre-processing said data to perform a first level of aggregation indexed by at least the client ID, said first level of aggregation including current sessions, completed sessions, and a snapshot of current activity, for the predetermined period of time, and calculating a plurality of metrics for said first level of aggregation;

streaming said data to a RAM server;

aggregating said data into multiple aggregate groups desired for display, said aggregate groups including the first level aggregation, further including a second level aggregation within each client ID for a plurality of time periods, further including a third level of aggregation within each time period, the third level of aggregation including current session data and complete session data, wherein said aggregating said data into multiple aggregate groups occurs prior to a request to display said data;

storing said aggregated data in a hierarchical structure in a RAM in said server system;

calculating the plurality of metrics for each of the multiple aggregate groups;

repeating the steps of collecting, pre-processing, streaming, aggregating, storing, and calculating; and

providing said plurality of metrics from said RAM to a client at a client computer.

2. The method of claim 1 wherein said data is indexed by at least a client ID and a time period aggregation.

3. The method of claim 1 further comprising pre-processing said data upstream of said RAM to calculate a plurality of metrics and to perform a first level of aggregation.

4. The method of claim 1 wherein a portion of said data in said RAM is provided to said client at intervals of ten minutes or less.

5. The method of claim 1 further comprising:

receiving additional data from at least one of an RSS feed, web services and alerts; and

storing said additional data in said RAM.

6. The method of claim 1 further comprising:

providing a plurality of report modules on said client computer;

asynchronously retrieving data from said RAM for a plurality of said report modules.

8. The method of claim 1 further comprising:

storing said data in said RAM in a flat file structure in local nonvolatile memory; and

re-loading said RAM from said nonvolatile memory in the event of data loss in said RAM.

9. The method of claim 8 further comprising:

re-aggregating a first level of aggregation upstream of said RAM and resending re-aggregated data and current data to said RAM.

7. A method for tracking web usage data, comprising:

collecting, in a server system, data corresponding to monitored actions on a web site;

aggregating said data into aggregate groups desired for display;

storing said aggregated data in a hierarchical structure in a RAM in said server system;

providing said data from said RAM to a client at a client computer; and

mirroring said data in said RAM in a fail-over RAM.

10. A method for tracking web usage data, comprising:

collecting, in a server system, data corresponding to monitored actions on at least one web site, wherein each of the at least one web sites is identified by a client ID;

pre-processing said data to calculate a plurality of metrics and to perform a first level of aggregation, and indexing said data by at least the client ID;

streaming said data to a RAM server;

aggregating said data into multiple aggregate groups desired for display, wherein said aggregating occurs prior to a request to display said data;

storing said aggregated data in a hierarchical structure in a RAM in said RAM server system;

calculating a plurality of metrics for the multiple aggregate groups;

storing the plurality of metrics in the RAM;

providing a plurality of report modules on a client computer; and

asynchronously retrieving said plurality of metrics from said RAM for a plurality of said report modules.

11. The method of claim 10 further comprising:

receiving additional data from at least one of an RSS feed, web services and alerts; and

storing said additional data in said RAM.

12. A system for tracking web usage data, comprising:

a collector server configured to collect data corresponding to monitored actions on at least one web site, wherein each of the at least one web sites is identified by a client ID;

a RAM server including an aggregation service to aggregate said data into aggregate groups desired for display, wherein said aggregation occurs prior to a request to display said data, wherein said data is indexed by at least the client ID and a time period aggregation;

a RAM for storing said aggregated data in a hierarchical structure; and

a remote client computer for displaying said data from said RAM.

13. The system of claim 12 further comprising:

a transformer server coupled upstream from said RAM to pre-process said data to calculate a plurality of metrics and to perform a first level of aggregation.

14. The system of claim 13 wherein said transformer further organizes said data by client, wherein said client is at least one website.

15. The system of claim 12 wherein said first level of aggregation comprises a time interval of ten minutes or less.

16. The system of claim 12 further comprising:

an input port for at least one of an RSS feed, web services and alerts, for receiving additional data; and

said RAM server being configured to store said additional data in said RAM.

17. The system of claim 12 further comprising:

a plurality of report modules on said client computer, said report modules being configured to asynchronously retrieve data from said RAM for a plurality of said report modules.

18. The system of claim 12 further comprising:

a fail-over RAM configured to mirror said data in said RAM.

19. The system of claim 12 further comprising:

a local nonvolatile memory;

said RAM server being configured to store said data in said RAM in a flat file structure in said local nonvolatile memory, and to re-load said RAM from said nonvolatile memory in the event of data loss in said RAM.

20. The system of claim 19 further comprising:

a transformer server coupled upstream from said RAM to pre-process said data to calculate a plurality of metrics and to perform a first level of aggregation, said transformer server being configured to re-aggregate a first level of aggregation upstream of said RAM and resending re-aggregated data and current data to said RAM.

21. A system for tracking web usage data, comprising:

a collector server configured to collect data corresponding to monitored actions on at least one web site, wherein each of the at least one web sites is identified by a client ID;

a transformer server in communication with said collector server to pre-process said data to calculate a plurality of metrics and to perform a first level of aggregation, said first level of aggregation indexed by at least said client ID;

a messaging system in communication with said transformer for transmitting processed data from said transformer;

a RAM server in communication with said messaging system and including an aggregation service to aggregate said data into multiple aggregate groups, said aggregate groups including the first level aggregation, further including a second level aggregation for a plurality of time periods, further including a third level of aggregation within each time period, the third level of aggregation including current session data and complete session data desired for display, wherein said aggregating said data into multiple aggregate groups occurs prior to a request to display said data;

a RAM coupled to said RAM server for storing said aggregated data in a hierarchical structure;

a remote client computer for displaying said data from said RAM; and

a plurality of report modules on said client computer, said report modules being configured to asynchronously retrieve data from said RAM for a plurality of said report modules.