System and method for aggregating analytics data

• US 9,305,105 B2
• Filed: 05/25/2010
• Issued: 04/05/2016
• Est. Priority Date: 05/26/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Nandy, Sagnik, Ben-Ayoun, Naomi
• Primary Examiner(s): Sall, El Hadji
• Application Number:
  US12/787,320
• Publication Number:
  US 20100312884A1
• Time in Patent Office: 2,142 Days
• Field of Search: 709201-226, 709/200, 707 1- 50, 707100-110, 707/802
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

A method and system for aggregating analytics data is discussed. The system differentiates between analytics data that is context sensitive and therefore cannot be reliably updated incrementally (e.g., unique page views, time on site, etc.) and analytics data that is not context sensitive. The system aggregates the context insensitive metrics and dimensions incrementally, while aggregating the context sensitive metrics and dimensions after a specified time duration, such as at the end of the day. It is estimated that less than 10% of all metrics and dimensions are context sensitive. In this way, the aggregator server 160 is able to improve the freshness of more than 90% of the analytics data (represented by the context insensitive metrics and dimensions) to a shorter period of time than the prior art. Further, this reduces the possibility of over-counting metrics.

Claims

1. A method for aggregating analytics data, performed at a server system with one or more processors and memory storing one or more programs for execution by the one or more processors, the one or more programs including instructions for:

identifying a plurality of context sensitive analytics data characteristics and a plurality of context insensitive analytics data characteristics, wherein the context insensitive analytics data characteristics are capable of being incrementally updated with definitively known values, and the context sensitive analytics data characteristics are not capable of being incrementally updated with definitively known values;

selecting a time period;

processing server hit data at the end of the selected time period to aggregate context sensitive analytics data, and not processing the server hit data to aggregate context sensitive analytics data during the selected time period, wherein the context sensitive analytics data correspond to the context sensitive analytics data characteristics, and the context insensitive analytics data correspond to the context insensitive analytics data characteristics; and

processing the server hit data incrementally during the selected time period to aggregate context insensitive analytics data.

2. The method of claim 1, wherein the incremental processing of server hit data to aggregate context insensitive analytics data includes processing of server hit data that is created during the selected time period.

3. The method of claim 1, further comprising:

generating the server hit data including accessing server log files, processing the server log files, and storing the server hit data in a database of server hit data.

4. The method of claim 3, further comprising:

storing a time at which the server log files are accessed so as to determine, for a successive iteration of accessing the server log files, which portions of the log files are previously unread.

5. The method of claim 1, wherein aggregating the context insensitive analytics data comprises:

generating a fingerprint for each hit session that is processed to aggregate the context insensitive analytics data.

6. The method of claim 5, wherein a respective fingerprint for a hit session includes a session identifier associated with the hit session, a timestamp for a first server hit in the hit session, and one or more delta values, each delta value representing a difference between a timestamp for a later server hit that is processed to aggregate the context insensitive analytics data and the timestamp for the first server hit in the hit session.

7. The method of claim 6, wherein aggregating the context insensitive analytics data further comprises:

determining that a second server hit is duplicative or redundant by comparing a timestamp of the second server hit and determining that a delta value representing a difference between a timestamp for the second server hit and the timestamp for the first server hit in the hit session already exists in the fingerprint for the associated hit session; and

ignoring the second server hit that is determined to be duplicative or redundant.

8. The method of claim 1, wherein:

processing the server hit data at the end of the selected time period includes aggregating the context insensitive analytics data.

9. The method of claim 1, wherein the time period is one day.

10. An analytics data aggregation system, comprising:

one or more processors;

memory; and

one or more programs stored in the memory, the one or more programs comprising instructions for;

identifying a plurality of context sensitive analytics data characteristics and a plurality of context insensitive analytics data characteristics, wherein the context insensitive analytics data characteristics are capable of being incrementally updated with definitively known values, and the context sensitive analytics data characteristics are not capable of being incrementally updated with definitively known values;

selecting a time period;

processing server hit data at the end of the selected time period to aggregate context sensitive analytics data, and not processing the server hit data to aggregate context sensitive analytics data during the selected time period, wherein the context sensitive analytics data correspond to the context sensitive analytics data characteristics, and the context insensitive analytics data correspond to the context insensitive analytics data characteristics; and

processing the server hit data incrementally during the selected time period to aggregate context insensitive analytics data.

11. The data aggregation system of claim 10, wherein the instructions for incremental processing of server hit data to aggregate context insensitive analytics data further include instructions for processing of server hit data that is created during the selected time period.

12. The data aggregation system of claim 10, wherein the one or more programs further comprise instructions for:

generating the server hit data including accessing server log files, processing the server log files, and storing the server hit data in a database of server hit data.

13. The data aggregation system of claim 12, wherein the one or more programs further comprise instructions for:

storing a time at which the server log files are accessed so as to determine, for a successive iteration of accessing the server log files, which portions of the log files are previously unread.

14. The data aggregation system of claim 10, wherein instructions for aggregating the context insensitive analytics data further include instructions for:

generating a fingerprint for each hit session that is processed to aggregate the context insensitive analytics data.

15. The data aggregation system of claim 14, wherein a respective fingerprint for a hit session includes a session identifier associated with the hit session, a timestamp for a first server hit in the hit session, and one or more delta values, each delta value representing a difference between a timestamp for a later server hit that is processed to aggregate the context insensitive analytics data and the timestamp for the first server hit in the hit session.

16. The data aggregation system of claim 15, wherein the instructions for aggregating the context insensitive analytics data further comprise instructions for:

determining that a second server hit is duplicative or redundant by comparing a timestamp of the second server hit and determining that a delta value representing a difference between a timestamp for the second server hit and the timestamp for the first server hit in the hit session already exists in the fingerprint for the associated hit session; and

ignoring the second server hit that is determined to be duplicative or redundant.

17. The data aggregation system of claim 10, wherein the instructions for processing the server hit data at the end of the selected time period include instructions for aggregating the context insensitive analytics data.

18. The data aggregation system of claim 10, wherein the time period is one day.

19. A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising instructions to:

identify a plurality of context sensitive analytics data characteristics and a plurality of context insensitive analytics data characteristics, wherein the context insensitive analytics data characteristics is capable of being incrementally updated with definitively known values, and the context sensitive analytics data characteristics are not capable of being incrementally updated with definitively known values;

select a time period;

process server hit data at the end of the selected time period to aggregate context sensitive analytics data, and not processing the server hit data to aggregate context sensitive analytics data during the selected time period, wherein the context sensitive analytics data correspond to the context sensitive analytics data characteristics, and the context insensitive analytics data correspond to the context insensitive analytics data characteristics; and

process the server hit data incrementally during the selected time period to aggregate context insensitive analytics data.

20. The computer readable storage medium of claim 19, wherein the instructions to incrementally process server hit data to aggregate context insensitive analytics data further include instructions to process server hit data that is created during the selected time period.

21. The computer readable storage medium of claim 19, wherein the one or more programs further comprise instructions to:

generate the server hit data including accessing server log files, process the server log files, and store the server hit data in a database of server hit data.

22. The computer readable storage medium of claim 21, wherein the one or more programs further comprise instructions to:

store a time at which the server log files are accessed so as to determine, for a successive iteration of accessing the server log files, which portions of the log tiles are previously unread.

23. The computer readable storage medium of claim 19, wherein the instructions to aggregate the context insensitive analytics data further comprise include instructions to:

generate a fingerprint for each hit session that is processed to aggregate the context insensitive analytics data.

24. The computer readable storage medium of claim 23, wherein a respective fingerprint for a hit session includes a session identifier associated with the hit session, a timestamp for a first server hit in the hit session, and one or more delta values, each delta value representing a difference between a timestamp for a later server hit that is processed to aggregate the context insensitive analytics data and the timestamp for the first server hit in the hit session.

25. The computer readable storage medium of claim 24, wherein the instructions to aggregate the context insensitive analytics data further comprise include instructions to:

determine that a second server hit is duplicative or redundant by comparing a timestamp of the second server hit and determining that a delta value representing a difference between a timestamp for the second server hit and the timestamp for the first server hit in the hit session already exists in the fingerprint for the associated hit session; and

ignore the second server hit that is determined to be duplicative or redundant.