Multi-tiered system for receiving and reporting web site traffic data

• US 8,606,907 B1
• Filed: 07/20/2011
• Issued: 12/10/2013
• Est. Priority Date: 07/20/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Wogulis, James Lee, Deshpande, Mayur Venktesh, Matthews, Jacob Burton, Marifet, Kasem
• Primary Examiner(s): Gee, Jason K.
• Assistant Examiner(s): Tsang, Henry
• Application Number:
  US13/187,452
• Time in Patent Office: 874 Days
• Field of Search: None
• US Class Current: 709/224
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

Configurations of a multi-tiered system for receiving and reporting web site traffic data with low latency are described. The multi-tiered system can include a tier of collecting servers for receiving incoming event tracking data communications (e.g., HTTP requests for logging web page requests at corresponding web sites). A collecting server receiving an incoming communication determines a particular logging server in a lower tier to route the incoming communication. Upon receiving the incoming communication, the logging server processes the communication by 1) logging the communication to persistent storage, and 2) routing the communication to i) a tier of long-term storage servers and ii) a tier of short term storage and low-latency servers (i.e., real-time servers) for aggregating and reporting web site traffic data. Each real-time server can be configured to aggregate the web site tracking data for reporting to a publisher with low latency.

Claims

1. A multi-tiered system for receiving and reporting web site traffic data with low latency, the system comprising:

a collecting server configured to receive and forward an event tracking data communication, wherein the event tracking data communication includes event tracking data for tracking an activity associated with a source of the event tracking data communication;

a logging server configured to receive the event tracking data communication from the collecting server; and

a real-time server configured to receive one or more event tracking data communications for temporary storage,

wherein the logging server is further configured to route the received event tracking data communication to both a long-term storage server for persistent storage and the real-time server for temporary storage, and

wherein the real-time server is configured to process the event tracking data for reporting to a publisher with low latency, determine whether the event tracking data can be processed within a specified period of time to meet a real-time constraint, and discard the event tracking data in response to determining that the event tracking data will fail to be processed within the specified period of time,

wherein, for a same predetermined time period, the logging server routes one or more event tracking data communications to the real-time server at a greater frequency than routing one or more event tracking data communications to the long-term storage server, and

for the same predetermined time period, an aggregate number of event tracking data communications routed to the long-term storage server is greater than an aggregate number of event tracking data communications routed to the real-time server.

2. The system of claim 1, wherein the event tracking data includes web site traffic data included in a Hypertext Transfer Protocol (HTTP) request associated with a web site.

3. The system of claim 1, further comprising a front-end server configured to present the event tracking data with low latency.

4. The system of claim 1, wherein the logging server is further configured to aggregate a respective plurality of event tracking data communications from a respective plurality of associated web sites.

5. The system of claim 1, wherein the logging server is further configured to verify routing to the long-term storage server to guarantee delivery of the event tracking data communication.

6. The system of claim 1, wherein the received event tracking data communication is replicated across a second long-term storage server.

7. The system of claim 1, wherein the event tracking data communication comprises a Hypertext Transfer Protocol (HTTP) request.

8. The system of claim 7, wherein the HTTP request includes randomized visitor identity data and geographical location data associated with the visitor.

9. The system of claim 1, wherein the real-time server is configured to determine an associated owner of the event tracking data communication.

10. The system of claim 1, wherein the real-time server stores real-time analytics data based on the event tracking data communication.

11. A machine-implemented method for receiving and reporting web site traffic data with low latency, the method comprising:

receiving an event tracking data communication, wherein the event tracking data communication includes event tracking data for tracking an activity associated with a source of the event tracking data communication;

routing the event tracking data communication to a long-term storage server for persistent storage;

selecting a real-time server for routing the event tracking data communication; and

routing the event tracking data communication to the selected real-time server for temporary storage, wherein the selected real-time server is configured to process the event tracking data for reporting to a publisher, determine whether the event tracking data can be processed within a specified period of time to meet a real-time constraint, and discard the event tracking data in response to determining that the event tracking data will fail to be processed within the specified period of time,

wherein, for a same predetermined time period, routing one or more event tracking data communications to the selected real-time server occurs at a greater frequency than routing one or more event tracking data communications to the long-term storage server, and

for the same predetermined time period, an aggregate number of event tracking data communications routed to the long-term storage server is greater than an aggregate number of event tracking data communications routed to the selected real-time server.

12. The method of claim 11, wherein the event tracking data includes web site traffic data included in a Hypertext Transfer Protocol (HTTP) request associated with a web site.

13. The method of claim 11, wherein selecting the real-time server comprises an arbitrary selection among a plurality of real-time servers.

14. The method of claim 11, wherein the routing the event tracking data communication to the long-term storage server further comprises verifying the routing to guarantee delivery of the event tracking data communication.

15. A machine-implemented method for receiving and storing real-time web site traffic data with low latency at a real-time server, the method comprising:

receiving, at a first real-time server, an event tracking data communication, wherein the event tracking data communication includes event tracking data for tracking an activity associated with a source of the event tracking data communication;

determining, at the first real-time server, at least one real-time server associated with the received event tracking data communication, wherein the at least one real-time server is configured to process the event tracking data within a specified period of time to meet a real-time constraint for processing the event tracking data, determine whether the event tracking data can be processed within the specified period of time to meet a real-time constraint, and discard the event tracking data in response to determining that the event tracking data will fail to be processed within the specified period of time;

storing, at the first real-time server, real-time analytics data based on the received event tracking data communication in response to determining that the first real-time server is associated with the received event tracking data communication; and

routing the event tracking data communication to a second real-time server in response to determining that the second real-time server, instead of the first real-time server, is associated with the received event tracking data communication, wherein the first and second real-time servers are both included in a group of real-time servers,

wherein, for a same predetermined time period, routing one or more event tracking data communications to the second real-time server occurs at a greater frequency than routing one or more event tracking data communications to a long-term storage server, and

for the same predetermined time period, an aggregate number of event tracking data communications routed to the long-term storage server is greater than an aggregate number of event tracking data communications routed to the second real-time server.

16. The method of claim 15, wherein the event tracking data includes web site traffic data included in a Hypertext Transfer Protocol (HTTP) request associated with a web site.

17. The method of claim 15, wherein the event tracking data communication is received from a logging server based on a random selection.

18. The method of claim 15, wherein storing the real-time analytics data comprises storing to non-persistent storage.

19. The method of claim 18, wherein the non-persistent storage comprises volatile memory.

20. A non-transitory machine-readable medium comprising instructions stored therein, which when executed by a machine, cause the machine to perform operations comprising:

receiving one or more event tracking data communications, wherein the event tracking data communication includes event tracking data for tracking an activity associated with a source of the event tracking data communication;

routing the one or more event tracking data communications to a long-term storage server for persistent storage, wherein the one or more event tracking data communications are stored in non-volatile storage on the long-term storage server;

selecting a real-time server among a plurality of real-time servers for routing the one or more event tracking data communications, wherein the real-time server is arbitrarily selected; and

routing the one or more event tracking data communications to the selected real-time server for temporary storage, wherein the selected real-time server is configured to process the event tracking data for reporting to a publisher, determine whether the event tracking data can be processed within a specified period of time to meet a real-time constraint, and discard the event tracking data in response to determining that the event tracking data will fail to be processed within the specified period of time,

wherein, for a same predetermined time period, routing one or more event tracking data communications to the selected real-time server occurs at a greater frequency than routing one or more event tracking data communications to the long-term storage server, and

for the same predetermined time period, an aggregate number of event tracking data communications routed to the long-term storage server is greater than an aggregate number of event tracking data communications routed to the selected real-time server.

21. The non-transitory machine-readable medium of claim 20, wherein the event tracking data includes web site traffic data included in a Hypertext Transfer Protocol (HTTP) request associated with a web site.

22. The non-transitory machine-readable medium of claim 20, wherein the one or more event tracking data communications are received from a collecting server.