Pushed based real-time analytics system

• US 10,021,202 B1
• Filed: 07/17/2014
• Issued: 07/10/2018
• Est. Priority Date: 07/20/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Deshpande, Mayur Venktesh, Matthews, Jacob Burton, Marifet, Kasem, Wogulis, James Lee
• Primary Examiner(s): Blair, Douglas
• Assistant Examiner(s): Cousins, Joseph M
• Application Number:
  US14/334,187
• Time in Patent Office: 1,454 Days
• CPC Class Codes: G06F 11/3048 where the topology of the computing system or computing system component explicitly influences the monitoring activity, e.g. serial, hierarchical systems
• CPC Class Codes: G06F 11/3072 where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 16/2343 Locking methods, e.g. distributed locking or locking implementation details
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/835 Timestamp
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/286 Time to live
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display

Abstract

Configurations for providing a pushed based real-time analytics system are disclosed. The system incorporates a network and distributed database configured to stream real-time analytics data through multiple tiers. The real-time analytics data is routed/pushed through the system without requiring an additional requests from a front-end client. For instance, after the front-end client initiates a query with the real-time analytics system, a pipeline is opened to the front-end client and web site traffic data is pushed without requiring polling or other explicit requests from the front-end client. As incoming web site tracking data communications are received, the incoming web site tracking data communications are automatically routed and pushed through different tiers of the system. Real-time web site tracking data is automatically presented to the front-end client without requiring constant refreshing (which introduces latency) of a web browser or similar client application.

Claims

1. A method for routing an event tracking data comprising:

collecting from a website, at a collecting tier, an event tracking data for the website, wherein the event tracking data comprises visitor data that is obtained from a user device in response to actions performed on the user device with respect to the website loaded in a browser of the user device;

pushing, from the collecting tier, the event tracking data to a logging tier based on a predetermined deadline;

selecting, at a logging server of the logging tier, a first real-time server of a real-time tier;

pushing, from the logging server, the event tracking data to the first real-time server based on the predetermined deadline, the first real-time server comprising a data structure indicating whether the first real-time server or a second real-time server of the real-time tier is assigned to process the event tracking data for the website;

determining, at the first real-time server, whether the second real-time server of the real time tier is assigned to process the event tracking data for the website based on the data structure, wherein being assigned to process the event tracking data for the website comprises being assigned to generate analytics data based on the visitor data of the event tracking data and report the analytics data to a front-end server;

comparing, at the first real-time server, a threshold time period with an elapsed time of the event tracking data;

pushing, in response to if the comparison showing the elapsed time as less than the threshold time and in response to determining that the second real-time server is assigned to process the event tracking data for the website, the event tracking data from the first real-time server to the second real-time server based on the predetermined deadline;

or

discarding, if the comparison shows the elapsed time greater than the threshold time, the pushed event tracking data.

2. The method of claim 1, wherein determining that the second real-time server is assigned to process the event tracking data comprises:

determining a tracking account associated with the event tracking data; and

determining that the second real-time server is associated with the tracking account.

3. The method of claim 1, wherein determining that the second real-time server is assigned to process the event tracking data comprises:

determining a web property identity associated with the event tracking data;

determining a tracking account associated with the web property identity; and

determining that the second real-time server is associated with the tracking account.

4. The method of claim 1, further comprising

calculating the elapsed time based on when the first real-time server received the event tracking data.

5. The method of claim 1, further comprising

calculating the elapsed time based on a timestamp associated with the event tracking data.

6. The method of claim 1, wherein pushing the event tracking data to the second real-time server further comprises

pushing the event tracking data without a request or polling.

7. The method of claim 1, wherein the predetermined deadline comprises a specified period of time by which the event tracking data must be processed at each respective tier.

8. The method of claim 1, wherein selecting a first real-time server comprises

randomly selecting, at a logging server of the logging tier, a real-time server of a real-time tier.

9. The method of claim 1, wherein selecting a first real-time server comprises

selecting, at a logging server of the logging tier, a real-time server of a real-time tier based on load-balancing requirements among a plurality of real-time servers of the real-time tier.

10. The method of claim 1, further comprising:

receiving, at the first real-time server of the real-timer tier, a second event tracking data;

determining, at the first real-time server, that the first real-time server is acting as an owner of the second event tracking data; and

processing, at the first real-time server, the second event tracking data.

11. A system for routing an event tracking data, comprising:

a collecting tier, in connection with a network and a logging tier, the collecting tier comprising a collecting server including a processor executing the steps of;

collecting, from a website, an event tracking data for the website wherein the event tracking data comprises visitor data that is obtained from a user device in response to actions performed on the user device with respect to the website loaded in a browser of the user device;

pushing the event tracking data to a logging tier based on a predetermined deadline;

the logging tier, in connection with the collection tier and a real-time tier, the logging tier comprising a logging server including a processor executing the steps of;

selecting a first real-time server of the real-time tier;

pushing the event tracking data to the first real-time server based on the predetermined deadline, the first real-time server comprising a data structure indicating whether the first real-time server or a second real-time server of the real-time tier is assigned to process the event tracking data for the website;

the real-time tier, in connection with the logging tier, the real-time tier comprising the first and the second real-time servers, the first real-time server including a processor executing the steps of;

determining whether the second real-time server of the real time tier is assigned to process the event tracking data for the website based on the data structure, wherein being assigned to process the event tracking data for the website comprises being assigned to generate analytics data based on the visitor data of the event tracking data and report the analytics data to a front-end server;

comparing a threshold time period with an elapsed time of the event tracking data;

pushing, in response to if the comparison showing the elapsed time as less than the threshold time and in response to determining that the second real-time server is assigned to process the event tracking data for the website, the event tracking data from the first real-time server to the second real-time server based on the predetermined deadline;

or

discarding, if the comparison shows the elapsed time greater than the threshold time, the pushed event tracking data.

12. The system of claim 11, wherein determining that the second real-time server is assigned to process the event tracking data comprises:

determining a tracking account associated with the event tracking data; and

determining that the second real-time server is associated with the tracking account.

13. The system of claim 11, wherein determining that the second real-time server is assigned to process the event tracking data comprises:

determining a web property identity associated with the event tracking data;

determining a tracking account associated with the web property identity; and

determining that the second real-time server is associated with the tracking account.

14. The system of claim 11, wherein the first real-time server further comprises

calculating the elapsed time based on when the first real-time server received the event tracking data.

15. The system of claim 11, wherein the first real-time server further comprises

calculating the elapsed time based on a timestamp associated with the event tracking data.

16. The system of claim 11, wherein the first real-time server pushing the event tracking data to the second real-time server further comprises

pushing the event tracking data without a request or polling.

17. The system of claim 11, wherein the predetermined deadline comprises a specified period of time by which the event tracking data must be processed at each respective tier.

18. The system of claim 11, wherein the logging tier selecting a first real-time server comprises

randomly selecting, at a logging server of the logging tier, a real-time server of a real-time tier.

19. The system of claim 11, wherein the logging tier selecting a first real-time server comprises

selecting, at a logging server of the logging tier, a real-time server of a real-time tier based on load-balancing requirements among a plurality of real-time servers of the real-time tier.

20. The system of claim 11, wherein the first real-time server further comprises:

receiving a second event tracking data;

determining that the first real-time server is acting as an owner of the second event tracking data; and

processing the second event tracking data.