DATA STREAM SPLITTING FOR LOW-LATENCY DATA ACCESS
First Claim
1. A method comprising:
- producing, at a plurality of front end servers, log data based on real-time user activities;
transmitting the log data to an aggregating server;
aggregating the log data at the aggregating server;
splitting the log data into a plurality of log data streams; and
feeding the log data streams to at least one back end server in parallel.
3 Assignments
0 Petitions
Accused Products
Abstract
Techniques for facilitating and accelerating log data processing by splitting data streams are disclosed herein. The front-end clusters generate large amount of log data in real time and transfer the log data to an aggregating cluster. The aggregating cluster is designed to aggregate incoming log data streams from different front-end servers and clusters. The aggregating cluster further splits the log data into a plurality of data streams so that the data streams are sent to a receiving application in parallel. In one embodiment, the log data are randomly split to ensure the log data are evenly distributed in the split data streams. In another embodiment, the application that receives the split data streams determines how to split the log data.
-
Citations
20 Claims
-
1. A method comprising:
-
producing, at a plurality of front end servers, log data based on real-time user activities; transmitting the log data to an aggregating server; aggregating the log data at the aggregating server; splitting the log data into a plurality of log data streams; and feeding the log data streams to at least one back end server in parallel. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17)
-
-
14. A computer-implemented system, comprising:
-
a plurality of front end servers configured for producing log data based on real-time user activities; and at least one aggregating server configured for aggregating the log data received from at least some of the front end servers, the aggregating server being connected with at least some of the front end servers via a network; wherein the aggregating server includes a data staging area configured for staging the log data, and the aggregating server is configured for splitting the log data into a plurality of log data streams so that one or more back end servers can retrieve the log data streams in parallel. - View Dependent Claims (15, 16)
-
-
18. An aggregating server, comprising:
-
a processor; a network interface, coupled to the processor, through which the aggregating server can communicate with a plurality of front end servers; a data storage including a data staging area; and a memory storing instructions which, when executed by the processor, cause the aggregating server to perform a process including; receiving log data from the front end servers, wherein the front end servers produce the log data based on real-time user activities, aggregating the log data, staging the log data at the data staging area, and splitting the log data into a plurality of log data streams so that one or more back end servers can retrieve the log data streams in parallel. - View Dependent Claims (19, 20)
-
Specification