IMPLEMENTATION OF STREAM ALGEBRA OVER CLASS INSTANCES
First Claim
1. A method of processing high-volume continuous streams of data while maintaining minimal state information associated with the data, said method comprising:
- receiving a query comprising query data and operator semantics, said query representing a request for data;
translating the received query into a plurality of stream algebra operators;
defining an execution plan representing a data flow implemented by the stream algebra operators;
receiving a first event from a data stream;
assigning an expiration time to the received first event as a function of the query data;
receiving a plurality of second events from the data stream;
executing one or more of the stream algebra operators according to the defined execution plan and as a function of the operator semantics from the received query to find a match between the received first event and at least one of the received plurality of second events prior to expiration of the first event based on the assigned expiration time associated with the first event; and
producing, as a function of said executing, a third event to be executed according to the execution plan.
2 Assignments
0 Petitions
Accused Products
Abstract
Creating and executing a distributed stream processing operator graph based on a query. The operator graph includes movable stream algebra operators for processing events received from high volume data streams. The operators are partially compiled and distributed to computing devices for completion of the compilation and subsequent execution. During execution, the operators maintain minimal state information associated with received events via an expiration time assigned to each of the event instances. Additional events are generated and aggregated by the operators for communication to a service responsible for the query.
140 Citations
20 Claims
-
1. A method of processing high-volume continuous streams of data while maintaining minimal state information associated with the data, said method comprising:
-
receiving a query comprising query data and operator semantics, said query representing a request for data; translating the received query into a plurality of stream algebra operators; defining an execution plan representing a data flow implemented by the stream algebra operators; receiving a first event from a data stream; assigning an expiration time to the received first event as a function of the query data; receiving a plurality of second events from the data stream; executing one or more of the stream algebra operators according to the defined execution plan and as a function of the operator semantics from the received query to find a match between the received first event and at least one of the received plurality of second events prior to expiration of the first event based on the assigned expiration time associated with the first event; and producing, as a function of said executing, a third event to be executed according to the execution plan. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A system for processing high-volume continuous streams of data while maintaining minimal state information associated with the data, said system comprising:
-
a memory area for storing a plurality of stream algebra operators for building a distributed stream processing operator graph, each of said stream algebra operators being adapted to perform one or more functions on data events from an event stream; and a processor configured to execute computer-executable instructions for building a distributed operator graph using the stream algebra operators stored in the memory area by; defining an object to include an expression, said expression representing the one or more functions to be performed on the data events from the event stream; serializing the defined object including the expression into a message corresponding to one of the plurality of stream algebra operators stored in the memory area; identifying one of a plurality of destination computing devices to host said one of the plurality of stream algebra operators; transmitting the message to the identified destination computing device to build the distributed operator graph, wherein the destination computing device compiles the expression from the message into executable code representing said one of the plurality of stream algebra operators implementing a portion of the operator graph, wherein the executable code operates on the data events received from the event stream to produce aggregated events; receiving the aggregated events from the destination computing device; and processing the aggregated events in accordance with another portion of the operator graph implemented by the processor. - View Dependent Claims (12, 13, 14, 15, 16, 17)
-
-
18. A method for configuring an operator graph to process a continuous stream of events, said method comprising:
-
defining an object to include an expression, said expression representing a function to be performed on events from an event stream; generating an abstract syntax tree corresponding to the object; serializing the generated abstract syntax tree into a message; and sending the message representing the object to a destination machine, said destination machine being one of a plurality of machines, wherein the destination machine compiles the expression from the abstract syntax tree into executable code, wherein the executable code is executed on events received from the event stream. - View Dependent Claims (19, 20)
-
Specification