Implementation of stream algebra over class instances
First Claim
1. A method for configuring an operator graph to process a high-volume continuous stream of events, said method comprising:
- defining, by a subscriber machine, an object to include an expression, said expression representing a function to be performed on events from an event stream;
generating, by the subscriber machine, an abstract syntax tree corresponding to the object;
serializing, by the subscriber machine, the generated abstract syntax tree into a message; and
sending, by the subscriber machine, the message including a query representing the object to a destination machine, said destination machine being one of a plurality of destination machines, wherein the expression is configured to be compiled by the destination machine from the abstract syntax tree into executable code to be executed by the designation machine on events received from the event stream.
2 Assignments
0 Petitions
Accused Products
Abstract
Creating and executing a distributed stream processing operator graph based on a query. The operator graph includes movable stream algebra operators for processing events received from high volume data streams. The operators are partially compiled and distributed to computing devices for completion of the compilation and subsequent execution. During execution, the operators maintain minimal state information associated with received events via an expiration time assigned to each of the event instances. Additional events are generated and aggregated by the operators for communication to a service responsible for the query.
-
Citations
20 Claims
-
1. A method for configuring an operator graph to process a high-volume continuous stream of events, said method comprising:
-
defining, by a subscriber machine, an object to include an expression, said expression representing a function to be performed on events from an event stream; generating, by the subscriber machine, an abstract syntax tree corresponding to the object; serializing, by the subscriber machine, the generated abstract syntax tree into a message; and sending, by the subscriber machine, the message including a query representing the object to a destination machine, said destination machine being one of a plurality of destination machines, wherein the expression is configured to be compiled by the destination machine from the abstract syntax tree into executable code to be executed by the designation machine on events received from the event stream. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A method of processing high-volume continuous streams of data comprising:
-
receiving, by a destination machine, a query from a subscriber machine, said query comprising query data and operator semantics, said query representing a request for data and further representing a function to be performed on events from an event stream; translating, by the destination machine, the received query into a plurality of stream algebra operators; defining, by the destination machine, an execution plan representing a data flow implemented by the stream algebra operators; receiving, by the destination machine, a first event from a high volume continuous data stream while maintaining minimal state information associated with the data; assigning, by the destination machine, an expiration time to the received first event as a function of the query data; receiving, by the destination machine, a plurality of second events from the data stream; executing, by the destination machine, one or more of the stream algebra operators according to the defined execution plan and as a function of the operator semantics from the received query to find a match between the received first event and at least one of the received plurality of second events prior to expiration of the first event based on the assigned expiration time associated with the first event wherein second events matching unexpired first events are combined; and executing, by the destination machine, combined events according to the execution plan.
-
-
14. A system for processing high-volume continuous streams of data while maintaining minimal state information associated with the data, said system comprising:
-
a plurality of destination machines; a subscriber machine having a subscriber processor configured to execute computer executable instructions for; defining an object to include an expression, said expression representing a function to be performed on events from an event stream; generating an abstract syntax tree corresponding to the object; serializing the generated abstract syntax tree into a message; and sending the message representing the object to one of the destination machines which compiles the expression from the abstract syntax tree into executable code, and wherein the executable code is executed by the one destination machine on events received from the event stream. - View Dependent Claims (15, 16, 17, 18, 19, 20)
-
Specification