EVENT STREAM PROCESSING PARTITIONING

• US 20150169786A1
• Filed: 03/10/2014
• Published: 06/18/2015
• Est. Priority Date: 12/16/2013
• Status: Active Grant

• Current Assignee: Sybase Incorporated (SAP SE)
• Original Assignee: Sybase Incorporated (SAP SE)
• Inventors: Jerzak, Zbigniew, AHMED, IBRAHIM, GANDHI, PALANIAPPAN
• Granted Patent: US 9,405,854 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24524 Access plan code generation and invalidation; Reuse of access plans
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence

Abstract

In an example embodiment, event stream processing is performed by first parsing an input query into a directed acyclic graph (DAG) including a plurality of operator nodes. Then a grouping of one or more of the operator nodes is created. One or more partitions are created, either by the user or automatically, in the DAG by forming one or more duplicates of the grouping. A splitter node is created in the DAG, the splitter node splits data from one or more event streams and distributes it among the grouping and the duplicates of the grouping. Then, the input query is resolved by processing data from one or more event streams using the DAG.

Claims

1. A method for performing event stream processing comprising:

receiving an input query;

parsing the input query into a directed acyclic graph (DAG) including a plurality of operator nodes;

creating a grouping of one or more of the operator nodes;

creating one or more partitions in the DAG by forming one or more duplicates of the grouping;

creating a splitter node in the DAG, the splitter node splitting data from one or more event streams among the grouping and the one or more duplicates of the grouping; and

resolving the input query by processing data from one or more event streams using the DAG.

2. The method of claim 1, wherein the creating of the grouping occurs at a server level, resulting in operator nodes of a server being grouped.

3. The method of claim 1, further comprising creating a merge node, the merge node consolidating results of execution of the one or more partitions.

4. The method of claim 1, wherein the creating the one or more partitions uses a round-robin strategy.

5. The method of claim 1, wherein the creating the one or more partitions uses a hash-based strategy.

6. The method of claim 1, wherein the creating the one or more partitions uses a custom strategy based on user input of a logic which determines a target copy for a given data item.

7. The method of claim 1, further comprising creating a second grouping of one or more operator nodes nested within the grouping of one or more operator nodes.

8. An apparatus comprising:

an event stream processing engine executable by a processor and configured to;

receive an input query;

parse the input query into a directed acyclic graph (DAG) including a plurality of operator nodes;

create a grouping of one or more of the operator nodes;

create one or more partitions in the DAG by forming one or more duplicates of the grouping;

create a splitter node in the DAG, the splitter node splitting data from one or more event streams among the grouping and the duplicates of the grouping; and

resolve the input query by processing data from one or more event streams using the DAG.

9. The apparatus of claim 8, wherein the apparatus is coupled to one or more input adapters receiving input from at least one of the following:

market data feeds, a message bus, network monitoring, application transactions, and internal event streams.

10. The apparatus of claim 8, wherein the apparatus is coupled to a database where results of the resolution of the input query are stored.

11. The apparatus of claim 8, wherein the apparatus is coupled to one or more applications where results of the resolution of the input query may be sent and displayed.

12. The apparatus of claim 8, wherein the creating of a grouping occurs at a server level, resulting in all operator nodes of a server being grouped.

13. The apparatus of claim 8, wherein the creating the one or more partitions uses a round-robin strategy.

14. The apparatus of claim 8, wherein the creating the one or more partitions uses a hash-based strategy.

15. A non-transitory machine-readable storage medium comprising instructions, which when implemented by one or more machines, cause the one or more machines to perform operations comprising:

receiving an input query;

parsing the input query into a directed acyclic graph (DAG) including a plurality of operator nodes;

creating a grouping of one or more of the operator nodes;

creating one or more partitions in the DAG by forming one or more duplicates of the grouping;

creating a splitter node in the DAG, the splitter node splitting data from one or more event streams among the grouping and the duplicates of the grouping; and

resolving the input query by processing data from one or more event streams using the DAG.

16. The non-transitory machine-readable storage medium of claim 15, wherein the operations further comprise creating a merge node, the merge node consolidating results of execution of the one or more partitions.

17. The non-transitory machine-readable storage medium of claim 15, wherein the creating the one or more partitions uses a round-robin strategy.

18. The non-transitory machine-readable storage medium of claim 15, wherein the creating the one or more partitions uses a hash-based strategy.

19. The non-transitory machine-readable storage medium of claim 15, wherein the creating the one or more partitions uses a custom strategy that allows a user to provide a logic which determines a target copy for a given data item.

20. The non-transitory machine-readable storage medium of claim 15, wherein the operations comprise creating a second grouping of one or more operator nodes nested within the grouping of one or more operator nodes.