Complex event processing (CEP) based system for handling performance issues of a CEP system and corresponding method

• US 10,229,162 B2
• Filed: 08/15/2014
• Issued: 03/12/2019
• Est. Priority Date: 11/06/2013
• Status: Active Grant

• Current Assignee: Software AG
• Original Assignee: Software AG
• Inventors: Heinz, Christoph, Cammert, Michael, Kramer, Jurgen, Schafer, Daniel, Riemenschneider, Tobias
• Primary Examiner(s): Mofiz, Apu M
• Assistant Examiner(s): Agharahimi, Farhad
• Application Number:
  US14/460,687
• Publication Number:
  US 20150286684A1
• Time in Patent Office: 1,670 Days
• Field of Search: 707769
• CPC Class Codes: G06F 11/3096 wherein the means or processing minimize the use of computing system or of computing system component resources, e.g. non-intrusive monitoring which minimizes the probe effect: sniffing, intercepting, indirectly deriving the monitored data from other directly available data
• CPC Class Codes: G06F 11/328 Computer systems status display G06F11/327 takes precedence
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 16/2452 Query translation
• CPC Class Codes: G06F 16/24549 Run-time optimisation
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: G06F 2201/88 Monitoring involving counting

Abstract

Certain example embodiments relate to a system (1) for handling performance issues of a production Complex Event Processing, CEP, system (2) during runtime. The production CEP system (2) includes at least one event source, at least one continuous query and at least one event sink. The system (1) includes: at least one monitoring sensor for producing a stream of status events relating to the production CEP system (2); and a monitoring CEP system (10) for executing at least one continuous analysis query on the stream of status events to produce a stream of monitoring events. The stream of monitoring events indicates performance issues of the production CEP system (2) relating to the throughput, the latency, and/or the memory consumption of the production CEP system (2).

Claims

1. A system for handling performance issues of a production Complex Event Processing (CEP) system during runtime, wherein the production CEP system is configured to execute a continuous production query that includes a plurality of operators that are executed as part the continuous query, the system comprising:

a processing system that includes at least one hardware processor, the processing system configured to;

receive a stream of status events produced via a plurality of monitoring sensors that are each assigned to an operator of the continuous query, where each monitoring sensor is configured to acquire status information from a corresponding operator that is then included in the stream of status events;

execute a monitoring CEP system that is configured to run at least one continuous analysis query on the stream of status events to produce a stream of monitoring events, wherein the stream of monitoring events indicates performance issues of the production CEP system relating to the throughput, the latency, and/or the memory consumption of the production CEP system;

execute a system stabilization component that is configured to;

in response to identification of a first performance issue indicated in the stream of monitoring events, (1) sort a plurality of continuous queries of the production CEP system based on memory consumption of the plurality of continuous queries, and (2) based on the sorted plurality of continuous queries, stop at least some of the plurality of continuous queries and/or move at least one query to another processing component, and

in response to identification of a second performance issue indicated in the stream of monitoring events, (1) sort input streams of the production CEP system by input rate, (2) sort the plurality of continuous queries of the production CEP system by output rate, and (3) based on the sorted inputted streams and sorted plurality of continuous queries, stop and/or move at least one of the plurality of continuous queries to another processing component.

2. The system of claim 1, wherein the at least one monitoring sensor is attached to an operator of the at least one continuous query of the production CEP system and is configured to count input and/or output events of the operator and/or to compute a memory consumption of the operator.

3. The system of claim 1, further comprising an analytics component configured to analyze the stream of monitoring events using stream mining and to generate at least one statistical model of the performance of the production CEP system.

4. The system of claim 3, further comprising a statistical model database configured to store the at least one statistical model generated by the analytics component,

wherein the analytics component is further configured to derive a forecast of the status of the production CEP system based on the at least one stored statistical model and a current statistical model of the production CEP system.

5. The system of claim 1, further comprising a graphical user interface configured to indicate at least one identified performance issue of the production CEP system.

6. The system of claim 5, wherein the graphical user interface is configured to display the at least one continuous query of the production CEP system as an operator graph, wherein operators of the operator graph involving a performance issue are indicated.

7. The system of claim 1, wherein the system stabilization component is further configured to:

generate and output an indication of a recommended action for resolving an identified performance issue of the production CEP system.

8. The system of claim 7, wherein the recommended action for resolving an identified performance issue of the production CEP system is selected from the group comprising:

stopping the at least one continuous query of the production CEP system,

moving the at least one continuous query of the production CEP system to another processing component, and/or

modifying the at least one continuous query of the production CEP system.

9. The system of claim 8, wherein modifying the at least one continuous query of the production CEP system comprises reducing a window size of the at least one continuous query, reducing an output rate of the at least one continuous query, and/or removing event attributes not used by the at least one continuous query.

10. The system of claim 1, wherein the system stabilization component is further configured to:

execute a query optimizer.

18. The system of claim 1, wherein the processing system is further configured to:

based on the produced stream of monitoring events from the monitoring CEP system, adjust a parameter of at least one of the plurality of monitoring sensors to change how the corresponding monitoring sensor acquires the status information that is included in the stream of status events.

11. A method for handling performance issues of a production Complex Event Processing (CEP) system during runtime, the production CEP system comprising at least one event source, at least one continuous query and at least one event sink, the method comprising:

producing a stream of status events based on status data acquired by a plurality of monitoring sensors that are each attached to a different operator of the at least one continuous query of the production CEP system;

executing, by a monitoring CEP system including at least one computer processor, at least one continuous analysis query on the stream of status events to produce a stream of monitoring events;

based on the produced stream of monitoring events from the monitoring CEP system, adjust a parameter of at least one of the plurality of monitoring sensors to change how the corresponding monitoring sensor acquires the status information that is included in the stream of status events;

executing, using the monitoring CEP system, a system stabilization component;

using the system stabilization component for;

identifying a first performance issue indicated in the stream of monitoring events and in response to identification of the first performance issue;

(1) sorting a plurality of continuous queries of the production CEP system based on memory consumption of the plurality of continuous queries, and (2) based on the sorted plurality of continuous queries, stopping, using the system stabilization component, at least some of the plurality of continuous queries and/or moving at least one query to another processing component;

identifying a second performance issue indicated in the stream of monitoring events and in response to identification of the second performance issue;

(1) sorting input streams of the production CEP system by input rate, (2) sorting the plurality of continuous queries of the production CEP system by output rate, and (3) based on the sorted inputted streams and sorted plurality of continuous queries, stopping and/or moving at least one of the plurality of continuous queries to another processing component.

12. The method of claim 11, further comprising analyzing, the stream of monitoring events using stream mining and generating at least one statistical model of the performance of the production CEP system.

13. The method of claim 11, further comprising indicating, by a graphical user interface, at least one identified performance issue of the production CEP system;

and/or indicating, by the system stabilization component, a recommended action for resolving an identified performance issue of the production CEP system.

19. The method of claim 11, wherein the system stabilization component is also used for:

identifying a third performance issue related to consumption of events by the at least one event sink, and, in response to identification of the third performance issue, decoupling output of the at least one continuous query from the at least one event sink.

14. A non-transitory computer readable storage medium tangibly storing a computer program for use with a processing system that includes at least one hardware processor, wherein the processing system is for use in handling performance issues of a production Complex Event Processing (CEP) system during runtime, the production CEP system is configured to execute a continuous production query that includes a plurality of operators that are executed as part the continuous query, the computer program comprising instructions that are configured to cause the processing system to:

receive, via monitoring sensors that are each attached to an operator of the executing continuous production query on the production CEP system, a stream of status events, where each of the monitoring sensors is configured to acquire status information from an attached operator based on execution of the continuous production by the production CEP system, the acquired status information being included in the stream of status events;

communicate the stream of status events to a monitoring CEP system that is separate from the production CEP system;

execute, as part of the monitoring CEP system, a continuous analysis query on the stream of status events to thereby create a stream of monitoring events, wherein the stream of monitoring events are performance metrics of the production CEP system relating to the throughput, the latency, and/or the memory consumption of the production CEP system;

determine, at the monitoring system and based on the stream of status events, whether the performance metrics exceed a threshold;

in response to determination that the performance metrics exceed the threshold, cause at least one parameter of at least one of the monitoring sensors to be adjusted to change how the monitoring sensor acquires the status information, which is included in the stream of status events, from the corresponding operator of the executing continuous production query;

identify a first performance issue indicated in the stream of monitoring events and in response to identification of the first performance issue;

(1) sort a plurality of continuous queries of the production CEP system based on memory consumption of the plurality of continuous queries, and (2) based on the sorted plurality of continuous queries, stop, using the system stabilization component, at least some of the plurality of continuous queries and/or moving at least one query to another processing component; and

identify a second performance issue indicated in the stream of monitoring events and in response to identification of the second performance issue;

(1) sort input streams of the production CEP system by input rate, (2) sort the plurality of continuous queries of the production CEP system by output rate, and (3) based on the sorted inputted streams and sorted plurality of continuous queries, stop and/or move at least one of the plurality of continuous queries to another processing component.

15. The non-transitory computer readable storage medium of claim 14, wherein the adjusted at least one parameter changes a frequency in which the stream of status events are communicated from the corresponding monitoring sensor to the monitoring CEP system.

16. The non-transitory computer readable storage medium of claim 14, wherein the adjusted at least one parameter changes causes an additional property of the production CEP system to be monitored and corresponding monitoring sensor to produce status events related to the additional property.

17. The non-transitory computer readable storage medium of claim 14, wherein the adjusted at least one parameter causes a corresponding monitoring sensor to stop producing events and/or monitoring the corresponding operator.