Data Processing Environment Event Correlation

• US 20120272249A1
• Filed: 06/26/2012
• Published: 10/25/2012
• Est. Priority Date: 02/25/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation, The Boeing Co.
• Original Assignee: International Business Machines Corporation
• Inventors: Beaty, Kirk A., Breitgand, David, Naik, Vijay K., Tan, Yih-Shin, Vogl, Norbert G., Epstein, Amir, Marom, Daniel
• Granted Patent: US 9,128,773 B2
• Time in Patent Office: Days
• US Class Current: 719/318
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: G06F 2209/508 Monitor
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system

Abstract

A system and method for correlating events in data processing resources processing a workload. An event correlation service comprising an event correlation service software framework is provided on a processor unit. The event correlation service is configured, using the event correlation service software framework, for correlating the events from monitored data for the workload, wherein the workload is processed using the data processing resources provided by a provider of the data processing resources. The event correlation service receives monitored data for the workload as the workload is being processed. The monitored data is processed by the event correlation service to identify an occurrence of an event of interest for the workload as the workload is being processed. An output is provided in response to identifying the occurrence of the event of interest.

Claims

1. A method for correlating events in data processing resources processing a workload, comprising:

providing, on a processor unit, an event correlation service comprising an event correlation service software framework;

configuring the event correlation service, using the event correlation service software framework, for correlating the events from monitored data for the workload, wherein the workload is processed using the data processing resources provided by a provider of the data processing resources;

receiving, by the event correlation service, monitored data for the workload as the workload is being processed;

processing the monitored data, by the event correlation service, to identify an occurrence of an event of interest for the workload as the workload is being processed; and

providing an output in response to identifying the occurrence of the event of interest.

2. The method of claim 1 further comprising:

configuring a monitoring infrastructure to collect the monitored data for the workload;

collecting the monitored data for the workload by the monitoring infrastructure as the workload is being processed; and

providing the monitored data from the monitoring infrastructure to the event correlation service as the workload is being processed.

3. The method of claim 1, wherein:

configuring the event correlation service comprises creating, by the processor unit, an event correlation software instance from an event correlation software plug-in, wherein the event correlation software plug-in is a software plug-in to the event correlation service software framework and wherein the event correlation software instance inherits from the event correlation software plug-in first correlation parameters used for correlating the events in processing the workload, and receiving, by the processor unit, second correlation parameters for the event correlation software instance, wherein the second correlation parameters comprise workload specific parameters used for correlating the events in processing the workload; and

processing the monitored data comprises using the event correlation software instance to identify the occurrence of the event of interest from the monitored data using the first correlation parameters and the second correlation parameters.

4. The method of claim 3, wherein configuring the event correlation service further comprises:

receiving, by the processor unit, a request to correlate the events in processing the workload;

responsive to receiving the request, determining, by the processor unit, whether the event correlation software plug-in is registered with the event correlation service software framework; and

responsive to a determination that the event correlation software plug-in is not registered with the event correlation service software framework, using an application programming interface of the event correlation service software framework to load and register the event correlation software plug-in.

5. The method of claim 4, wherein the request is received from a workload manager and further comprising sending the output to the workload manager.

6. The method of claim 1, wherein processing the monitored data comprises:

correlating the monitored data from a plurality of data processing resources processing the workload to provide correlated data;

determining whether the correlated data exceeds a threshold; and

identifying the occurrence of the event of interest responsive to a determination that the correlated data exceeds the threshold.

7. The method of claim 1, wherein processing the monitored data comprises:

correlating the monitored data from a number of a plurality of data processing resources processing the workload to provide correlated data;

determining whether the number of the plurality of data processing resources meets or exceeds a quorum number, wherein the quorum number is less than a total number of the plurality of data processing resources processing the workload; and

processing the correlated data to identify the occurrence of the event of interest responsive to a determination that the number of the plurality of data processing resources meets or exceeds the quorum number.

8. The method of claim 1, wherein the data processing resources comprise first data processing resources and second data processing resources, wherein the first data processing resources and the second data processing resources comprise a hybrid cloud, and wherein the hybrid cloud is selected from a group of hybrid clouds consisting of:

a first hybrid cloud wherein the first data processing resources are provided as a service on a public cloud and the second data processing resources are provided on a private cloud;

a second hybrid cloud wherein the first data processing resources are provided as a service on the private cloud and the second data processing resources are provided on another private cloud; and

a third hybrid cloud wherein the first data processing resources are provided as the service on a public cloud and the second data processing resources are provided on another public cloud.

9. A computer program product for correlating events in data processing resources processing a workload, comprising:

a computer readable storage medium;

first program instructions to provide an event correlation service comprising an event correlation service software framework;

second program instructions to configure the event correlation service, using the event correlation service software framework, for correlating events from monitored data for the workload, wherein the workload is processed using data processing resources provided by a provider of the data processing resources;

third program instructions to receive, by the event correlation service, the monitored data for the workload as the workload is being processed;

fourth program instructions to process the monitored data, by the event correlation service, to identify an occurrence of an event of interest for the workload as the workload is being processed;

fifth program instructions to provide an output in response to identifying the occurrence of the event of interest; and

wherein the first program instructions, second program instructions, third program instructions, fourth program instructions, and fifth program instructions are stored on the computer readable storage medium.

10. The computer program product of claim 9, wherein:

the second program instructions comprise program instructions to create an event correlation software instance from an event correlation software plug-in, wherein the event correlation software plug-in is a software plug-in to the event correlation service software framework and wherein the event correlation software instance inherits from the event correlation software plug-in first correlation parameters used for correlating the events in processing the workload, and program instructions to receive second correlation parameters for the event correlation software instance, wherein the second correlation parameters comprise workload specific parameters used for correlating the events in processing the workload; and

the fourth program instructions comprise program instructions to process the monitored data using the event correlation software instance to identify the occurrence of the event of interest from the monitored data using the first correlation parameters and the second correlation parameters.

11. The computer program product of claim 10, wherein the second program instructions further comprise:

program instructions to receive a request to correlate events in processing the workload;

program instructions to determine, responsive to receiving a request, whether the event correlation software plug-in is registered with the event correlation service software framework; and

program instructions to load and register the event correlation software plug-in with the event correlation service software framework responsive to a determination that the event correlation software plug-in is not registered with the event correlation service software framework.

12. The computer program product of claim 11, wherein:

the second program instructions comprise the program instructions to receive the request from a workload manager; and

the fifth program instructions comprise program instructions to send the output to the workload manager.

13. The computer program product of claim 9, wherein the fourth program instructions comprise:

program instructions to correlate the monitored data from a plurality of data processing resources processing the workload to provide correlated data;

program instructions to determine whether the correlated data exceeds a threshold; and

program instructions to identify the occurrence of the event of interest responsive to a determination that the correlated data exceeds the threshold.

14. The computer program product of claim 9, wherein the fourth program instructions comprise:

program instructions to correlate the monitored data from a number of a plurality of data processing resources processing the workload to provide correlated data;

program instructions to determine whether a number of plurality of data processing resources meets or exceeds a quorum number, wherein the quorum number is less than a total number of the plurality of data processing resources processing the workload; and

program instructions to process the correlated data to identify the occurrence of the event of interest responsive to a determination that the number of the plurality of data processing resources meets or exceeds the quorum number.

15. The computer program product of claim 9, wherein the data processing resources comprise first data processing resources and second data processing resources, wherein the first data processing resources and the second data processing resources comprise a hybrid cloud, and wherein the hybrid cloud is selected from a group of hybrid clouds consisting of:

a first hybrid cloud wherein the first data processing resources are provided as a service on a public cloud and the second data processing resources are provided on a private cloud;

a second hybrid cloud wherein the first data processing resources are provided as the service on the private cloud and the second data processing resources are provided on another private cloud; and

a third hybrid cloud wherein the first data processing resources are provided as the service on the public cloud and the second data processing resources are provided on another public cloud.

16. An apparatus, comprising:

a software framework;

a software plug-in loaded on the software framework, the software plug-in comprising first correlation parameters used for correlating events in data processing resources;

an instance of the software plug-in loaded on the software framework, the instance comprising the first correlation parameters inherited from the software plug-in and second correlation parameters comprising workload specific parameters used for correlating the events in processing a workload; and

a processor unit configured to receive monitored data for the workload as the workload is being processed by the data processing resources, process the monitored data using the instance to identify an occurrence of an event of interest from the monitored data using the first correlation parameters and the second correlation parameters, and provide an output responsive to identifying the occurrence of the event of interest.

17. The apparatus of claim 16, wherein the processor unit is configured to:

receive a request to correlate the events in processing the workload;

responsive to receiving the request, determine whether the software plug-in is loaded on the software framework; and

responsive to a determination that the software plug-in is not loaded on the software framework, load the software plug-in on the software framework.

18. The apparatus of claim 17, wherein the processor unit is configured to:

receive the request from a workload manager; and

send the output to the workload manager.

19. The apparatus of claim 16, wherein the processor unit is configured to:

correlate the monitored data from a plurality of data processing resources processing the workload to provide correlated data;

determine whether the correlated data exceeds a threshold; and

identify the occurrence of the event of interest responsive to a determination that the correlated data exceeds the threshold.

20. The apparatus of claim 16, wherein processor unit is configured to:

correlate the monitored data from a number of a plurality of data processing resources processing the workload to provide correlated data;

determine whether the number of the plurality of data processing resources meets or exceeds a quorum number, wherein the quorum number is less than a total number of the plurality of data processing resources processing the workload; and

process the correlated data to identify the occurrence of the event of interest responsive to a determination that the number of the plurality of data processing resources meets or exceeds the quorum number.

21. The apparatus of claim 16, wherein the data processing resources comprise first data processing resources and second data processing resources, wherein the first data processing resources and the second data processing resources comprise a hybrid cloud, and wherein the hybrid cloud is selected from a group of hybrid clouds consisting of:

a first hybrid cloud wherein the first data processing resources are provided as a service on a public cloud and the second data processing resources are provided on a private cloud;

a second hybrid cloud wherein the first data processing resources are provided as the service on the private cloud and the second data processing resources are provided on another private cloud; and

a third hybrid cloud wherein the first data processing resources are provided as the service on the public cloud and the second data processing resources are provided on another public cloud.

22. A method for correlating events in data processing resources processing a workload, comprising:

receiving, by a processor unit, a request to correlate the events in the data processing resources processing the workload;

responsive to receiving the request, determining, by the processor unit, whether a software plug-in is loaded on a framework, the software plug-in comprising first correlation parameters used for correlating events in processing the workload;

responsive to a determination that the software plug-in is not loaded on the framework, using an application programming interface of the framework to load the software plug-in on the framework;

creating an instance of the software plug-in loaded on the framework, the instance comprising the first correlation parameters inherited from the software plug-in and second correlation parameters comprising workload specific parameters used for correlating the events in processing the workload;

receiving, using the framework, monitored data for the workload as the workload is being processed;

processing the monitored data using the instance to identify an occurrence of an event of interest from the monitored data using the first correlation parameters and the second correlation parameters; and

providing an output responsive to identifying the occurrence of the event of interest.

23. The method of claim 22, wherein:

the request comprises the second correlation parameters; and

the request is received from a workload manager.

24. The method of claim 23, further comprising sending the output to the workload manager.

25. The method of claim 22, wherein processing the monitored data comprises:

correlating the monitored data from a number of plurality of data processing resources processing the workload to provide correlated data;

determining whether the number of the plurality of data processing resources meets or exceeds a quorum number, wherein the quorum number is less than a total number of the plurality of data processing resources processing the workload;

responsive to a determination that the number of the plurality of data processing resources meets or exceeds the quorum number, determining whether the correlated data exceeds a threshold; and

identifying the occurrence of the event of interest responsive to the determination that the correlated data exceeds the threshold.