MONITOR-MINE-MANAGE CYCLE

• US 20140195465A1
• Filed: 01/05/2013
• Published: 07/10/2014
• Est. Priority Date: 01/05/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Periorellis, Panos, Akchurin, Eldar, Nano, Olivier, Claessens, Joris, Santos, Ivo Jose Garcia dos
• Granted Patent: US 10,095,978 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/301 where the computing system is a virtual computing platform, e.g. logically partitioned systems virtual machines G06F9/45533; logical partitioning of resources G06F9/5077
• CPC Class Codes: G06F 11/3082 the data filtering being achieved by aggregating or compressing the monitored data
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A monitor-mine-manage cycle is described, for example, for managing a data center, a manufacturing process, an engineering process or other processes. In various example, the following steps are performed as a continuous automated loop: receiving raw events from an observed system; monitoring the raw events and transforming them into complex events; mining the complex events and reasoning on results; making a set of proposed actions based on the mining; and managing the observed system by applying one or more of the proposed actions to the system. In various examples, the continuous automated loop proceeds while raw events are continuously received from the observed system and monitored. In some examples an application programming interface is described comprising programming statements which allow a user to implement a monitor-mine-manage loop.

Claims

1. A computer-implemented method, wherein the following steps are performed as a continuous automated loop:

receiving raw events from an observed system;

monitoring the raw events and transforming them into complex events;

mining the complex events and reasoning on results;

making a set of proposed actions based on the mining; and

managing the observed system by applying the proposed actions to the system, while raw events are continuously received from the observed system and monitored.

2. The method of claim 1, wherein the steps of monitoring, mining and managing are performed in real-time or near real-time and an input to a step is based on output of a previous step with little or no time delay.

3. The method of claim 1, wherein the raw events are received as streams and the transformation from raw events into complex events is performed by event stream processing technology.

4. The method of claim 1, wherein the transformation from raw events into complex events comprises aggregation and processing of the raw events.

5. The method of claim 1, wherein the mining comprises performing at least one of statistical analysis, pattern recognition and machine learning.

6. The method of claim 1, wherein the raw events are the changes of the length of a queue of user requests and the mining comprises using a Bayesian model for estimating a length of the queue.

7. The method of claim 6, wherein the mining further comprises receiving a service level agreement and comprises predicting whether the service level agreement will be fulfilled.

8. The method of claim 1, wherein the proposed actions based on the mining are represented as a stream.

9. The method of claim 1, wherein the managing comprises performing repairs, updates, backups and configurations on the observed system.

10. The method of claim 7, wherein the managing comprises deploying and removing a server from a set of servers which serve the user requests.

11. The method of claim 1, wherein the managing is based on static constraints.

12. The method of claim 11, wherein the static constraints are costs involved with the proposed actions to be performed.

13. The method of claim 11, wherein the static constraints are at least one of the maximum number of servers deployed, the maximum number of electricity spent per day, the maximum number of cash spent to have the servers deployed.

14. The method of claim 1, wherein a streaming engine is located on the observed system and a streaming client is located on a monitoring system.

15. The method of claim 1, wherein the method is used to make the observed system a self-healing system.

16. A monitor-mine-manage engine comprising:

a communications interface arranged to receiving raw events from an observed system;

a processor arranged to monitor the raw events by transforming them into complex events;

the processor being arranged to mine the complex events and reasoning on results;

the processor being arranged to make a set of proposed actions based on the mining, and to

manage the observed system by applying the proposed actions to the system, while raw events are continuously received from the observed system and monitored wherein the engine is arranged to carry out the steps of monitoring, mining and managing in real-time or near real-time and an input to a step is based on output of a previous step with little or no time delay.

17. A monitor-mine-manage engine as claimed in claim 16 at least partially implemented using hardware logic selected from any one or more of:

a field-programmable gate array, a program-specific integrated circuit, a program-specific standard product, a system-on-a-chip, a complex programmable logic device.

18. An application programming interface comprising programming statements which allow a user to implement a monitor-mine-manage loop, the application programming interface comprising the following entities:

a data source entity which enables the user to express a source of monitoring data to be observed;

a feature extraction entity which allows a user to express a set of features to be extracted from the raw data;

a model application entity which allows a user to apply a model to the complex events created from the feature extraction; and

an actions entity which allows a user to describe management operations on the observed infrastructure.

19. The application programming interface of claim 18, wherein the programming statements are declarative constructs.

20. The application programming interface of claim 18, comprising a monitoring statement which takes as input a type of the component it monitors.