Managing alarms from distributed applications

• US 10,637,737 B2
• Filed: 03/28/2017
• Issued: 04/28/2020
• Est. Priority Date: 03/28/2017
• Status: Active Grant

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Inventors: Whitner, Richard, Nelson, Ben
• Primary Examiner(s): Yohannes, Tesfay
• Application Number:
  US15/471,597
• Publication Number:
  US 20180287856A1
• Time in Patent Office: 1,127 Days
• Field of Search: 709221, 709224, 370532
• CPC Class Codes: H04L 41/0613 based on the type or category of the network elements
• CPC Class Codes: H04L 41/0681 Configuration of triggering conditions
• CPC Class Codes: H04L 41/0695 the faulty arrangement being the maintenance, administration or management system
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 41/34 Signalling channels for network management communication
• CPC Class Codes: H04L 41/344 Out-of-band transfers
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Provided is a process including: receiving, with one or more processors, an alarm of a monitoring application that is monitoring a distributed application; determining, with one or more processors, a type of the identified computational entity among a plurality of different types of computational entities, wherein at least some of the different types take longer to provision than other types among the plurality of different types; classifying, with one or more processors, the alarm based on the type of the identified computational entity that emitted the alarm or a metric or event upon which the alarm is based; and based on the classification of the alarm, adjusting, with one or more processors, a configuration of the monitoring application or a logical-architecture or network-architecture topology of the distributed application.

Claims

1. A method of classifying alarms of a monitoring application, the method comprising:

receiving, with one or more processors, an alarm of a monitoring application that is monitoring a distributed application, wherein;

the distributed application comprises a plurality of application components,

the plurality of application components execute within a plurality of computational entities, at least some application components executing within different respective computational entities,

the plurality of computational entities includes at least two types of the following types of computational entities;

a container,

a virtual machine,

a non-virtualized computer,

a microkernel, or

a field-programmable gate array,

the plurality of computational entities are executed by a plurality of computers, at least some computers executing more than one computational entity among the plurality of computational entities,

the alarm identifies a computational entity among the plurality of computational entities,

the identified computational entity emitted the alarm or a metric or event upon which the alarm is based, and

the alarm indicates a deterioration in performance of the identified computational entity;

determining, with one or more processors, a type of the identified computational entity among a plurality of different types of computational entities, wherein at least some of the different types take longer to provision than other types among the plurality of different types;

classifying, with one or more processors, the alarm based on the type of the identified computational entity that emitted the alarm and a metric or event upon which the alarm is based; and

based on the classification of the alarm, adjusting, with one or more processors, a configuration of the monitoring application or a logical-architecture or network-architecture topology of the distributed application.

2. The method of claim 1, wherein:

containers are faster to provision than virtual machines;

virtual machines are faster to provision than non-virtualized computers; and

the plurality of computational entities include at least three types of the following types of computational entities;

a container,

a virtual machine,

a non-virtualized operating system,

a microkernel, or

a field-programmable gate array.

3. The method of claim 1, wherein:

classifying the alarm comprises classifying the alarm based on a task performed by an application component executing within the identified computational entity, the task being different from tasks performed by at least some other application components of the distributed application.

4. The method of claim 1, comprising:

before adjusting the configuration of the monitoring application or the logical-architecture or network-architecture topology of the distributed application, determining that a condition indicated by the alarm has persisted for more than a threshold duration of time or with greater than a threshold frequency.

5. The method of claim 1, comprising:

determining that a condition indicated by the alarm has persisted for more than a first duration of time and, in response, sending a first message to a first address corresponding to a first role in an information-technology department; and

determining that the condition indicated by the alarm has persisted for more than a second duration of time and, in response, sending a second message to a second address corresponding to a second role in the information-technology department, the second duration being longer than the first duration, wherein;

adjusting the configuration of the monitoring application or the logical-architecture or network-architecture topology of the distributed application is performed in response to determining that the condition indicated by the alarm has persisted for more than a third duration of time, the third duration of time being longer than the second duration of time.

6. The method of claim 1, comprising:

before adjusting the configuration of the monitoring application or the logical-architecture or network-architecture topology of the distributed application;

selecting a policy corresponding to the alarm from a repository of policies, wherein;

at least some policies in the repository of policies specify that a threshold of the monitoring application is to be adjusted in response to a first type of alarm,

at least some policies in the repository of policies specify that a computational entity is to be terminated in response to a second type of alarm, and

at least some policies in the repository of policies specify that another instance of a computational entity is to be provisioned in response to a third type of alarm; and

determining a type of adjustment based on the selected policy.

7. The method of claim 1, comprising:

obtaining a log of previous alarms and events indicating actions taken to respond to the previous alarms, the log labeling each previous alarm as being paired with the corresponding events indicating actions taken to respond;

training a supervised machine learning model based on a first subset of the log;

cross-validating the trained machine learning model with a second subset of the log different at least in part from the first subset; and

selecting an adjustment responsive to the alarm based on the cross-validated, trained machine learning model.

8. The method of claim 1, wherein:

the classification either is an input to, or an output of, the trained machine learning model;

training the supervised machine learning model comprises;

determining a plurality of candidate trained models by repeatedly;

selecting initial parameters of the model that differ, at least in part, from initial parameters of other repetitions; and

iteratively, until a termination condition occurs, adjusting current parameters of the machine learning model to reduce an aggregate amount of disagreement between an output of the model and at least some of the logged events indicating actions taken; and

selecting a candidate model based on respective aggregate amounts of disagreement between the outputs of the candidate models and at least some of the logged events.

9. The method of claim 1, wherein:

classifying the alarm comprises classifying the alarm based on a priority indicator associated with at task performed by the identified computational entity or based on a priority indicator associated with the identified computational entity.

10. The method of claim 1, wherein:

classifying the alarm comprises classifying the alarm based on a type of computing environment associated with the distributed application, the type of computing environment indicating whether the distributed application is a production instance of the distributed application or a development instance of the distributed application.

11. The method of claim 1, wherein:

adjusting comprises adjusting a configuration of an agent of the monitoring application executing within the identified computational entity.

12. The method of claim 1, wherein:

adjusting comprises adjusting a configuration of the identified computational entity;

or

adjusting comprises terminating the identified computational entity.

13. The method of claim 1, wherein the classification or the adjustment is based on an alarm score determined with steps for scoring alarms.

14. The method of claim 1, comprising:

steps for monitoring infrastructure;

or

steps for monitoring application performance.

18. The method of claim 1, comprising:

before adjusting the configuration of the monitoring application or the logical-architecture or network-architecture topology of the distributed application;

selecting a policy corresponding to the alarm from a repository of policies, wherein at least some policies in the repository of policies specify that a computational entity is to be terminated and a new computational entity is spun up in response to an alarm.

19. The method of claim 1, wherein adjusting the configuration of the monitoring application or the logical-architecture or network-architecture topology of the distributed application is determined based upon a consolidated supervised machine learning model using a history of previous alarms and their classification.

15. A tangible, non-transitory, machine-readable medium storing instructions that when executed by one or more computers effectuate operations comprising:

receiving, with one or more processors, an alarm of a monitoring application that is monitoring a distributed application, wherein;

the distributed application comprises a plurality of application components,

the plurality of application components execute within a plurality of computational entities, at least some application components executing within different respective computational entities,

the plurality of computational entities includes at least two types of the following types of computational entities;

a container,

a virtual machine,

a non-virtualized computer,

a microkernel, or

a field-programmable gate array,

the plurality of computational entities are executed by a plurality of computers, at least some computers executing more than one computational entity among the plurality of computational entities,

the alarm identifies a computational entity among the plurality of computational entities,

the identified computational entity emitted the alarm or a metric or event upon which the alarm is based, and

the alarm indicates a deterioration in performance of the identified computational entity;

determining, with one or more processors, a type of the identified computational entity among a plurality of different types of computational entities, wherein at least some of the different types take longer to provision than other types among the plurality of different types;

classifying, with one or more processors, the alarm based on the type of the identified computational entity that emitted the alarm and a metric or event upon which the alarm is based; and

based on the classification of the alarm, adjusting, with one or more processors, a configuration of the monitoring application or a logical-architecture or network-architecture topology of the distributed application.

16. The medium of claim 15, wherein:

classifying the alarm comprises classifying the alarm based on a task performed by an application component executing within the identified computational entity, the task being different from tasks performed by at least some other application components of the distributed application.

17. The medium of claim 15, comprising:

before adjusting the configuration of the monitoring application or the logical-architecture or network-architecture topology of the distributed application, determining that a condition indicated by the alarm has persisted for more than a threshold duration of time or with greater than a threshold frequency.

20. A tangible, non-transitory, machine-readable medium storing instructions that when executed by one or more computers effectuate operations comprising:

receiving, with one or more processors, an alarm of a monitoring application that is monitoring a distributed application, wherein;

the distributed application comprises a plurality of application components,

the plurality of application components execute within a plurality of computational entities, at least some application components executing within different respective computational entities,

the plurality of computational entities includes at least one type of the following types of computational entities;

a container,

a virtual machine,

a non-virtualized computer,

a microkernel, or

a field-programmable gate array,

the plurality of computational entities are executed by a plurality of computers, at least some computers executing more than one computational entity among the plurality of computational entities,

the alarm identifies a computational entity among the plurality of computational entities,

the identified computational entity emitted the alarm or a metric or event upon which the alarm is based, and

the alarm indicates a deterioration in performance of the identified computational entity;

determining, with one or more processors, a type of the identified computational entity among a plurality of different types of computational entities, wherein at least some of the different types take longer to provision than other types among the plurality of different types;

classifying, with one or more processors, the alarm based on the type of the identified computational entity that emitted the alarm and a metric or event upon which the alarm is based;

based on the classification of the alarm, adjusting, with one or more processors, a configuration of the monitoring application or a logical-architecture or network-architecture topology of the distributed application;

before adjusting the configuration of the monitoring application or the logical-architecture or network-architecture topology of the distributed application;

selecting a policy corresponding to the alarm from a repository of policies, wherein;

at least some policies in the repository of policies specify that a threshold of the monitoring application is to be adjusted in response to a first type of alarm,

at least some policies in the repository of policies specify that a computational entity is to be terminated in response to a second type of alarm, and

at least some policies in the repository of policies specify that another instance of a computational entity is to be provisioned in response to a third type of alarm; and

determining a type of adjustment based on the selected policy;

obtaining a log of previous alarms and events indicating actions taken to respond to the previous alarms, the log labeling each previous alarm as being paired with the corresponding events indicating actions taken to respond;

training a supervised machine learning model based on a first subset of the log;

cross-validating the trained machine learning model with a second subset of the log different at least in part from the first subset; and

selecting an adjustment responsive to the alarm based on the cross-validated, trained machine learning model.