Intelligent services for application dependency discovery, reporting, and management tool

• US 10,642,719 B1
• Filed: 06/27/2019
• Issued: 05/05/2020
• Est. Priority Date: 06/27/2019
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Balasubramanian, Muralidharan, Barnum, Eric K., Dallen, Julie, Watson, David
• Primary Examiner(s): Dao, Thuy
• Application Number:
  US16/454,562
• Time in Patent Office: 313 Days
• CPC Class Codes: G06F 11/302 where the computing system component is a software system
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 11/3668 Software testing software testing in telephone exchanges H04M3/242, testing of hardware G06F11/22
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition

Abstract

Techniques for monitoring operating statuses of an application and its dependencies are provided. A monitoring application may collect and report the operating status of the monitored application and each dependency. Through use of existing monitoring interfaces, the monitoring application can collect operating status without requiring modification of the underlying monitored application or dependencies. The monitoring application may determine a problem service that is a root cause of an unhealthy state of the monitored application. Dependency analyzer and discovery crawler techniques may automatically configure and update the monitoring application. Machine learning techniques may be used to determine patterns of performance based on system state information associated with performance events and provide health reports relative to a baseline status of the monitored application. Also provided are techniques for testing a response of the monitored application through modifications to API calls. Such tests may be used to train the machine learning model.

Claims

1. A computer-implemented method comprising:

configuring a monitoring application to monitor a first application and a plurality of dependencies of the first application using a plurality of monitoring interfaces;

detecting, by the monitoring application and based on the plurality of monitoring interfaces, that the first application has an unhealthy operating status;

collecting, by one or more data collecting agents and based on detecting that the first application has the unhealthy operating status, system state information corresponding to the first application and each of the plurality of dependencies;

storing the collected system state information in a database as a first incident record corresponding to a first incident event and comprising incident attribute information for the first application and each of the plurality of dependencies;

training a machine learning model based on a plurality of incident records including the first incident record, wherein training the machine learning model comprises;

clustering incident events corresponding to each of the plurality of incident records for the first application, wherein clustering the incident events is based on attributes of the system state information corresponding to each of the plurality of dependencies;

determining one or more patterns of performance based on the clustered incident events, wherein a first pattern of performance of the one or more patterns of performance indicates a potential correlation between a first attribute of the system state information corresponding to a first dependency and the first application having the unhealthy operating status; and

updating the machine learning model based on the determined patterns of performance;

detecting, by the monitoring application and based on the plurality of monitoring interfaces, a current operating status of the first application and the plurality of dependencies; and

generating, using the machine learning model and based on the first pattern of performance and the current operating status, a recommendation regarding operation of the first application or the first dependency.

2. The method of claim 1, wherein generating the recommendation regarding the operation of the first application or the first dependency comprises:

determining, using the machine learning model, a suggested action based on incident records corresponding to the first pattern of performance.

3. The method of claim 2, wherein the first incident record further comprises information indicating a corrective action taken in response to the first incident event, and wherein determining the suggested action based on incident records corresponding to the first pattern of performance comprises:

determining, using the machine learning model, the suggested action based on the corrective action taken in response to the first incident event.

4. The method of claim 2, wherein generating the recommendation regarding the operation of the first application or the first dependency further comprises:

generating a user notification regarding the suggested action.

5. The method of claim 2, wherein generating the recommendation regarding the operation of the first application or the first dependency further comprises:

automatically implementing, by the monitoring application, the suggested action.

6. The method of claim 2, wherein generating the recommendation regarding the operation of the first application or the first dependency further comprises:

generating a user notification regarding a first portion of the suggested action; and

automatically implementing, by the monitoring application, a second portion of the suggested action.

7. The method of claim 2, wherein the suggested action comprises bypassing the first dependency.

8. The method of claim 1, wherein the first dependency corresponds to an Application Programming Interface (API) associated with a resource utilized by the first application.

9. The method of claim 1, wherein the first dependency corresponds to a network utilized by the first application to communicate with another dependency.

10. The method of claim 1, wherein the first application is determined to have an unhealthy status based on whether one or more metrics associated with the first application satisfy one or more operating status thresholds.

11. The method of claim 1, wherein the incident attribute information of the first incident record and corresponding to the first dependency comprises information indicating one or more of:

whether a resource associated with the first dependency is accessible;

a response latency associated with requests to the first dependency;

an error rate associated with requests to the first dependency;

or

an error state or error message provided by the first dependency.

12. The method of claim 1, wherein the plurality of monitoring interfaces comprises a first monitoring interface configured to enable monitoring of the first dependency,

wherein the first monitoring interface is generated by a monitoring interface application and is configured to determine at least one metric associated with the first dependency, and

wherein configuring the monitoring application to monitor the first application and the plurality of dependencies comprises configuring the monitoring application to utilize the first monitoring interface through at least one monitoring query associated with the monitoring interface application.

13. The method of claim 1, wherein the first pattern of performance is a pattern of failure and indicates a potential correlation between the first attribute of the system state information corresponding to the first dependency and the first application entering the unhealthy operating status.

14. The method of claim 1, wherein the first pattern of performance is a pattern of risk and indicates a potential correlation between the first attribute of the system state information corresponding to the first dependency and a level of security risk to the first application.

15. The method of claim 1, wherein the first pattern of performance is a pattern of latency and indicates a potential correlation between the first attribute of the system state information corresponding to the first dependency and a latency associated with requests to the first application.

16. The method of claim 1, wherein the first incident record further comprises timing information associated with the first incident event, and wherein determining the first pattern of performance is based on the timing information associated with the first incident event and timing information associated with other incident events.

17. A system comprising:

a first application having a plurality of dependencies, wherein a first dependency of the plurality of dependencies comprises an Application Programming Interface (API) utilized by the first application;

a monitoring interface application providing a plurality of monitoring interfaces, wherein a first monitoring interface of the plurality of monitoring interfaces is configured to retrieve operating status information for the first application and a second monitoring interface of the plurality of monitoring interfaces is configured to retrieve operating status information for the first dependency;

a database configured to store a plurality of incident records associated with the first application; and

a monitoring device implementing a monitoring application and comprising one or more processors and memory storing instructions that, when executed by the one or more processors, cause the monitoring device to;

configure the monitoring application to monitor the first application and the plurality of dependencies of the first application using the plurality of monitoring interfaces;

detect, based on the plurality of monitoring interfaces, that the first application has an unhealthy operating status;

collect, by one or more data collecting agents and based on detecting that the first application has the unhealthy operating status, system state information corresponding to the first application and each of the plurality of dependencies;

store the collected system state information in the database as a first incident record corresponding to a first incident event and comprising incident attribute information for the first application and each of the plurality of dependencies;

train a machine learning model based on a plurality of incident records including the first incident record, wherein the instructions cause the monitoring device to train the machine learning model by causing the monitoring device to;

cluster incident events corresponding to each of the plurality of incident records for the first application based on attributes of the system state information corresponding to each of the plurality of dependencies;

determine one or more patterns of performance based on the clustered incident events, wherein a first pattern of performance of the one or more patterns of performance indicates a potential correlation between a first attribute of the system state information corresponding to a first dependency and the first application having the unhealthy operating status; and

update the machine learning model based on the determined patterns of performance;

detect, based on the plurality of monitoring interfaces, a current operating status of the first application and the plurality of dependencies; and

generate, using the machine learning model and based on the first pattern of performance and the current operating status, a recommendation regarding operation of the first application or the first dependency.

18. The system of claim 17, wherein the first incident record further comprises information indicating a corrective action taken in response to the first incident event, and wherein the instructions cause the monitoring device to generate the recommendation regarding the operation of the first application or the first dependency by causing the monitoring device to:

determine, using the machine learning model, a suggested action based on incident records corresponding to the first pattern of performance.

19. The system of claim 17, wherein the instructions cause the monitoring device to configure the monitoring application to monitor the first application and the plurality of dependencies by causing the monitoring device to:

configure the monitoring application to utilize the first monitoring interface through at least one monitoring query associated with the monitoring interface application.

20. One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause a monitoring device to perform steps comprising:

configuring a monitoring application to monitor a first application and a plurality of dependencies of the first application using a plurality of monitoring interfaces, wherein the plurality of monitoring interfaces comprises;

a first monitoring interface configured to determine incident attribute information associated with the first application; and

a second monitoring interface configured to determine incident attributed information associated with a first dependency of the plurality of dependencies;

detecting, by the monitoring application and based on the plurality of monitoring interfaces, that the first application has an unhealthy operating status;

collecting, by one or more data collecting agents and based on detecting that the first application has the unhealthy operating status, system state information corresponding to the first application and each of the plurality of dependencies;

storing the collected system state information in a database as a first incident record corresponding to a first incident event and comprising incident attribute information for the first application and each of the plurality of dependencies;

updating the first incident record to indicate a corrective action taken in response to the first application having the unhealthy state;

training a machine learning model based on a plurality of incident records including the first incident record, wherein training the machine learning model comprises;

clustering incident events corresponding to each of the plurality of incident records for the first application, wherein clustering the incident events is based on attributes of the system state information corresponding to each of the plurality of dependencies;

determining one or more patterns of performance based on the clustered incident events, wherein a first pattern of performance of the one or more patterns of performance indicates a potential correlation between a first attribute of the system state information corresponding to the first dependency and the first application having the unhealthy operating status; and

updating the machine learning model based on the determined patterns of performance;

detecting, by the monitoring application and based on the plurality of monitoring interfaces, a current operating status of the first application and the plurality of dependencies; and

determining, using the machine learning model and based on the first pattern of performance and the current operating status, a suggested action based on the corrective action taken in response to the first incident event.