Systems and methods for detecting and remedying software anomalies

• US 10,635,519 B1
• Filed: 11/30/2017
• Issued: 04/28/2020
• Est. Priority Date: 11/30/2017
• Status: Active Grant

• Current Assignee: Uptake Technologies, Inc.
• Original Assignee: Uptake Technologies, Inc.
• Inventors: Tang, Yuan, Li, Tuo, Herzog, James
• Primary Examiner(s): Kudirka, Joseph R
• Application Number:
  US15/827,987
• Time in Patent Office: 880 Days
• Field of Search: 714 26, 714 37, 714 471
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1476 in neural networks
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484

Abstract

A computing platform may obtain observed data vectors related to the operation of a topology of nodes that represents a software application running on an uncontrolled platform, wherein each observed data vector comprises data values captured for a given set of operating variables at a particular point in time. After obtaining the observed data vectors, the computing platform may apply an anomaly detection model to the observed data vectors and then based on the anomaly detection model, may identify an anomaly in at least one operating variable. In turn, the computing platform may determine whether each identified anomaly is indicative of a problem related to the application, and based on a determination that an identified anomaly is indicative of a problem related to the software application, cause a client station to present a notification.

Claims

1. A computing system comprising:

a network interface;

at least one processor;

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to;

obtain a set of observed data vectors indicating operation of a topology of nodes that represents a given software application while the given software application is running on a first computing platform that is not controlled by a provider of the given software application, wherein each observed data vector in the set comprises data values captured for a given set of operating variables at a particular point in time;

apply an anomaly detection model to the obtained set of observed data vectors, wherein the anomaly detection model is;

(i) defined by applying an unsupervised learning technique to a set of training data vectors indicating operation of the topology of nodes that represents the given software application while the given software application is running on a second computing platform that is controlled by the provider of the given software application, and (ii) configured to evaluate whether a deviation between observed and predicted values for each operating variable of the given set of operating variables is indicative of an anomaly;

based on the anomaly detection model, identify an anomaly in at least one operating variable in the given set of operating variables;

evaluate whether any identified anomaly is indicative of a problem related to the given software application and thereby determine that at least one identified anomaly is indicative of a problem related to the given software application; and

based on the determination that the at least one identified anomaly is indicative of a problem related to the given software application, cause a client station to present a notification indicating the at least one identified anomaly.

2. The computing system of claim 1, wherein the computing system is part of the second computing platform, and wherein the computing system further comprises program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

run an instance of the given software application under conditions intended to simulate normal operation of the given software application;

while running the instance of the given software application, capture data values for the given set of operating variables at various points in time, thereby producing the set of training data vectors, wherein each training data vector in the set comprises data values captured for the given set of operating variables at a particular point in time; and

use the set of training data vectors to define the anomaly detection model.

3. The computing system of claim 2, wherein the unsupervised learning technique comprises a principal component analysis (PCA) technique, and wherein the program instructions that are executable by the at least one processor to cause the computing system to use the set of training data vectors to define the anomaly detection model comprise program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

use the set of training data vectors to define transformed coordinate space comprising a set of principal components; and

use the set of training data vectors to define a set of anomaly thresholds that specify whether the deviation between observed and predicted values for each operating variable of the given set of operating variables is indicative of an anomaly.

4. The computing system of claim 1, wherein the unsupervised learning technique comprises a principal component analysis (PCA) technique, wherein the set of observed data vectors are originally represented in an observed coordinate space, and wherein the anomaly detection model uses the PCA technique to evaluate whether the deviation between observed and predicted values for each operating variable of the given set of operating variables is indicative of an anomaly by:

transforming each observed data vector from the observed coordinate space to a transformed coordinate space comprising a set of principal components;

inversely transforming each observed data vector from the transformed coordinate space back to the observed coordinate space, thereby producing a predictive data vector that corresponds to each observed data vector; and

comparing the observed data vectors to the predictive data vectors to determine whether the deviation between observed and predicted values for each operating variable of the given set of operating variables is indicative of an anomaly.

5. The computing system of claim 1, wherein the unsupervised learning technique comprises a linear regression technique, and wherein the anomaly detection model uses the linear regression technique to evaluate whether the deviation between observed and predicted values for each operating variable of the given set of operating variables is indicative of an anomaly by:

applying linear regression to the set of training data vectors that are indicative of normal operation of the topology of nodes that represents the given software application while the given software application is running on the second computing platform and thereby defining a predictive function; and

comparing each observed data vector to the predictive function to determine whether the deviation between observed and predicted values for each operating variable of the given set of operating variables is indicative of an anomaly.

6. The computing system of claim 1, wherein the program instructions that are executable by the at least one processor to cause the computing system to determine that at least one identified anomaly is indicative of a problem related to the given software application comprise program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing to:

determine that a given identified anomaly corresponds to a given node of the topology that represents the given software application;

evaluate operation of one or more other nodes in the topology that share dependencies with the given node to determine whether the one or more other nodes appear to be anomalous; and

determine that a root cause of the given anomaly is likely a problem with the given node in the given software application if the one or more other nodes do not appear anomalous.

7. The computing system of claim 1, wherein the computing system is part of the second computing platform, and wherein the program instructions that are executable by the at least one processor to cause the computing system to determine that at least one identified anomaly is indicative of a problem related to the given software application comprise program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

run an instance of the given software application at the computing system under conditions that are selected to replicate those that led to a given anomaly being identified; and

determine that the given anomaly is replicated at the computing system.

8. The computing system of claim 1, wherein the program instructions that are executable by the at least one processor to cause the computing system to determine that at least one identified anomaly is indicative of a problem related to the given software application comprise program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

instruct the first computing platform to re-run the given software application under conditions that are selected to replicate those that led to a given anomaly being identified; and

determine that the given anomaly is replicated at the first computing platform.

9. The computing system of claim 1, wherein the client station is associated with an individual responsible for the given software application.

10. The computing system of claim 1, wherein the topology of nodes that represents the given software application comprises at least one source node, at least one processor node, and at least one sink node.

11. The computing system of claim 1, wherein the given set of operating variables comprises at least one respective operating variable for each node in the topology.

12. The computing system of claim 11, wherein the at least one respective operating variable for each node in the topology comprises one or more of a cache-related metric, a cluster-related metric, a virtual-machine metric, a user-interface metric, a database-query-related metric, a failure-type metric, a software-lifecycle metric, or an application-created metric.

13. A method executed by a computing system, the method comprising:

obtaining a set of observed data vectors indicating operation of a topology of nodes that represents a given software application while the given software application is running on a first computing platform that is not controlled by a provider of the given software application, wherein each observed data vector in the set comprises data values captured for a given set of operating variables at a particular point in time;

applying an anomaly detection model to the obtained set of observed data vectors, wherein the anomaly detection model is;

(i) defined by applying an unsupervised learning technique to a set of training data vectors indicating operation of the topology of nodes that represents the given software application while the given software application is running on a second computing platform that is controlled by the provider of the given software application, and (ii) configured to evaluate whether a deviation between observed and predicted values for each operating variable of the given set of operating variables is indicative of an anomaly;

based on the anomaly detection model, identifying an anomaly in at least one operating variable in the given set of operating variables;

evaluating whether any identified anomaly is indicative of a problem related to the given software application and thereby determining that at least one identified anomaly is indicative of a problem related to the given software application; and

based on determining that the at least one identified anomaly is indicative of a problem related to the given software application, causing a client station to present a notification indicating the at least one identified anomaly.

14. The method of claim 13, wherein the computing system is part of the second computing platform, the method further comprising:

running an instance of the given software application under conditions intended to simulate normal operation of the given software application;

while running the instance of the given software application, capturing data values for the given set of operating variables at various points in time, thereby producing the set of training data vectors, wherein each training data vector in the set comprises data values captured for the given set of operating variables at a particular point in time; and

using the set of training data vectors to define the anomaly detection model.

16. The method of claim 13, wherein the unsupervised learning technique comprises a principal component analysis (PCA) technique, wherein the set of observed data vectors are originally represented in an observed coordinate space, and wherein the anomaly detection model uses the PCA technique to evaluate whether the deviation between observed and predicted values for each operating variable of the given set of operating variables is indicative of an anomaly by:

transforming each observed data vector from the observed coordinate space to a transformed coordinate space comprising a set of principal components;

inversely transforming each observed data vector from the transformed coordinate space back to the observed coordinate space, thereby producing a predictive data vector that corresponds to each observed data vector; and

comparing the observed data vectors to the predictive data vectors to determine whether the deviation between observed and predicted values for each operating variable of the given set of operating variables is indicative of an anomaly.

17. The method of claim 13, wherein the unsupervised learning technique comprises a linear regression technique, and wherein the anomaly detection model uses the linear regression technique to evaluate whether the deviation between observed and predicted values for each operating variable of the given set of operating variables is indicative of an anomaly by:

applying linear regression to the set of training data vectors that are indicative of normal operation of the topology of nodes that represents the given software application while the given software application is running on the second computing platform and thereby defining a predictive function; and

comparing each observed data vector to the predictive function to determine whether the deviation between observed and predicted values for each operating variable of the given set of operating variables is indicative of an anomaly.

15. The method of 14, wherein the unsupervised learning technique comprises a principal component analysis (PCA) technique, and wherein using the set of training data vectors to define the anomaly detection model comprises:

using the set of training data vectors to define transformed coordinate space comprising a set of principal components; and

using the set of training data vectors to define a set of anomaly thresholds that specify whether the deviation between observed and predicted values for each operating variable of the given set of operating variables is indicative of an anomaly.

18. A non-transitory computer-readable medium having instructions stored thereon that are executable by at least one processor to cause a computing system to:

obtain a set of observed data vectors indicating operation of a topology of nodes that represents a given software application while the given software application is running on a first computing platform that is not controlled by a provider of the given software application, wherein each observed data vector in the set comprises data values captured for a given set of operating variables at a particular point in time;

apply an anomaly detection model to the obtained set of observed data vectors, wherein the anomaly detection model is;

(i) defined by applying an unsupervised learning technique to a set of training data vectors indicating operation of the topology of nodes that represents the given software application while the given software application is running on a second computing platform that is controlled by the provider of the given software application, and (ii) configured to evaluate whether a deviation between observed and predicted values for each operating variable of the given set of operating variables is indicative of an anomaly;

based on the anomaly detection model, identify an anomaly in at least one operating variable in the given set of operating variables;

evaluate whether any identified anomaly is indicative of a problem related to the given software application and thereby determine that at least one identified anomaly is indicative of a problem related to the given software application; and

based on the determination that the at least one identified anomaly is indicative of a problem related to the given software application, cause a client station to present a notification indicating the at least one identified anomaly.

19. The non-transitory computer-readable storage medium of claim 18, wherein the computing system is part of the second computing platform, and wherein the non-transitory computer-readable storage medium further comprises program instructions that are executable by the at least one processor to cause the computing system to:

run an instance of the given software application under conditions intended to simulate normal operation of the given software application;

while running the instance of the given software application, capture data values for the given set of operating variables at various points in time, thereby producing the set of training data vectors, wherein each training data vector in the set comprises data values captured for the given set of operating variables at a particular point in time; and

use the set of training data vectors to define the anomaly detection model.

20. The non-transitory computer-readable storage medium of claim 18, wherein the program instructions that are executable by the at least one processor to cause the computing system to determine that at least one identified anomaly is indicative of a problem related to the given software application comprise program instructions that are executable by the at least one processor to cause the computing system to:

determine that a given identified anomaly corresponds to a given node of the topology that represents the given software application;

evaluate operation of one or more other nodes in the topology that share dependencies with the given node to determine whether the one or more other nodes appear to be anomalous; and

determine that a root cause of the given anomaly is likely a problem with the given node in the given software application if the one or more other nodes do not appear anomalous.