Systems and methods for detecting behavioral features of software application deployments for automated deployment management

• US 7,954,090 B1
• Filed: 12/21/2005
• Issued: 05/31/2011
• Est. Priority Date: 12/21/2004
• Status: Active Grant

• Current Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Original Assignee: Zenprise, Inc. (Cloud Software Group)
• Inventors: Hassan, Tanvir, Roach, Kelly Brian, Qureshi, Waheed
• Primary Examiner(s): Puente; Emerson C
• Assistant Examiner(s): MILLS, PAUL V
• Application Number:
  US11/316,396
• Time in Patent Office: 1,987 Days
• Field of Search: None
• US Class Current: 717/127
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06N 5/048 Fuzzy inferencing

Abstract

A system for monitoring and analyzing a deployment of a software application is disclosed. The system includes a monitoring component that collects streams of data values of time-variant state metrics associated with the deployment, and a feature detector that analyzes the streams to identify features known to be associated with potential problems. Detected features can include deviations of a signal from a normalcy band, a rising or descending trend in signal behavior, deviations from seasonal or cyclical normalcy bands, and the like.

Claims

1. A system for monitoring and analyzing a deployment of a software application, comprising:

a monitor for gathering time-variant state metric data from the deployment, the monitor associated with a computer system that runs components of the software application, the monitor configured to gather state metric data for at least one deployment state metric from the computer system; and

a non-transitory computer-readable medium that stores a numerical chart feature detector that, when executed by a processor, analyzes a numerical telemetry signal that comprises state metric data gathered by the monitor, said numerical chart feature detector configured to use a threshold and a prescribed sustain period to detect the presence of a feature based on the numerical telemetry signal, the feature representing a condition or behavior of the software application that is known to be associated with a potential problem of the deployment of the software application;

wherein the threshold comprises a zero-confidence level representing substantially zero confidence in the existence of a feature, a trigger-confidence level representing a prescribed intermediate confidence in the existence of a feature, and a full-confidence level representing a substantially maximum confidence in the existence of a feature;

wherein the numerical chart feature detector is configured to detect a feature when the telemetry signal exceeds the trigger-confidence level and remains in excess of the zero-confidence level subsequently throughout the prescribed sustain period immediately after exceeding the trigger-confidence level;

wherein an occurrence of the feature is represented by at least a name, confidence value, and time of occurrence.

2. The system of claim 1, wherein the numerical chart feature detector is capable of detecting multiple features after the telemetry signal exceeds the trigger-confidence level and before the telemetry signal falls below the zero-confidence level, each feature having a respective confidence value.

3. The system of claim 1, wherein the sustain period is a prescribed sustain time representing a minimum time required to sustain a positive confidence deviation before the feature is detected.

4. The system of claim 1, wherein the sustain period is a prescribed number of sustain samples comprising the minimum number of points required to sustain a positive confidence deviation before the feature is detected.

5. The system of claim 1, wherein the system is configured to compute at least the zero-confidence level and trigger-confidence level statistically from moving statistics determined from signal history.

6. The system of claim 5, wherein the system is configured to use an exponentially weighted moving average to obtain moving mean and standard deviation statistics for computing at least the zero-confidence level and trigger-confidence level.

7. The system of claim 1, wherein the zero-confidence level, trigger-confidence level, and full-confidence level are absolutely prescribed.

8. The system of claim 1, wherein the numerical chart feature detector uses both a low threshold and a high threshold, each of which includes a respective zero-confidence level, trigger-confidence level, and full-confidence level.

9. The system of claim 1, wherein the system is configured to use a smoothing average window to preprocess the numerical telemetry signal prior to analysis of the numerical telemetry signal by the numerical chart feature detector.

10. The system of claim 1, wherein the numerical chart feature detector is a control chart feature detector.

11. The system of claim 1, wherein the numerical chart feature detector is a trend chart feature detector that passes the derivative of a smoothed input signal into a control chart feature detector.

12. The system of claim 11, wherein the numerical chart feature detector detects a rising or descending trend in the behavior of the numerical telemetry signal.

13. The system of claim 1, wherein the numerical chart feature detector comprises a seasonal control chart feature detector that uses moving statistics that are based at least partly on a periodic distance function and kernel smoothing.

14. The system of claim 13, wherein the telemetry signal fluctuates approximately cyclically and the seasonal control chart feature detector computes a cyclically varying normalcy band of the telemetry signal.

15. The system of claim 1, wherein the system increases and decreases monitoring of time-variant state metrics based at least in part on whether the feature is detected by the numerical chart feature detector.

16. The system of claim 1, wherein the system comprises analysis subsystem monitors that are configured to use postprocessors to apply filtering, aggregation, and/or transformation to the collected state metric data to generate a telemetry signal to the numerical chart feature detector.

17. The system of claim 1, further comprising a problem detector that diagnoses problems with the deployment, at least in part, by programmatically using features of the deployment detected by the numerical chart feature detector.

18. The system of claim 1, further comprising an automation subsystem that automatically selects remedies and executes the remedies to resolve problems with the deployment, at least in part by programmatically using features of the deployment detected by the numerical chart feature detector.

19. The system of claim 1, further comprising a graphical user interface that displays data regarding features detected by the numerical chart feature detector.

20. A method of monitoring and analyzing a deployment of a software application, comprising:

receiving a numerical telemetry signal that comprises time-variant state metric data collected by a monitor that runs on a computer of said deployment; and

analyzing the numerical telemetry signal to determine whether a feature exists, the feature representing a condition or behavior of the software application that is known to be associated with a potential problem of the deployment;

wherein analyzing the numerical telemetry signal comprises detecting the feature when the numerical telemetry signal exceeds a trigger-confidence level and then remains in excess of a zero-confidence level throughout a prescribed sustain period, said zero-confidence level corresponding to a substantially zero confidence in the existence of the feature, said trigger-confidence level corresponding to an intermediate confidence that is between the zero-confidence level and a full-confidence level;

said method performed by a computer system.

21. The method of claim 20, wherein detecting the feature comprises outputting a name, confidence value, and time of occurrence.

22. The method of claim 20, wherein the method comprises detecting multiple features after the numerical telemetry signal exceeds the trigger-confidence level and before the telemetry signal thereafter falls below the zero-confidence level, each feature having a respective confidence value.

23. The method of claim 20, wherein the sustain period is specified in terms of time.

24. The method of claim 20, wherein the sustain period is specified in terms of a number of telemetry signal samples.

25. The method of claim 20, further comprising computing at least the zero-confidence level and trigger-confidence level statistically from moving statistics determined from signal history.

26. The method of claim 20, further comprising using a smoothing average window to preprocess the numerical telemetry signal prior to using said confidence levels to determine whether the feature exists.

27. The method of claim 20, wherein analyzing the numerical telemetry signal further comprises accounting for seasonality by using moving statistics that are based at least partly on a periodic distance function.

28. The method of claim 20, wherein the telemetry signal fluctuates approximately cyclically, and the method comprises computing a cyclically varying normalcy band of the telemetry signal, and using the cyclically varying normalcy band to assess whether the feature exists.

29. The method of claim 20, further comprising causing the collection of telemetry data associated with a monitored state metric to be increased or decreased based on whether the feature is detected.

30. The method of claim 20, further comprising, by said computer system, using features generated from said analysis of the numerical telemetry signal, in combination with other detected features, to determine whether a problem exists in the deployment.

31. The method of claim 30, further comprising automatically selecting and executing a remedy associated with the problem.

32. The method of claim 30, wherein the zero-confidence level and trigger-confidence level vary with time.

33. The method of claim 20, further comprising, by said computer system, using at least features generated from said analysis of the numerical telemetry signal to predict a future problem in the deployment.

34. A non-transitory computer-readable medium that comprises executable code that, when executed by a computer system, causes the computer system to analyze a deployment of a software application by a process that comprises:

receiving a numerical telemetry signal that comprises time-variant state metric data collected by a monitor that runs on a computer of said deployment; and

analyzing the numerical telemetry signal to assess whether a feature exists, the feature representing a condition or behavior of the software application that is associated with a potential deployment problem;

wherein analyzing the numerical telemetry signal comprises detecting the feature when the numerical telemetry signal exceeds a trigger-confidence level and then remains in excess of a zero-confidence level throughout a prescribed sustain period, said zero-confidence level corresponding to a substantially zero confidence in the existence of the feature, said trigger-confidence level corresponding to an intermediate confidence that is between the zero-confidence level and a full-confidence level.

35. The computer-readable medium of claim 34, wherein the executable code directs the computer system to record a feature in association with a name, a confidence value, and a time of occurrence.

36. The computer-readable medium of claim 34, wherein the executable code is capable of causing the computer system to detect multiple features after the numerical telemetry signal exceeds the trigger-confidence level and before the telemetry signal thereafter falls below the zero-confidence level, each feature having a respective confidence value.

37. The computer-readable medium of claim 34, wherein the sustain period is specified in terms of time.

38. The computer-readable medium of claim 34, wherein the sustain period is specified in terms of a number of telemetry signal samples.

39. The computer-readable medium of claim 34, wherein the executable code directs the computer system to compute at least the zero-confidence level and trigger-confidence level statistically from moving statistics determined from a signal history.

40. The computer-readable medium of claim 34, wherein the executable code directs the computer system to use a smoothing average window to preprocess the numerical telemetry signal prior to using confidence levels to assess whether the feature exists.

41. The computer-readable medium of claim 34, wherein the telemetry signal fluctuates approximately cyclically, and the process comprises computing a cyclically varying normalcy band based on the telemetry signal and using the cyclically varying normalcy band to assess whether the feature exists.

42. The computer-readable medium of claim 34, wherein the executable code directs the computer system to cause the collection of telemetry data associated with a monitored state metric to be increased or decreased based at least partly on whether the feature is detected.

43. The computer-readable medium of claim 34, wherein the executable code directs the computer system to use features detected by said analysis of the numerical telemetry signal, in combination with other detected features, to assess whether a problem exists in the deployment.

44. The computer-readable medium of claim 34, wherein the executable code directs the computer system to use features detected by said analysis of the numerical telemetry signal to predict a future problem in the deployment.

45. The computer-readable medium of claim 34, wherein the zero-confidence level and trigger-confidence level vary with time.

46. The computer-readable medium of claim 34, in combination with a computer system that is programmed, via said executable code, to perform said process.