Free memory trending for detecting out-of-memory events in virtual machines

• US 9,720,823 B2
• Filed: 06/18/2015
• Issued: 08/01/2017
• Est. Priority Date: 06/18/2015
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Garvey, Dustin R., Wong, Lik, Urmanov, Aleksey M.
• Primary Examiner(s): DUNCAN, MARC M
• Application Number:
  US14/743,805
• Publication Number:
  US 20160371180A1
• Time in Patent Office: 775 Days
• CPC Class Codes: G06F 11/0712 in a virtual computing platform, e.g. logically partitioned systems
• CPC Class Codes: G06F 11/0781 Error filtering or prioritizing based on a policy defined by the user or on a policy defined by a hardware/software module, e.g. according to a severity level
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1484 involving virtual machines
• CPC Class Codes: G06F 11/301 where the computing system is a virtual computing platform, e.g. logically partitioned systems virtual machines G06F9/45533; logical partitioning of resources G06F9/5077
• CPC Class Codes: G06F 11/302 where the computing system component is a software system
• CPC Class Codes: G06F 11/3075 the data filtering being achieved in order to maintain consistency among the monitored data, e.g. ensuring that the monitored data belong to the same timeframe, to the same system or component
• CPC Class Codes: G06F 11/3423 where the assessed time is active or idle time
• CPC Class Codes: G06F 12/0253 Garbage collection, i.e. reclamation of unreferenced memory
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: G06F 2212/702 Conservative garbage collection
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]

Abstract

The disclosed embodiments provide a system that detects anomalous events in a virtual machine. During operation, the system obtains time-series virtual machine (VM) data including garbage-collection (GC) data collected during execution of a virtual machine in a computer system. Next, the system computes, by a service processor, a time window for analyzing the time-series VM data based at least in part on a working time scale of high-activity patterns in the time-series GC data. The system then uses a trend-estimation technique to analyze the time-series VM data within the time window to determine an out-of-memory (OOM) risk in the virtual machine. Finally, the system stores an indication of the OOM risk for the virtual machine based at least in part on determining the OOM risk in the virtual machine.

Claims

1. A method, comprising:

obtaining time-series virtual machine (VM) data including garbage-collection (GC) data collected during execution of a virtual machine in a computer system;

computing, by a processor, a time window for analyzing the time-series VM data based at least in part on a working time scale of high-activity patterns in the time-series VM data, wherein the time window is computed by multiplying the working time scale by an average time between bursts from a series of high-activity events in the time-series VM data;

using a trend-estimation technique to analyze the time-series VM data within the time window to determine an out-of-memory (OOM) risk in the virtual machine; and

storing an indication of the OOM risk for the virtual machine based at least in part on determining the OOM risk in the virtual machine.

2. The method of claim 1, further comprising:

estimating a remaining time to an OOM event in the virtual machine; and

including the remaining time to the OOM event in the indication.

3. The method of claim 1, wherein computing the time window for analyzing the time-series VM data based at least in part on the working time scale of high-activity patterns in the time-series VM data comprises:

detecting a series of high-activity events in the time-series VM data;

calculating, from the high-activity events, an average time between bursts for the virtual machine; and

computing the time window by multiplying the average time between bursts by the working time scale.

4. The method of claim 1, wherein using the trend-estimation technique to analyze the time-series VM data within the time window to determine the OOM risk in the virtual machine comprises:

estimating a set of features from a free memory metric in the time-series VM data within the time window; and

matching the set of features to an OOM pattern to detect the OOM risk in the virtual machine.

5. The method of claim 4, wherein using the trend-estimation technique to analyze the time-series VM data within the time window to determine the OOM risk in the virtual machine further comprises:

when the OOM pattern is no longer detected in the set of features, detecting a flattened OOM risk in the virtual machine; and

when one or more of the features match a clear condition, clearing the OOM risk.

6. The method of claim 5, wherein the set of features comprise at least one of:

a slope of the free memory metric;

a slope of the free memory metric in a left half of the time window;

a slope of the free memory metric in a right half of the time window; and

a probability of the free memory metric reaching a low level.

7. The method of claim 6, wherein the OOM pattern comprises at least one of:

a statistically significant negative slope of the free memory metric;

a minimum magnitude of the statistically significant negative slope;

a minimum threshold for the probability of the free memory metric reaching the low level; and

a lack of step change in the free memory metric.

8. The method of claim 4, wherein using the trend-estimation technique to analyze the time-series VM data within the time window to determine the OOM risk in the virtual machine further comprises:

when the OOM pattern continues to be identified in the set of features, detecting an increased OOM risk in the virtual machine.

9. The method of claim 4, wherein the OOM pattern comprises one or more rules to be applied to the set of features.

10. The method of claim 1, wherein the time-series VM data comprises at least one of:

a time spent on GC;

a number of GC invocations; and

a free memory metric.

11. An apparatus, comprising:

one or more processors; and

memory storing instructions that, when executed by the one or more processors, cause the apparatus to;

obtain time-series virtual machine (VM) data including garbage-collection (GC) data collected during execution of a virtual machine in a computer system;

compute a time window for analyzing the time-series VM data based at least in part on a working time scale of high-activity patterns in the time-series VM data, wherein the time window is computed by multiplying the working time scale by an average time between bursts from a series of high-activity events in the time-series VM data;

use a trend-estimation technique to analyze the time-series VM data within the time window to determine an out-of-memory (OOM) risk in the virtual machine; and

store an indication of the OOM risk for the virtual machine based at least in part on determining the OOM risk in the virtual machine.

12. The apparatus of claim 11, wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:

estimate a remaining time to an OOM event in the virtual machine; and

include the remaining time to the OOM event in the indication.

13. The apparatus of claim 11, wherein computing the time window for analyzing the time-series VM data based at least in part on the working time scale of high-activity patterns in the time-series VM data comprises:

detecting a series of high-activity events in the time-series VM data;

calculating, from the high-activity events, an average time between bursts for the virtual machine; and

computing the time window by multiplying the average time between bursts by the working time scale.

14. The apparatus of claim 11, wherein using the trend-estimation technique to analyze the time-series VM data within the time window to determine the OOM risk in the virtual machine comprises:

estimating a set of features from a free memory metric in the time-series VM data within the time window; and

matching the set of features to an OOM pattern to detect the OOM risk in the virtual machine.

15. The apparatus of claim 14, wherein using the trend-estimation technique to analyze the time-series VM data within the time window to determine the OOM risk in the virtual machine further comprises:

when the OOM pattern is no longer detected in the set of features, detecting a flattened OOM risk in the virtual machine;

when one or more of the features match a clear condition, clearing the OOM risk; and

when the OOM pattern continues to be identified in the set of features, detecting an increased OOM risk in the virtual machine.

16. The apparatus of claim 15, wherein the set of features comprise at least one of:

a slope of the free memory metric;

a slope of the free memory metric in a left half of the time window;

a slope of the free memory metric in a right half of the time window; and

a probability of the free memory metric reaching a low level.

17. The apparatus of claim 16, wherein the OOM pattern comprises at least one of:

a statistically significant negative slope of the free memory metric;

a minimum threshold for the probability of the free memory metric reaching the low level; and

a lack of step change in the free memory metric.

18. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method, the method comprising:

obtaining time-series virtual machine (VM) data including garbage-collection (GC) data collected during execution of a virtual machine in a computer system;

computing a time window for analyzing the time-series VM data based at least in part on a working time scale of high-activity patterns in the time-series VM data, wherein the time window is computed by multiplying the working time scale by an average time between bursts from a series of high-activity events in the time-series VM data;

using a trend-estimation technique to analyze the time-series VM data within the time window to determine an out-of-memory (OOM) risk in the virtual machine; and

storing an indication of the OOM risk for the virtual machine based at least in part on determining the OOM risk in the virtual machine.

19. The non-transitory computer-readable storage medium of claim 18, wherein computing the time window for analyzing the time-series VM data based at least in part on the working time scale of high-activity patterns in the time-series VM data comprises:

detecting a series of high-activity events in the time-series VM data;

calculating, from the high-activity events, an average time between bursts for the virtual machine; and

computing the time window by multiplying the average time between bursts by the working time scale.

20. The non-transitory computer-readable storage medium of claim 18, wherein using the trend-estimation technique to analyze the time-series VM data within the time window to determine the OOM risk in the virtual machine comprises:

estimating a set of features from a free memory metric in the time-series VM data within the time window; and

matching the set of features to an OOM pattern to detect the OOM risk in the virtual machine.