Stateless detection of out-of-memory events in virtual machines

• US 10,248,561 B2
• Filed: 06/18/2015
• Issued: 04/02/2019
• Est. Priority Date: 06/18/2015
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Garvey, Dustin R., Salunke, Sampanna S., Wong, Lik, Gao, Xuemei, Zhang, Yongqiang, Chan, Eric S., Gross, Kenny C.
• Primary Examiner(s): Pyo, Monica M
• Application Number:
  US14/743,817
• Publication Number:
  US 20160371181A1
• Time in Patent Office: 1,384 Days
• Field of Search: 707813, 711132
• CPC Class Codes: G06F 11/30 Monitoring
• CPC Class Codes: G06F 11/3072 where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06F 12/0253 Garbage collection, i.e. reclamation of unreferenced memory
• CPC Class Codes: G06F 2201/815 Virtual
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0671 In-line storage system

Abstract

The disclosed embodiments provide a system that detects anomalous events in a virtual machine. During operation, the system obtains time-series garbage-collection (GC) data collected during execution of a virtual machine in a computer system. Next, the system generates one or more seasonal features from the time-series GC data. The system then uses a sequential-analysis technique to analyze the time-series GC data and the one or more seasonal features for an anomaly in the GC activity of the virtual machine. Finally, the system stores an indication of a potential out-of-memory (OOM) event for the virtual machine based at least in part on identifying the anomaly in the GC activity of the virtual machine.

Claims

1. A method of modifying memory allocations of virtual machines executing within a computing system, comprising:

executing one or more virtual machines, including a first virtual machine, on a computing system, wherein each said virtual machine is allocated an amount of heap memory from the computer system,

performing a plurality of garbage collection processes on the computing system during execution of the first virtual machine on the computer system;

determining, by the computing system, using one or more processors of the computing system, one or more execution metrics for each of the plurality of garbage collection processes performed, the execution metrics including an execution duration for each of the plurality of garbage collection process,

using a sequential-analysis technique to analyze, by the computing system, the execution metrics for each of the plurality of garbage collection processes for stateless detection of a memory usage trend for the first virtual machine, based on the execution durations for the plurality of garbage collection processes; and

in response to detecting an upward memory usage trend for the first virtual machine based on the analysis of the execution durations for the plurality of garbage collection processes, modifying, by the computing system, the amount of heap memory allocated to the first virtual machine.

2. The method of claim 1, further comprising:

dividing the execution metrics for each of the plurality of garbage collection processes into a training set and a test set; and

validating the training set and the test set prior to analyzing the execution metrics the plurality of garbage collection processes.

3. The method of claim 2, wherein validating the training set and the test set comprises:

using the training set to verify a minimum workload on the first virtual machine; and

using the test set to verify a minimum level of garbage collection activity in the first virtual machine.

4. The method of claim 2, wherein the training set comprises a first subset of the execution metrics for the garbage collection processes that is collected prior to a second subset of the execution metrics for the garbage collection processes in the test set.

5. The method of claim 1, wherein analyzing the execution metrics for the garbage collection processes comprises determining one or more seasonal features for the execution metrics for the garbage collection processes of the first virtual machine comprises:

using the one or more seasonal features to obtain a seasonal adjustment of the execution metrics for the plurality of garbage collection processes; and

applying the sequential-analysis technique to the seasonal adjustment to test the detection of the upward memory usage trend for the first virtual machine.

6. The method of claim 1, wherein the execution metrics for each of the plurality of garbage collection processes performed further comprises:

a heap size value after each of the plurality of garbage collection processes.

7. The method of claim 1, wherein the sequential-analysis technique comprises a sequential probability ratio test (SPRT).

8. The method of claim 1, further comprising transmitting an indication of a potential out-of-memory (OOM) event for the first virtual machine, in response to detecting an upward memory usage trend for the first virtual machine, wherein the indication is an alert that is transmitted to a recipient that is registered to monitor alerts for the first virtual machine.

9. The method of claim 1, wherein modifying the amount of heap memory allocated to the first virtual machine comprises restarting the execution of the first virtual machine on the computing system.

10. The method of claim 1, wherein modifying the amount of heap memory allocated to the first virtual machine comprises increasing the allocation of heap memory from the computing system to the first virtual machine.

11. The method of claim 1, wherein the execution metrics for each of the plurality of garbage collection processes performed further comprises a type of each of the plurality of garbage collection processes, and wherein the detection of the memory usage trend for the first virtual machine is based on (a) the execution durations for the plurality of garbage collection processes, and (b) the types of the plurality of garbage collection processes.

12. The method of claim 1, wherein the execution metrics for each of the plurality of garbage collection processes performed further comprises a type of each of the plurality of garbage collection processes and a heap size value after each of the plurality of garbage collection processes, and

wherein the detection of the memory usage trend for the first virtual machine is based on (a) the execution durations for the plurality of garbage collection processes, and (b) the types of the plurality of garbage collection processes.

13. A computing system for modifying memory allocations of virtual machines executing within the computing system, comprising:

one or more processors; and

memory storing instructions that, when executed by the one or more processors, cause the computing system to;

execute one or more virtual machines, including a first virtual machine, on the computing system, wherein each said virtual machine is allocated an amount of heap memory from the computer system;

perform a plurality of garbage collection processes on the computing system during execution of the first virtual machine on the computer system;

determine one or more execution metrics for each of the plurality of garbage collection processes performed, the execution metrics including an execution duration for each of the plurality of garbage collection process;

use a sequential-analysis technique to analyze, by the computer system, the execution metrics for each of the plurality of garbage collection processes for stateless detection of a memory usage trend for the first virtual machine, based on the execution durations for the plurality of garbage collection processes; and

in response to detecting an upward memory usage trend for the first virtual machine based on the analysis of the execution durations for the plurality of garbage collection processes, modify the amount of heap memory allocated to the first virtual machine.

14. The computing system of claim 13, wherein the memory further stores instructions that, when executed by the one or more processors, cause the computing system to:

divide the execution metrics for each of the plurality of garbage collection processes into a training set and a test set; and

validate the training set and the test set prior to analyzing the execution metrics the plurality of garbage collection processes.

15. The computing system of claim 14, wherein validating the training set and the test set comprises:

using the training set to verify a minimum workload on the first virtual machine; and

using the test set to verify a minimum level of garbage collection activity in the first virtual machine.

16. The computing system of claim 13, wherein analyzing the execution metrics for the garbage collection processes comprises determining one or more seasonal features for the execution metrics for the garbage collection processes of the first virtual machine comprises:

using the one or more seasonal features to obtain a seasonal adjustment of the execution metrics for the plurality of garbage collection processes; and

applying the sequential-analysis technique to the seasonal adjustment to test the detection of the upward memory usage trend for the first virtual machine.

17. The computing system of claim 13, wherein the execution metrics for each of the plurality of garbage collection processes performed further comprises:

a heap size value after each of the plurality of garbage collection processes.

18. The computing system of claim 13, wherein the sequential-analysis technique comprises a sequential probability ratio test (SPRT).

19. A non-transitory computer-readable storage medium storing instructions that when executed by a computer system cause the computer system to modify memory allocations of virtual machines executing within the computing system by:

executing one or more virtual machines, including a first virtual machine, on the computing system, wherein each said virtual machine is allocated an amount of heap memory from the computer system;

performing a plurality of garbage collection processes on the computing system during execution of the first virtual machine on the computer system;

determining, by the computer system, one or more execution metrics for each of the plurality of garbage collection processes performed, the execution metrics including an execution duration for each of the plurality of garbage collection process,

using a sequential-analysis technique to analyze the execution metrics for each of the plurality of garbage collection processes for stateless detection of a memory usage trend for the first virtual machine, based on the execution durations for the plurality of garbage collection processes; and

in response to detecting an upward memory usage trend for the first virtual machine based on the analysis of the execution durations for the plurality of garbage collection processes, modify the amount of heap memory allocated to the first virtual machine.

20. The non-transitory computer-readable storage medium of claim 19, wherein analyzing the execution metrics for the garbage collection processes comprises determining one or more seasonal features for the execution metrics for the garbage collection processes of the first virtual machine comprises:

using the one or more seasonal features to obtain a seasonal adjustment of the execution metrics for the plurality of garbage collection processes; and

applying the sequential-analysis technique to the seasonal adjustment to test the detection of the upward memory usage trend for the first virtual machine.