Detection and remediation of memory leaks

• US 10,289,347 B2
• Filed: 10/26/2016
• Issued: 05/14/2019
• Est. Priority Date: 04/26/2016
• Status: Active Grant

• Current Assignee: ServiceNow Incorporated
• Original Assignee: ServiceNow Incorporated
• Inventors: Mangione-Tran, Carmine
• Primary Examiner(s): Bonzo, Bryce P
• Assistant Examiner(s): Gibson, Jonathan D
• Application Number:
  US15/334,382
• Publication Number:
  US 20170308449A1
• Time in Patent Office: 930 Days
• CPC Class Codes: G06F 11/0778 Dumping, i.e. gathering error/state information after a fault for later diagnosis
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 12/00 Accessing, addressing or allocating within memory systems or architectures digital input from, or digital output to record carriers, e.g. to disk storage units, G06F3/06
• CPC Class Codes: G06F 12/0253 Garbage collection, i.e. reclamation of unreferenced memory
• CPC Class Codes: G06F 2212/7205 Cleaning, compaction, garbage collection, erase control
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: G06F 9/5022 Mechanisms to release resources

Abstract

Detecting and remediating memory leaks associated with an application environment can include monitoring allocations of memory from a managed memory space to respective operations to produce memory allocation data and monitoring deallocations of memory to at least some of the respective operations to produce memory deallocation data. A trend in memory leakage can be determined based on samples of the memory allocation or deallocation data. A projection of future memory usage by operations associated with the trend can be determined using binned sets of the memory allocation data and the memory deallocation data. A predicted time at which memory usage by the operations associated with the trend is expected to exceed a threshold can be determined using the projection of future memory usage. A remediation action can be performed before the predicted time to prevent a memory constraint from occurring with respect to the application environment. The application environment can be configured to automatically allocate unallocated portions of memory to execute operations and periodically deallocate unused allocated memory.

Claims

1. A system for detecting and remediating memory leaks associated with an application environment, the system comprising:

a memory configured to store data and one or more processor-executable routines; and

a processor configured to communicate with the memory and to execute the routines stored therein, wherein the routines, when executed, cause the system to;

monitor allocations of memory from a managed memory space to respective operations to produce memory allocation data, wherein the managed memory space is provisioned to the application environment;

monitor deallocations of memory to at least some of the respective operations to produce memory deallocation data;

determine that samples of memory allocation data and memory deallocation data observed over a time interval indicate a trend in memory leakage or a trend change in memory leakage;

determine a projection of future memory usage by operations associated with the trend using binned sets of the memory allocation data and the memory deallocation data;

determine a predicted time at which memory usage by the operations associated with the trend is expected to exceed a threshold using the projection of future memory usage; and

perform a remediation action before the predicted time, wherein the remediation action comprises;

terminating an executing operation within the application environment in response to a determination that the projection of future memory usage comprises a fast memory leak, wherein the executing operation is the oldest executing operation or the executing operation is using the most amount of memory in the application environment;

or

terminating an executing operation within the application environment in response to a determination that the projection of future memory usage comprises a slow memory leak, wherein the executing operation executes throughout the trend.

2. The system of claim 1, wherein the routines, when executed, cause the system to:

identify one or more operation runs and a rate of change of memory usage by operations of the one or more operation runs based on a binned set of the memory allocation data and the memory deallocation data;

apply a regression analysis to the one or more operation runs to determine a slope corresponding to the trend of memory usage for the operations of the one or more operation runs; and

project the rate of change of memory usage based on the trend of memory usage to determine that a memory constraint is expected to occur with respect to the application environment at the predicted time.

3. The system of claim 2, wherein the routines to identify the one or more operation runs, when executed, cause the system to:

determine a distribution of samples of the memory allocation data and the memory deallocation data;

determine whether the distribution of samples of memory usage is random; and

if the distribution is random, remove samples of the distribution of samples from the binned set of the memory allocation data and the memory deallocation data.

4. The system of claim 2, wherein the routines, when executed, cause the system to:

trigger performance of the remediation action responsive to a determination that a current time is less than a threshold amount of time from the predicted time.

5. The system of claim 2, wherein the remediation action includes a reset of a server operating in the application environment, wherein the server executes one or more operations of the respective operations.

6. The system of claim 1, wherein unused allocated memory of the application environment is automatically deallocated by a performance of a garbage collection operation.

7. The system of claim 1, wherein the application environment includes an application server executing the respective operations, the application server operating on a server device.

8. The system of claim 7, wherein the application server is a Java Virtual Machine.

9. A method for detecting and remediating memory leaks associated with an application environment, the method comprising:

monitoring allocations of memory from a managed memory space to respective operations to produce memory allocation data, wherein the managed memory space is provisioned to the application environment;

monitoring deallocations of memory to at least some of the respective operations to produce memory deallocation data;

determining that samples of memory allocation data and memory deallocation data observed over a time interval indicate a trend in memory leakage or a trend change in memory leakage;

determining a projection of future memory usage by operations associated with the trend using binned sets of the memory allocation data and the memory deallocation data;

determining a predicted time at which memory usage by the operations associated with the trend is expected to exceed a threshold using the projection of future memory usage; and

performing a remediation action before the predicted time, wherein the remediation action comprises;

terminating an executing operation within the application environment in response to a determination that the projection of future memory usage comprises a fast memory leak, wherein the executing operation is the oldest executing operation or the executing operation is using the most amount of memory in the application environment;

or

terminating an executing operation within the application environment in response to a determination that the projection of future memory usage comprises a slow memory leak, wherein the executing operation executes throughout the trend.

10. The method of claim 9, the method further comprising:

identifying one or more operation runs and a rate of change of memory usage by operations of the one or more operation runs based on a binned set of the memory allocation data and the memory deallocation data;

applying a regression analysis to the one or more operation runs to determine a slope corresponding to the trend of memory usage for operations of the one or more operation runs; and

projecting the rate of change of memory usage based on the trend of memory usage to determine that a memory constraint is expected to occur with respect to the application environment at the predicted time.

11. The method of claim 10, wherein identifying the one or more operation runs comprises:

determining a distribution of samples of the memory allocation data and the memory deallocation data;

determining whether the distribution of samples of memory usage is random; and

if the distribution is random, removing samples of the distribution of samples from the binned set of the memory allocation data and the memory deallocation data.

12. The method of claim 10, the method further comprising:

triggering performance of the remediation action responsive to determining that a current time is less than a threshold amount of time from the predicted time.

13. The method of claim 10, wherein the remediation action includes resetting a server operating in the application environment, wherein the server executes one or more operations of the respective operations.

14. The method of claim 9, wherein unused allocated memory of the application environment is automatically deallocated by performing a garbage collection operation.

15. The method of claim 9, wherein the application environment includes an application server executing the respective operations, the application server operating on a server device.

16. The method of claim 15, wherein the application server is a Java Virtual Machine.

17. A system comprising:

an application environment configured to automatically allocate unallocated memory from a managed memory space to operations executed in the application environment and periodically deallocate unused allocated memory; and

a memory monitor module configured to;

determine that sampled observation points of memory allocation data and memory deallocation data observed over a time interval indicate a trend in memory leakage or a trend change in memory leakage;

determine a predicted time at which a total memory usage of the application environment is expected to exceed a threshold using a projection of future memory usage by operations associated with the trend, the projection of future memory usage determined using binned sets of the sampled observation points; and

perform a remediation action before the predicted time to prevent a memory constraint from occurring within application environment, wherein the remediation action comprises;

terminating an executing operation within the application environment in response to a determination that the projection of future memory usage comprises a fast memory leak, wherein the executing operation is the oldest executing operation or the executing operation is using the most amount of memory in the application environment;

or

terminating an executing operation within the application environment in response to a determination that the projection of future memory usage comprises a slow memory leak, wherein the executing operation executes throughout the trend.

18. The system of claim 17, wherein the memory allocation data is generated responsive to monitoring allocations of memory to respective operations of the application environment, and wherein the memory deallocation data is generated responsive to monitoring deallocations of memory to at least some of the respective operations of the application environment.

19. The system of claim 18, wherein the memory monitor module is further configured to:

identify one or more operation runs and a rate of change of memory usage by operations of the one or more operation runs based on a binned set of the memory allocation data and the memory deallocation data;

apply a regression analysis to the one or more operation runs to determine a slope corresponding to the trend of memory usage for the operations of the one or more operation runs;

project the rate of change of memory usage based on the trend of memory usage to determine that a memory constraint is expected to occur with respect to the application environment at the predicted time; and

trigger performance of the remediation action responsive to a determination that a current time is less than a threshold amount of time from the predicted time.

20. The system of claim 19, wherein the memory monitor module is configured to identify the one or more operation runs by:

determining a distribution of sampled observation points of the memory allocation data and the memory deallocation data;

determining whether the distribution of samples of memory usage is random; and

if the distribution is random, removing the samples from the binned sets of the memory allocation data and the memory deallocation data.

21. The system of claim 19, wherein the remediation action includes a reset of a server operating in the application environment, the server executing one or more operations of the respective operations.