Correlation of thread intensity and heap usage to identify heap-hoarding stack traces

• US 10,534,643 B2
• Filed: 05/05/2017
• Issued: 01/14/2020
• Est. Priority Date: 05/09/2016
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Chan, Eric S.
• Primary Examiner(s): Swift, Charles M
• Application Number:
  US15/588,531
• Publication Number:
  US 20170337085A1
• Time in Patent Office: 984 Days
• CPC Class Codes: G06F 11/3037 where the computing system component is a memory, e.g. virtual memory, cache accessing, addressing or allocating within memory systems or architectures G06F12/00; checking stores for correct operation G11C29/00
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3442 for planning or managing the needed capacity
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 11/3471 Address tracing
• CPC Class Codes: G06F 11/36 Preventing errors by testing or debugging software
• CPC Class Codes: G06F 11/3612 by runtime analysis performance monitoring G06F11/3466
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5005 to service a request
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: G06F 9/5022 Mechanisms to release resources
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5044 considering hardware capabilities
• CPC Class Codes: G06F 9/505 : considering the load
• CPC Class Codes: G06F 9/5055 : considering software capabilities, i.e. software resources associated or available to the machine
• CPC Class Codes: G06F 9/5083 : Techniques for rebalancing the load in a distributed system
• CPC Class Codes: H04L 41/5009 : Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]

Abstract

Embodiments identify heap-hoarding stack traces to optimize memory efficiency. Some embodiments can determine a length of time when heap usage by processes exceeds a threshold. Some embodiments may then determine heap information of the processes for the length of time, where the heap information comprise heap usage information for each interval in the length of time. Next, some embodiments can determine thread information of the one or more processes for the length of time, wherein determining the thread information comprises determining classes of threads and wherein the thread information comprises, for each of the classes of threads, thread intensity information for each of the intervals. Some embodiments may then correlate the heap information with the thread information to identify code that correspond to the heap usage exceeding the threshold. Some embodiments may then initiate actions associated with the code.

Claims

1. A computer-implemented method comprising:

determining, by one or more computer systems, a length of time when heap usage by one or more processes exceeds a threshold, wherein the length of time spans at least one period, wherein the at least one period is divided into a plurality of intervals, and wherein each of the plurality of intervals is mapped to a season of a plurality of seasons;

determining heap information of the one or more processes during the length of time, the heap information comprising heap usage information and a heap usage seasonal factor for each of the plurality of intervals in the length of time, wherein the heap usage seasonal factor for each of the plurality of intervals corresponds to a smoothed heap usage seasonal factor associated with the season to which each of the plurality of intervals is mapped;

determining thread information of the one or more processes during the length of time, wherein determining the thread information comprises determining one or more classes of threads and wherein the thread information comprises, for each of the one or more classes of threads, thread intensity information and a thread intensity seasonal factor for each of the plurality of intervals, wherein the thread intensity seasonal factor for each of the plurality of intervals and for each of the one or more classes of threads corresponds to a smoothed thread intensity seasonal factor associated with the season to which each of the plurality of intervals is mapped;

correlating the heap information with the thread information to identify one or more lines of code of the one or more processes that correspond to the heap usage exceeding the threshold; and

responsive to identifying the one or more lines of code, initiating one or more actions associated with the one or more lines of code.

2. The method of claim 1, wherein correlating the heap information with the thread information comprises:

for each of the one or more classes of threads, determining, based at least in part on the heap usage seasonal factors of the plurality of intervals and the thread intensity seasonal factors of the class of threads and the plurality of intervals, a degree of correlation between the class of threads and the heap usage exceeding the threshold;

selecting, from the one or more classes of threads, a given class of threads that has a highest degree of correlation to the heap usage exceeding the threshold amongst the one or more classes of threads; and

identifying, based at least in part on the given class of threads, the one or more lines of code.

3. The method of claim 2, wherein determining the degree of correlation between the class of threads and the heap usage exceeding the threshold comprises:

calculating a mean of the heap usage seasonal factors of the plurality of intervals;

calculating a mean of the thread intensity seasonal factors of the class of threads and the plurality of intervals;

calculating a variance of the heap usage seasonal factors of the plurality of intervals;

calculating a variance of the thread intensity seasonal factors of the class of threads and the plurality of intervals; and

calculating the degree of correlation based at least in part on the mean of the heap usage seasonal factors of the plurality of intervals, the mean of the thread intensity seasonal factors of the class of threads and the plurality of intervals, the variance of the heap usage seasonal factors of the plurality of intervals, and the variance of the thread intensity seasonal factors of the class of threads and the plurality of intervals.

4. The method of claim 2, wherein each thread of the one or more processes that belongs to the given class of threads executes the one or more lines of code.

5. The method of claim 1:

wherein the length of time spans one or more first cycles of a first period having a first length and one or more second cycles of a second period having a second length;

wherein the first period having the first length is divided into a first plurality of seasons each associated with the smoothed heap usage seasonal factor of a first type and the smoothed thread intensity seasonal factor of a first type for each of the one or more classes of threads;

wherein the second period having the second length is divided into a second plurality of seasons each associated with the smoothed heap usage seasonal factor of a second type and the smoothed thread intensity seasonal factor of a second type for each of the one or more classes of threads;

wherein each of the plurality of intervals is mapped to one of the first plurality of seasons or one of the second plurality of seasons;

wherein, for each of the plurality of intervals, the smoothed heap usage seasonal factor is associated with the one of the first plurality of seasons or the one of the second plurality of seasons that the interval is mapped to; and

wherein, for each of the plurality of intervals, and for each of the one or more classes of threads, the smoothed thread intensity seasonal factor of the class of threads is associated with the one of the first plurality of seasons or the one of the second plurality of seasons that the interval is mapped to.

6. The method of claim 5:

wherein the smoothed heap usage seasonal factors of the first type are determined by;

for each of the first plurality of seasons, determining a heap usage seasonal factor of the first type by comparing an average heap usage of the season of the first plurality of seasons with an average heap usage of the period having the first length;

for each of the heap usage seasonal factors of the first type, dividing the heap usage seasonal factor of the first type by a normalization factor to obtain a renormalized heap usage seasonal factor of the first type;

applying a first spline function to the renormalized heap usage seasonal factors of the first type to obtain the smoothed heap usage seasonal factors of the first type; and

wherein the smoothed heap usage seasonal factors of the second type are determined by;

for each of the second plurality of seasons, determining a heap usage seasonal factor of the second type by comparing an average heap usage of the season of the second plurality of seasons with an average heap usage of the period having the second length;

for each of the heap usage seasonal factors of the second type, dividing the heap usage seasonal factors of the second type by the normalization factor to obtain a renormalized heap usage seasonal factor of the second type; and

applying a second spline function to the renormalized heap usage seasonal factors of the second type to obtain the smoothed heap usage seasonal factors of the second type.

7. The method of claim 1, wherein determining the one or more classes of threads comprises:

obtaining one or more thread dumps of the one or more processes, wherein each of the thread dumps are taken at a different point in time while the one or more processes are executing; and

for each of one or more threads of the one or more thread dumps, classifying the thread based at least in part on a stack trace that corresponds to the thread, the stack trace indicating code executed by the thread when the thread dump was taken.

8. The method of claim 7, wherein classifying the thread based at least in part on the stack trace comprises:

determining whether a class of threads that corresponds to a combination of stack frames included in the stack trace exists;

responsive to determining that the class of threads that corresponds to the combination of stack frames included in the stack trace does not exist, creating a new class of threads that corresponds to the combination of stack frames included in the stack trace; and

incrementing one or more counters associated with the class of threads that corresponds to the combination of stack frames included in the stack trace.

9. The method of claim 1, wherein for each of the one or more classes of threads, for each of the plurality of intervals, the thread intensity information for the interval indicates an average number of threads belonging to the class of threads that are executing during throughout the interval.

10. The method of claim 1, wherein the one or more actions includes at least one of:

generating an alert associated with the one or more lines of code;

or

optimizing the one or more lines of code.

11. The method of claim 10, wherein optimizing the one or more lines of code comprises modifying the one or more lines of code to reduce an amount of heap memory used or reduce a duration of a usage of the heap memory.

12. The method of claim 1, wherein:

the length of time spans a first period and a second period;

the second period immediately follows the first period;

the first period is associated with a plurality of first heap usage seasonal factors and a plurality of first thread intensity seasonal factors;

the second period is associated with a plurality of second heap usage seasonal factors and a plurality of second thread intensity seasonal factors;

the method further comprises;

forming a sequence of heap usage seasonal factors based at least in part on concatenating the first plurality of heap usage seasonal factors and the second plurality of heap usage seasonal factors;

forming a sequence of thread intensity seasonal factors based at least in part on concatenating the first plurality of thread intensity seasonal factors and the second plurality of thread intensity seasonal factors;

performing a first smooth-spline fitting on the sequence of heap usage seasonal factors to generate a sequence of smoothed heap usage seasonal factors to be included in the heap information; and

performing a second smooth-spline fitting on the sequence of thread intensity seasonal factors to generate a sequence of smoothed thread intensity seasonal factors to be included in the thread information;

and the correlation of the heap information with the thread information is based at least in part on the sequence of smoothed heap usage seasonal factors and the sequence of smoothed thread intensity seasonal factors.

13. The method of claim 12, wherein the first period is associated with a weekday, and wherein the second period is associated with at least one of:

a weekend, or a holiday.

14. A system comprising:

one or more processors; and

a memory accessible to the one or more processors, the memory storing one or more instructions that, upon execution by the one or more processors, causes the one or more processors to;

determine a length of time when heap usage by one or more processes exceeds a threshold, wherein the length of time spans at least one period, wherein the at least one period is divided into a plurality of intervals, and wherein each of the plurality of intervals is mapped to a season of a plurality of seasons;

determine heap information of the one or more processes during the length of time, the heap information comprising heap usage information and a heap usage seasonal factor for each of the plurality of intervals in the length of time, wherein the heap usage seasonal factor for each of the plurality of intervals is associated with the season to which each of the plurality of intervals is mapped, and wherein the heap information is determined based at least in part on a first smooth-spline fitting on the heap usage seasonal factors for the plurality of intervals;

determine thread information of the one or more processes during the length of time, wherein determining the thread information comprises determining one or more classes of threads and wherein the thread information comprises, for each of the one or more classes of threads, thread intensity information and a thread intensity seasonal factor for each of the plurality of intervals, wherein the thread intensity seasonal factor for each of the plurality of intervals is associated with the season the each of the plurality of intervals is mapped to, and wherein the thread information is determined based at least in part on performing a second smooth-spline fitting on the thread intensity seasonal factors for the plurality of intervals;

correlate the heap information with the thread information based at least in part on the smooth-spline fitted heap usage seasonal factors and the smooth-spline fitted thread intensity seasonal factors to identify one or more lines of code of the one or more processes that correspond to the heap usage exceeding the threshold; and

responsive to identifying the one or more lines of code, initiate one or more actions associated with the one or more lines of code.

15. The system of claim 14, wherein correlating the heap information with the thread information comprises:

for each of the one or more classes of threads, determining, based at least in part on the heap usage seasonal factors of the plurality of intervals and the thread intensity seasonal factors of the class of threads and the plurality of intervals, a degree of correlation between the class of threads and the heap usage exceeding the threshold;

selecting, from the one or more classes of threads, a given class of threads that has a highest degree of correlation to the heap usage exceeding the threshold amongst the one or more classes of threads; and

identifying, based at least in part on the given class of threads, the one or more lines of code.

16. The system of claim 15, wherein determining the degree of correlation between the class of threads and the heap usage exceeding the threshold comprises:

calculating a mean of the heap usage seasonal factors of the plurality of intervals;

calculating a mean of the thread intensity seasonal factors of the class of threads and the plurality of intervals;

calculating a variance of the heap usage seasonal factors of the plurality of intervals;

calculating a variance of the thread intensity seasonal factors of the class of threads and the plurality of intervals; and

calculating the degree of correlation based at least in part on the mean of the heap usage seasonal factors of the plurality of intervals, the mean of the thread intensity seasonal factors of the class of threads and the plurality of intervals, the variance of the heap usage seasonal factors of the plurality of intervals, and the variance of the thread intensity seasonal factors of the class of threads and the plurality of intervals.

17. The system of claim 14:

wherein the length of time spans one or more first cycles of a first period having a first length and one or more second cycles of a second period having a second length;

wherein the first period having the first length is divided into a first plurality of seasons each associated with a smoothed heap usage seasonal factor of a first type and a smoothed thread intensity seasonal factor of a first type for each of the one or more classes of threads;

wherein the second period having the second length is divided into a second plurality of seasons each associated with a smoothed heap usage seasonal factor of a second type and a smoothed thread intensity seasonal factor of a second type for each of the one or more classes of threads;

wherein each of the plurality of intervals is mapped to one of the first plurality of seasons or one of the second plurality of seasons;

wherein, for each of the plurality of intervals, the heap usage seasonal factor of the interval corresponds to a smoothed heap usage seasonal factor that is associated with the one of the first plurality of seasons or the one of the second plurality of seasons that the interval is mapped to; and

wherein, for each of the plurality of intervals, for each of the one or more classes of threads, the thread intensity seasonal factor of the class of threads of the interval corresponds to a smoothed thread intensity seasonal factor of the class of threads that is associated with the one of the first plurality of seasons or the one of the second plurality of seasons that the interval is mapped to.

18. The system of claim 17:

wherein the smoothed heap usage seasonal factors of the first type are determined by;

for each of the first plurality of seasons, determining a heap usage seasonal factor of the first type by comparing an average heap usage of the season of the first plurality of seasons with an average heap usage of the period having the first length;

for each of the heap usage seasonal factors of the first type, dividing the heap usage seasonal factor of the first type by a normalization factor to obtain a renormalized heap usage seasonal factor of the first type;

applying a first spline function to the renormalized heap usage seasonal factors of the first type to obtain the smoothed heap usage seasonal factors of the first type; and

wherein the smoothed heap usage seasonal factors of the second type are determined by;

for each of the second plurality of seasons, determining a heap usage seasonal factor of the second type by comparing an average heap usage of the season of the second plurality of seasons with an average heap usage of the period having the second length; and

for each of the heap usage seasonal factors of the second type, dividing the heap usage seasonal factors of the second type by the normalization factor to obtain a renormalized heap usage seasonal factor of the second type; and

applying a second spline function to the renormalized heap usage seasonal factors of the second type to obtain the smoothed heap usage seasonal factors of the second type.

19. The system of claim 14, wherein the one or more actions includes at least one of:

generating an alert associated with the one or more lines of code;

or

optimizing the one or more lines of code.

20. A non-transitory computer-readable medium storing one or more instructions that, upon execution by one or more processors, cause the one or more processors to:

determine a length of time when heap usage by one or more processes exceeds a threshold, wherein the length of time spans at least one period, wherein the at least one period is divided into a plurality of intervals, and wherein each of the plurality of intervals is mapped to a season of a plurality of seasons;

determine heap information of the one or more processes during the length of time, the heap information comprising heap usage information and a heap usage seasonal factor for each of the plurality of intervals in the length of time, wherein the heap usage seasonal factor for each of the plurality of intervals corresponds to a smoothed heap usage seasonal factor associated with the season to which each of the plurality of intervals is mapped;

determine thread information of the one or more processes during the length of time, wherein determining the thread information comprises determining one or more classes of threads and wherein the thread information comprises, for each of the one or more classes of threads, thread intensity information and a thread intensity seasonal factor for each of the plurality of intervals, wherein the thread intensity seasonal factor for each of the plurality of intervals and for each of the one or more classes of threads corresponds to a smoothed thread intensity seasonal factor associated with the season to which each of the plurality of intervals is mapped;

correlate the heap information with the thread information to identify one or more lines of code of the one or more processes that correspond to the heap usage exceeding the threshold; and

responsive to identifying the one or more lines of code, initiate one or more actions associated with the one or more lines of code.