Seasonal trending, forecasting, anomaly detection, and endpoint prediction of thread intensity statistics

• US 10,333,798 B2
• Filed: 05/06/2015
• Issued: 06/25/2019
• Est. Priority Date: 04/11/2013
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Chan, Eric S., Ahad, Rafiul, Ghoneimy, Adel, Santos, Adriano Covello
• Primary Examiner(s): Ehne, Charles
• Application Number:
  US14/705,304
• Publication Number:
  US 20150234869A1
• Time in Patent Office: 1,511 Days
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/0715 in a system implementing multitasking multitasking per se G06F9/46
• CPC Class Codes: G06F 11/0718 in an object-oriented system
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/0778 Dumping, i.e. gathering error/state information after a fault for later diagnosis
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3003 Monitoring arrangements specially adapted to the computing system or computing system component being monitored
• CPC Class Codes: G06F 11/3037 where the computing system component is a memory, e.g. virtual memory, cache accessing, addressing or allocating within memory systems or architectures G06F12/00; checking stores for correct operation G11C29/00
• CPC Class Codes: G06F 11/34 Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3471 Address tracing
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: G06F 2201/88 Monitoring involving counting
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/4881 : Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: H04L 41/5009 : Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]

Abstract

Data can be categorized into facts, information, hypothesis, and directives. Activities that generate certain categories of data based on other categories of data through the application of knowledge which can be categorized into classifications, assessments, resolutions, and enactments. Activities can be driven by a Classification-Assessment-Resolution-Enactment (CARE) control engine. The CARE control and these categorizations can be used to enhance a multitude of systems, for example diagnostic system, such as through historical record keeping, machine learning, and automation. Such a diagnostic system can include a system that forecasts computing system failures based on the application of knowledge to system vital signs such as thread or stack segment intensity and memory heap usage. These vital signs are facts that can be classified to produce information such as memory leaks, convoy effects, or other problems. Classification can involve the automatic generation of classes, states, observations, predictions, norms, objectives, and the processing of sample intervals having irregular durations.

Claims

1. A method comprising:

receiving, at a computer system, a plurality of thread dumps associated within a Java Virtual Machine;

identifying, by the computer system, a class of threads within the plurality of thread dumps;

calculating, by the computer system, a plurality of thread intensities for the class of threads, each of the plurality of thread intensities corresponding to one of the plurality of thread dumps;

determining, by the computer system, one or more trends for the class of threads, based on the plurality of thread intensities for the class of threads; and

determining, by the computer system, that the Java Virtual Machine has a memory leak, based on the one or more trends for the class of threads.

2. The method of claim 1, wherein identifying the class of threads within the plurality of thread dumps comprises:

storing the plurality of thread dumps in one or more Hadoop Distributed File System (HDFS) clusters; and

extracting a plurality of thread classification signatures from the plurality of thread dumps stored in the HDFS clusters, using a MapReduce algorithm.

3. The method of claim 1, further comprising:

determining that the one or more trends for the class of threads includes at least one seasonal trend; and

filtering the at least one seasonal trend before determining at the Java Virtual Machine has a memory leak.

4. The method of claim 3, wherein filtering the at least one season trend comprises:

determining a value of a feature for trend data pertaining to a transition from a weekend to a weekday; and

determining a value of a feature for trend data pertaining to a transition from a weekday to a weekend.

5. The method of claim 1, wherein calculating each of the plurality of thread intensities comprises:

identifying a plurality of threads from one of the plurality of thread dumps; and

measuring an intensity for each of the plurality of threads.

6. The method of claim 5, further comprising:

adjusting the intensity measurements for the plurality of threads, based on variable latencies arising from thread scheduling and garbage collection.

7. The method of claim 5, further comprising:

adjusting the intensity measurements for the plurality of threads, based on variable computation times arising from classification of stack segments.

8. The method of claim 5, further comprising:

applying to the intensity measurements, in parallel, a plurality of filters that are associated with differing time scales.

9. The method of claim 1, further comprising:

extracting one or more feature vectors from the plurality of thread intensities, the one or more feature vectors including at least one of;

Java Virtual Machine heap size, histogram, seasonal factors, linear trends, non-linear trends, level spike, level shift, level drift, variance change, outliers, or endpoint prediction.

10. A system comprising:

one or more processors; and

a storage medium storing instructions which, when executed by the one or more processors, cause the processors to;

receive a plurality of thread dumps associated within a Java Virtual Machine;

identify a class of threads within the plurality of thread dumps;

calculate a plurality of thread intensities for the class of threads, each of the plurality of thread intensities corresponding to one of the plurality of thread dumps;

determine one or more trends for the class of threads, based on the plurality of thread intensities for the class of threads; and

determine that the Java Virtual Machine has a memory leak, based on the one or more trends for the class of threads.

11. The system of claim 10, wherein identifying the class of threads within the plurality of thread dumps comprises:

storing the plurality of thread dumps in one or more Hadoop Distributed File System (HDFS) clusters; and

extracting a plurality of thread classification signatures from the plurality of thread dumps stored in the HDFS clusters, using a MapReduce algorithm.

12. The system of claim 10, the storage medium storing further instructions which, when executed by the one or more processors, cause the processors to:

determine that the one or more trends for the class of threads includes at least one seasonal trend; and

filter the at least one seasonal trend before determining at the Java Virtual Machine has a memory leak.

13. The system of claim 12, wherein filtering the at least one season trend comprises:

determining a value of a feature for trend data pertaining to a transition from a weekend to a weekday; and

determining a value of a feature for trend data pertaining to a transition from a weekday to a weekend.

14. The system of claim 10, wherein calculating each of the plurality of thread intensities comprises:

identifying a plurality of threads from one of the plurality of thread dumps; and

measuring an intensity for each of the plurality of threads.

15. The system of claim 14, the storage medium storing further instructions which, when executed by the one or more processors, cause the processors to:

adjust the intensity measurements for the plurality of threads, based on variable latencies arising from thread scheduling and garbage collection.

16. The system of claim 14, the storage medium storing further instructions which, when executed by the one or more processors, cause the processors to:

adjust the intensity measurements for the plurality of threads, based on variable computation times arising from classification of stack segments.

17. The system of claim 14, the storage medium storing further instructions which, when executed by the one or more processors, cause the processors to:

apply to the intensity measurements, in parallel, a plurality of filters that are associated with differing time scales.

18. The system of claim 10, the storage medium storing further instructions which, when executed by the one or more processors, cause the processors to:

extract one or more feature vectors from the plurality of thread intensities, the one or more feature vectors including at least one of;

Java Virtual Machine heap size, histogram, seasonal factors, linear trends, non-linear trends, level spike, level shift, level drift, variance change, outliers, or endpoint prediction.

19. A non-transitory computer-readable storage memory storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving, at a computer system, a plurality of thread dumps associated within a Java Virtual Machine;

identifying, by the computer system, a class of threads within the plurality of thread dumps;

calculating, by the computer system, a plurality of thread intensities for the class of threads, each of the plurality of thread intensities corresponding to one of the plurality of thread dumps;

determining, by the computer system, one or more trends for the class of threads, based on the plurality of thread intensities for the class of threads; and

determining, by the computer system, that the Java Virtual Machine has a memory leak, based on the one or more trends for the class of threads.

20. The computer-readable storage memory of claim 19, storing further instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

extracting one or more feature vectors from the plurality of thread intensities, the one or more feature vectors including at least one of;

Java Virtual Machine heap size, histogram, seasonal factors, linear trends, non-linear trends, level spike, level shift, level drift, variance change, outliers, or endpoint prediction.