Problem determination and diagnosis in shared dynamic clouds

• US 8,862,727 B2
• Filed: 05/14/2012
• Issued: 10/14/2014
• Est. Priority Date: 05/14/2012
• Status: Active Grant

• Current Assignee: SINOEAST CONCEPT LIMITED (Tencent Holdings Limited)
• Original Assignee: International Business Machines Corporation
• Inventors: Jayachandran, Praveen, Sharma, Bikash, Verma, Akshat
• Primary Examiner(s): Nguyen, Thu
• Assistant Examiner(s): Golabbakhsh, Ebrahim
• Application Number:
  US13/470,589
• Publication Number:
  US 20130305092A1
• Time in Patent Office: 883 Days
• Field of Search: 709/224, 709/226, 726/22
• US Class Current: 709/224
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0754 by exceeding limits
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/815 Virtual
• CPC Class Codes: H04L 41/0613 based on the type or category of the network elements
• CPC Class Codes: H04L 41/0631 using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
• CPC Class Codes: H04L 41/5096 wherein the managed service relates to distributed or central networked applications

Abstract

An apparatus and an article of manufacture for problem determination and diagnosis in a shared dynamic cloud environment include monitoring each virtual machine and physical server in the shared dynamic cloud environment for at least one metric, identifying a symptom of a problem and generating an event based on said monitoring, analyzing the event to determine a deviation from normal behavior, and classifying the event as a cloud-based anomaly or an application fault based on existing knowledge.

Claims

1. An article of manufacture comprising a computer readable storage device having computer readable instructions for problem determination and diagnosis in a shared dynamic cloud environment during live virtual machine migration tangibly embodied thereon which, when implemented, cause a computer to carry out a plurality of method steps comprising:

monitoring each virtual machine and each physical server in a shared dynamic cloud environment for at least one metric;

identifying a symptom of a problem within the shared dynamic cloud environment and generating an event based on said monitoring, wherein said event corresponds to said symptom;

analyzing the event to determine a deviation from normal behavior, wherein said normal behavior is determined based on said monitoring; and

classifying the event as a cloud-based anomaly or an application fault based on a comparison of the event with multiple fault signatures, wherein said multiple fault signatures capture a set of deviations from normal behavior associated with (i) one or more cloud-based anomalies and (ii) one or more application faults, and wherein said set of deviations comprises information pertaining to a given virtual machine and a physical server hosting the given virtual machine in the shared dynamic cloud environment, and wherein said classifying comprises;

matching the deviation from normal behavior associated with the event with one of the one or more fault signatures based on the at least one monitored metric monitored for each virtual machine and for each physical server corresponding to said event.

2. The article of manufacture of claim 1, wherein said monitoring comprises outputting a stream of data-points, each data-point being derived from a monitoring data time series, corresponding to each system and application metric in the shared dynamic cloud environment.

3. The article of manufacture of claim 1, wherein the method steps comprise:

building a model of normal application behavior by leveraging a machine learning technique; and

using the model to detect a deviation in the monitored data from normal behavior.

4. The article of manufacture of claim 1, wherein the at least one metric comprises a metric pertaining to at least one of central processing unit, memory, cache, network resources and disk resources.

5. The article of manufacture of claim 1, wherein said identifying comprises identifying a trend from time series data.

6. The article of manufacture of claim 1, wherein said analyzing comprises using statistical correlation across virtual machines and resources to pinpoint the location of the deviation to an affected resource and virtual machine.

7. A system for problem determination and diagnosis in a shared dynamic cloud environment during live virtual machine migration, comprising:

at least one distinct software module, each distinct software module being embodied on a tangible computer-readable medium;

a memory; and

at least one processor coupled to the memory and operative for;

monitoring each virtual machine and each physical server in a shared dynamic cloud environment for at least one metric;

identifying a symptom of a problem within the shared dynamic cloud environment and generating an event based on said monitoring, wherein said event corresponds to said symptom;

analyzing the event to determine a deviation from normal behavior, wherein said normal behavior is determined based on said monitoring; and

classifying the event as a cloud-based anomaly or an application fault based on a comparison of the event with multiple fault signatures, wherein said multiple fault signatures capture a set of deviations from normal behavior associated with (i) one or more cloud-based anomalies and (ii) one or more application faults, and wherein said set of deviations comprises information pertaining to a given virtual machine and a physical server hosting the given virtual machine in the shared dynamic cloud environment, and wherein said classifying comprises;

matching the deviation from normal behavior associated with the event with one of the one or more fault signatures based on the at least one monitored metric monitored for each virtual machine and for each physical server corresponding to said event.

8. The system of claim 7, wherein the at least one metric comprises a metric pertaining to at least one of central processing unit, memory, cache, network resources and disk resources.

9. The system of claim 7, wherein said identifying comprises identifying a trend from time series data.

10. The system of claim 7, wherein said analyzing comprises using statistical correlation across virtual machines and resources to pinpoint the location of the deviation to an affected resource and virtual machine.

11. A system for problem determination and diagnosis in a shared dynamic cloud environment during live virtual machine migration, comprising:

a memory;

at least one processor coupled to the memory; and

at least one distinct software module, each distinct software module being embodied on a tangible computer-readable medium, the at least one distinct software module comprising;

a monitoring engine module, executing on the processor, for monitoring each virtual machine and each physical server in a shared dynamic cloud environment for at least one metric and outputting a monitoring data time-series corresponding to each metric;

an event generation engine module, executing on the processor, for identifying a symptom of a problem within the shared dynamic cloud environment and generating an event based on said monitoring, wherein said event corresponds to said symptom;

a problem determination engine module, executing on the processor, for analyzing the event to determine and locate a deviation from normal behavior, wherein said normal behavior is determined based on said monitoring; and

a diagnosis engine module, executing on the processor, for classifying the event as a cloud-based anomaly or an application fault based on a comparison of the event with multiple fault signatures, wherein said multiple fault signatures capture a set of deviations from normal behavior associated with (i) one or more cloud-based anomalies and (ii) one or more application faults, and wherein said set of deviations comprises information pertaining to a given virtual machine and a physical server hosting the given virtual machine in the shared dynamic cloud environment, and wherein said classifying comprises;

matching the deviation from normal behavior associated with the event with one of the one or more fault signatures based on the at least one monitored metric monitored for each virtual machine and for each physical server corresponding to said event.