Anomalous usage of resources by a process in a software defined data center

• US 9,952,891 B2
• Filed: 03/22/2016
• Issued: 04/24/2018
• Est. Priority Date: 03/22/2016
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Zada, Dan, Peretz, Noam, Paz, Hilik
• Primary Examiner(s): KIM, DONG U
• Application Number:
  US15/077,192
• Publication Number:
  US 20170277553A1
• Time in Patent Office: 763 Days
• Field of Search: None
• CPC Class Codes: G06F 11/301 where the computing system is a virtual computing platform, e.g. logically partitioned systems virtual machines G06F9/45533; logical partitioning of resources G06F9/5077
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources

Abstract

The present disclosure is related to systems and methods for analysis of anomalous usage of a resource. An example system can include a plurality of virtual computing instances (VCIs) deployed in a software defined data center and a respective monitoring agent associated with each of the plurality of VCIs. The monitoring agent can be configured to collect information regarding resources used by a plurality of processes running on an associated VCI, maintain a sliding window of top resource users among the plurality of processes, analyze the top resource users among the plurality of processes for anomalous usage of the resources, and take an action in response to the anomalous usage.

Claims

1. A system, comprising:

a plurality of virtual computing instances (VCIs) provisioned with a plurality of computing resources and deployed in a software defined data center that are ultimately executed on hardware; and

a respective monitoring agent comprising a processing resource and ultimately executed on hardware associated with each of the plurality of VCIs, wherein the monitoring agent is configured to execute instructions to;

collect information regarding resources used by a plurality of processes running on an associated VCI, wherein the information regarding the resources used includes a name of the process, an argument associated with the process, and a resource consumption associated with the process;

maintain a sliding window of top resource users among the plurality of processes, wherein the sliding window includes the information regarding the resources for each of the top resource users;

analyze the top resource users among the plurality of processes to determine that a process among the plurality of processes is an anomalous process, wherein the anomalous process is a process that is consuming an amount of resources that is greater than a threshold amount of resources during a particular time period;

responsive to the determination that the process is anomalous, monitor the argument regarding the anomalous process to determine if the anomalous process continues to be anomalous over time; and

remediate the anomalous process by dynamically allocating computing resources to the VCI in response to the anomalous usage.

2. The system of claim 1, wherein the system includes a management server for the software defined data center;

and

wherein the monitoring agent is further configured to execute instructions to report the anomalous usage to the management server.

3. The system of claim 1, wherein the system includes a management server for the software defined data center;

and

wherein the monitoring agent is further configured to execute instructions to report the collected information regarding resources used by the top resource user associated with the anomalous usage to the management server.

4. The system of claim 1, wherein the system includes a management server for the software defined data center;

and

wherein the monitoring agent is further configured to execute instructions to report the collected information regarding resources used by the top resource users in the sliding window to the management server.

5. The system of claim 4, wherein the management server is configured to:

further analyze the collected information; and

report a result of the further analysis to an end user.

6. The system of claim 5, wherein the further analysis comprises sorting the plurality of processes that exhibit anomalous usage based, at least in part, on an amount of processing resources consumed by each process that exhibits anomalous usage among the plurality of processes.

7. The system of claim 5, wherein the further analysis includes analyzing at least an amount of processing resources consumed, an amount of memory resources consumed, and an amount of input/output resources consumed.

8. The system of claim 1, wherein dynamically allocating computing resources to the VCI further comprises terminating the anomalous process.

9. A method, comprising:

identifying, via a respective monitoring agent provisioned with a plurality of computing resources deployed in a software defined data center and ultimately executed by hardware and associated with each of a plurality of virtual computing instances (VCIs), anomalous processes running on respective VCIs among the plurality of VCIs, wherein the anomalous processes are processes that are consuming an amount of resources that is greater than a threshold amount of resources during a particular time period or are processes that are consuming resources sporadically;

collecting information regarding the amount of resources consumed, wherein the information regarding the resources used includes names of the processes, arguments associated with the processes, and resource consumption associated with the processes;

monitoring the arguments associated with the anomalous processes in response to identifying the anomalous processes to determine if the anomalous processes continue to be anomalous over time;

maintaining a sliding window of top resource users among the processes, wherein the sliding window includes the information regarding the resources for each of the top resource users;

analyzing the amount of resources consumed by each of the anomalous processes periodically in response to the determination that the amount of resources consumed by the anomalous processes exceeds the threshold resource consumption value or in response to the determination that the anomalous process is consuming resources sporadically; and

remediating the anomalous process by dynamically allocating computing resources to the VCIs in response to the determination that the amount of resources consumed by the anomalous processes exceeds the threshold resource consumption value.

10. The method of claim 9, further comprising monitoring, via the monitoring agent, the amount of resources consumed by the anomalous process by analyzing at least one performance metric associated with the amount of resources consumed by the anomalous process.

11. The method of claim 9 further comprising providing a sliding window including a configurable number of anomalous process based, at least in part, on the amount of resources consumed by each anomalous process;

wherein

anomalous processes among the configurable number of anomalous processes are displayed in the sliding window based, at least in part, on the amount of resources consumed by each anomalous process.

12. The method of claim 9, further comprising providing information associated with the amount of resources consumed by each anomalous process to a management VCI that is in communication with each of the respective monitoring agents.

13. The method of claim 12, further comprising providing, via the management agent, information associated with at least one anomalous process to a monitoring agent that is different than the respective monitoring agent that provided the information associated with the at least one anomalous process to the management VCI.

14. The method of claim 9, further comprising storing, for each of the anomalous processes, a process name, at least one argument associated with the process, and an amount of resources consumed by the process;

and

monitoring each of the anomalous processes based, at least in part, on the at least one argument associated with the process and the amount of resources consumed by the process.

15. The method of claim 9, further comprising analyzing, via the respective monitoring agent, processes running on the respective VCI that are similar to the anomalous processes to determine if the processes that are similar to the anomalous processes are anomalous.

16. A system, comprising:

an anomalous process engine provisioned with a plurality of computing resources deployed in a software defined data center and ultimately executed by hardware to execute instructions to;

allocate resources to a plurality of virtual computing instances (VCIs);

monitor the resources allocated to the VCIs;

collect information regarding resources used by the VCIs, wherein the information regarding the resources used includes a name of the process, an argument associated with the process, and a resource consumption associated with the process;

determine that a resource allocated to the VCIs is an anomalous process, wherein the anomalous process is a process that is consuming an amount of resources that is greater than a threshold amount of resources during a particular time period;

responsive to the determination that the process is anomalous, monitor the argument regarding with the anomalous process to determine if the anomalous process continues to be anomalous over time;

receive an indication for each of a plurality of anomalous processes running on the plurality of VCIs; and

remediate at least one anomalous process by dynamically allocating resources to the VCI in response to the determination that the process is consuming the amount of resources that is greater than the threshold amount of resources during the particular time period;

a collection engine provisioned with a plurality of computing resources deployed in a software defined data center and ultimately executed by hardware to execute instructions to collect resource consumption metrics associated with the anomalous processes in response to the indication; and

an analysis engine provisioned with a plurality of computing resources deployed in a software defined data center and ultimately executed by hardware to execute instructions to;

maintain a sliding window of a configurable number of top resource users among the plurality of anomalous processes based, at least in part, on the resource consumption metrics, wherein the sliding window includes information regarding the resources for each of the top resource users; and

analyze the plurality of anomalous processes.

17. The system of claim 16, wherein the resource consumption metrics include processing resource consumption, memory resource consumption, and input/output (I/O) resource consumption.

18. The system of claim 16, wherein the collection engine is configured to collect advanced performance metrics, wherein advanced performance metrics include a measure of how often an anomalous process among the plurality of anomalous processes is consuming resources and how often the anomalous process among the plurality of anomalous processes is not consuming resources.

19. The system of claim 16, wherein the analysis engine is configured to monitor the plurality of anomalous processes to determine if the processes continue to be anomalous over time.

20. The system of claim 16, wherein the analysis engine is configured to:

analyze processes that are similar to anomalous processes among the plurality of anomalous processes to determine if the processes that are similar are anomalous; and

query a monitoring agent associated with a software defined data center to request information about the processes that are similar to the anomalous processes.