Malware detection using risk analysis based on file system and network activity

• US 8,479,276 B1
• Filed: 12/29/2010
• Issued: 07/02/2013
• Est. Priority Date: 12/29/2010
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Saklikar, Samir Dilipkumar, Vaystikh, Alex, Polansky, Robert, Liptz, Liron
• Primary Examiner(s): Chai, Longbit
• Application Number:
  US12/981,072
• Time in Patent Office: 916 Days
• Field of Search: 726/13
• US Class Current: 726/13
• CPC Class Codes: G06F 21/577 Assessing vulnerabilities and evaluating computer system security

Abstract

A virtual machine computing platform uses a security virtual machine (SVM) in operational communications with a risk engine which has access to a database including stored patterns corresponding to patterns of filtered operational data that are expected to be generated during operation of the monitored virtual machine when malware is executing. The stored patterns may have been generated during preceding design and training phases. The SVM is operated to (1) receive raw operational data from a virtual machine monitor, the raw operational data obtained from file system operations and network operations of the monitored virtual machine; (2) apply rule-based filtering to the raw operational data to generate filtered operational data; and (3) in conjunction with the risk engine, perform a mathematical (e.g., Bayesian) analysis based on the filtered operational data and the stored patterns in the database to calculate a likelihood that the malware is executing in the monitored virtual machine. A control action is taken if the likelihood is sufficiently high.

Claims

1. A method of detecting and responding to presence of malware persistently executing in a monitored virtual machine of a virtual computing platform, comprising:

establishing operational communications between a security virtual machine, which is separate from said monitored virtual machine, of the virtual computing platform and a risk engine, the risk engine having access to a database which includes stored patterns corresponding to patterns of filtered operational data expected to be generated during operation of the monitored virtual machine when the malware is executing;

operating the security virtual machine to (1) receive raw operational data from a virtual machine monitor, which can be insulated from anti-detection activity of said executing malware, of the virtual computing platform, the raw operational data obtained from both of file system operations and network operations of the monitored virtual machine which are visible to the virtual machine monitor;

(2) apply rule-based filtering to the raw operational data to generate filtered operational data; and

(3) in conjunction with the risk engine, perform a mathematical analysis based on the filtered operational data and the stored patterns in the database to calculate a likelihood that the malware is executing in the monitored virtual machine; and

in response to the likelihood exceeding a predetermined threshold, generating a control signal to initiate a control action.

2. A method according to claim 1, wherein the control action is selected from generating an alert and taking an action to prevent effective ongoing execution of the malware in the monitored virtual machine.

3. A method according to claim 1, wherein the mathematical analysis is a Bayesian analysis.

4. A method according to claim 1, wherein the raw operational data includes network communications packets and the filtered operational data includes one or more of direct data obtained from the packets, supporting data about the packets, and summary data summarizing contents of the network communications packets.

5. A method according to claim 4, wherein the direct data includes network addresses, the supporting data includes transmission and/or reception times, and the summary data includes a digest or hash of packet contents.

6. A method according to claim 1, wherein the raw operational data includes file system commands and the filtered operational data includes one or more of direct data obtained from data files, supporting data about the data files, and summary data summarizing contents of the data files.

7. A method according to claim 6, wherein the direct data includes file names, the supporting data includes file creation times, and the summary data includes a digest or hash of file contents.

8. A method according to claim 1, wherein the rule-based filtering effects a deep inspection of network communications packets and/or data files to provide filtered operational data describing activity at an application level.

9. A method according to claim 8, wherein the activity includes commands from one or more of hypertext transfer protocol (HTTP) commands and messaging application commands, the HTTP commands being used to retrieve named network resources, the messaging application commands used by a messaging application and being directed to named network users, messaging accounts or addresses.

10. A method according to claim 1, wherein the rule-based filtering identifies combinations of file operations and network operations indicative of unauthorized copying of network communications packet information and/or data file information to an external computer.

11. A method according to claim 10, wherein the combinations include legitimate network or file transactions followed by network traffic containing an illegitimate duplication of data from the legitimate network or file transactions.

12. A method according to claim 1, wherein the rule-based filtering identifies particular patterns of operations on a registry of an operating system of a virtual machine.

13. A method according to claim 1, wherein the database is populated with filtered operational data obtained during a preceding training phase in which the malware is executing on one of the monitored virtual machines.

14. A method according to claim 13, wherein an organization of the database is determined in part by a preceding design phase involving monitoring of the raw operational data during operation of the virtual computing platform.

15. A method according to claim 1, wherein the stored patterns include both positive and negative sample sets, a positive sample set corresponding to the presence of the malware on a virtual machine, a negative sample set corresponding to the absence of the malware on a virtual machine.

16. A method according to claim 1, wherein rules for generating the filtered operational data from the raw operational data are selected based on input criteria provided to the security virtual machine by the risk engine.

17. A virtual computing platform, comprising:

a physical machine including physical computing resources, the physical computing resources including memory, one or more processors, and input/output circuitry; and

software executing on the physical machine, the software including a virtual machine monitor, a monitored virtual machine, and a security virtual machine, the security virtual machine being operative, separate from said monitored virtual machine, to establish operational communications with a risk engine, the risk engine having access to a database which includes stored patterns corresponding to patterns of filtered operational data expected to be generated during operation of the monitored virtual machine when the malware is executing, wherein the security virtual machine is further operative to;

receive raw operational data from the virtual machine monitor, which can be insulated from anti-detection activity of said executing malware, the raw operational data obtained from both of file system operations and network operations of the monitored virtual machine which are visible to the virtual machine monitor;

apply rule-based filtering to the raw operational data to generate filtered operational data; and

in conjunction with the risk engine, perform a mathematical analysis based on the filtered operational data and the stored patterns in the database to calculate a likelihood that the malware is executing in the monitored virtual machine; and

in conjunction with the virtual machine monitor, in response to the likelihood exceeding a predetermined threshold, generating a control signal to initiate a control action.

18. A virtual computing platform according to claim 17, wherein the control action is selected from generating an alert and taking an action to prevent effective ongoing execution of the malware in the monitored virtual machine.

19. A virtual computing platform according to claim 17, wherein the mathematical analysis is a Bayesian analysis.

20. A virtual computing platform according to claim 17, wherein the raw operational data includes network communications packets and the filtered operational data includes one or more of direct data obtained from the packets, supporting data about the packets, and summary data summarizing contents of the network communications packets.

21. A virtual computing platform according to claim 20, wherein the direct data includes network addresses, the supporting data includes transmission and/or reception times, and the summary data includes a digest or hash of packet contents.

22. A virtual computing platform according to claim 17, wherein the raw operational data includes one or more of direct data obtained from data files, supporting data about the data files, and summary data summarizing contents of the data files.

23. A virtual computing platform according to claim 22, wherein the direct data includes file names, the supporting data includes file creation times, and the summary data includes a digest or hash of file contents.

24. A virtual computing platform according to claim 17, wherein the rule-based filtering effects a deep inspection of network communications packets and/or data files to provide filtered operational data describing activity at an application level.

25. A virtual computing platform according to claim 24, wherein the activity includes commands from one or more of hypertext transfer protocol (HTTP) commands and messaging application commands, the HTTP commands being used to retrieve named network resources, the messaging application commands used by a messaging application and being directed to named network users, accounts or addresses.

26. A virtual computing platform according to claim 17, wherein the rule-based filtering identifies combinations of file operations and network operations indicative of unauthorized copying of network communications packet information and/or data file information to an external computer.

27. A virtual computing platform according to claim 26, wherein the combinations include legitimate network or file transactions followed by network traffic containing an illegitimate duplication of data from the legitimate network or file transactions.

28. A virtual computing platform according to claim 17, wherein the rule-based filtering identifies particular patterns of operations on a registry of an operating system of a virtual machine.

29. A virtual computing platform according to claim 17, wherein the database is populated with filtered operational data obtained during a preceding training phase in which the malware is executing on one of the monitored virtual machines.

30. A virtual computing platform according to claim 29, wherein an organization of the database is determined in part by a preceding design phase involving monitoring of the raw operational data during operation of the virtual computing platform.

31. A virtual computing platform according to claim 17, wherein the stored patterns include both positive and negative sample sets, a positive sample set corresponding to the presence of the malware on a virtual machine, a negative sample set corresponding to the absence of the malware on a virtual machine.

32. A virtual computing platform according to claim 17, wherein rules for generating the filtered operational data from the raw operational data are selected based on input criteria provided to the security virtual machine by the risk engine.