Unsupervised detection of anomalous processes using hardware features

• US 9,996,694 B2
• Filed: 03/14/2014
• Issued: 06/12/2018
• Est. Priority Date: 03/18/2013
• Status: Active Grant

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Sethumadhavan, Lakshminarasimhan, Tang, Adrian, Stolfo, Salvatore
• Primary Examiner(s): Jeudy, Josnel
• Application Number:
  US14/778,043
• Publication Number:
  US 20160275289A1
• Time in Patent Office: 1,551 Days
• CPC Class Codes: G06F 21/52 during program execution, e.g. stack integrity ; Preventing unwanted data erasure; Buffer overflow
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 21/566 Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities
• CPC Class Codes: G06F 2221/034 Test or assess a computer or a system
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 9/3239 involving non-keyed hash functions, e.g. modification detection codes [MDCs], MD5, SHA or RIPEMD

Abstract

Disclosed are devices, systems, apparatus, methods, products, media and other implementations, including a method that includes obtaining current hardware performance data, including hardware performance counter data, for a hardware device executing a first process associated with pre-recorded hardware performance data representative of the first process'"'"' normal behavior, and determining whether a malicious process is affecting performance of the first process based on a determination of an extent of deviation of the obtained current hardware performance data corresponding to the first process from the pre-recorded hardware performance data representative of the normal behavior of the first process.

Claims

1. A method for unsupervised anomaly-based malware detection using hardware features, the method comprising:

obtaining current hardware performance data, including hardware performance time-varying counter data, for a hardware device executing a first process associated with recorded hardware performance data representative of the first process'"'"' normal behavior;

identifying a set of hardware performance data from the obtained current hardware performance data based at least on a quantitative measure of how effective one or more features associated with the current hardware performance data can discriminate between hardware performance data obtained during clean execution of a victim process and hardware performance data obtained during infected execution of the victim process,

wherein the quantitative measures are computed for both an exploitation stage and a take-over stage of a multi-stage malware infection that hijacks control of the victim process, and

wherein the quantitative measures taken at both the exploitation stage and the take-over stage enable the determination of which features are most useful in differentiating clean execution for the victim process from infected execution of the victim process;

aggregating the identified set of hardware performance data;

transforming the aggregated set of hardware performance data based on one or more transform functions, the transforming the aggregated set of hardware performance data comprising deriving a normalized hardware performance value, normalized_i, for an event i, from hardware performance data value, raw_i for the event i, according to;

2. The method of claim 1, wherein the hardware performance data is obtained at various time instances and wherein the method further comprises:

performing one or more of a data push operation initiated by the hardware device to send the current hardware performance data, or a data pull operation, initiated by an antivirus engine, to cause the current hardware performance data to be sent by the hardware device.

3. The method of claim 1, wherein identifying the set of hardware performance data comprises:

selecting the one or more features based on respective computed one or more scores representative of the quantitative measure of how effective one or more features associated with the current hardware performance data can discriminate between hardware performance data obtained during clean execution of a victim process and hardware performance data obtained during infected execution of the victim process; and

obtaining performance data from the current hardware performance data only for the selected one or more features.

4. The method of claim 3, wherein computing the one or more scores comprises computing one or more Fisher scores.

5. The method of claim 1, wherein determining whether the anomalous process is affecting the performance of the first process comprises:

applying one or more machine-learning procedures, trained using the recorded hardware performance data representative of the normal behavior of the first process, to the transformed set of hardware performance data to determine whether the transformed set of hardware performance data for the hardware device executing the first process deviates from the recorded hardware performance data for the first process.

6. The method of claim 5, wherein the one or more machine learning procedures comprise one or more of:

a support vector machine implementing a non-linear radial basis function (RBF) kernel, a k-nearest neighbor procedure, a decision tree procedure, a random forest procedure, an artificial neural network procedure, a tensor density procedure, a Support Vector Machine (SVM), or a hidden Markov model procedure.

7. The method of claim 1, wherein the hardware performance data comprise one or more of:

processor load density data, branch prediction performance data, or data regarding instruction cache misses.

14. The method of claim 1, wherein the multi-stage malware infection comprises a Return-Oriented Programming (“

ROP”

) and Stage 1 shellcode.

8. A system for unsupervised anomaly-based malware detection using hardware features, the system comprising:

a hardware device executing a first process; and

an antivirus engine in communication with the hardware device, the antivirus engine configured to;

obtain current hardware performance data, including hardware performance time-varying counter data, for the hardware device executing the first process, the first process associated with recorded hardware performance data representative of the first process'"'"' normal behavior;

select one or more features based on respective computed one or more scores representative of a quantitative measure of how effective one or more features associated with the current hardware performance data can discriminate between hardware performance data obtained during clean execution of a victim process and hardware performance data obtained during infected execution of the victim process,

wherein the quantitative measures are computed for both an exploitation stage and a take-over stage of a multi-stage malware infection that hijacks control of the victim process, and

wherein the quantitative measures taken at both the exploitation stage and the take-over stage enable the determination of which features are most useful in differentiating clean execution for the victim process from infected execution of the victim process;

select performance data from the current hardware performance data only for the selected one or more features;

aggregate the selected hardware performance data according to a sampling duration;

transform the aggregated hardware performance data based on one or more transform functions, the transforming the aggregated hardware performance data comprising deriving a normalized hardware performance value, normalized_i, for an event i, from hardware performance data value, raw_i for the event i, according to;

9. The system of claim 8, wherein the antivirus engine configured to determine whether the anomalous process is affecting the performance of the first process is configured to:

apply one or more machine-learning procedures, trained using the recorded hardware performance data representative of the normal behavior of the first process, to the aggregated current hardware performance data to determine whether the aggregated current hardware performance data for the hardware device executing the first process deviates from the recorded hardware performance data for the first process.

10. The system of claim 9, wherein the one or more machine learning procedures comprise one or more of:

a support vector machine implementing a non-linear radial basis function (RBF) kernel, a k-nearest neighbor procedure, a decision tree procedure, a random forest procedure, an artificial neural network procedure, a tensor density procedure, a Support Vector Machine (SVM), or a hidden Markov model procedure.

11. A non-transitory computer readable media storing a set of instructions executable on at least one programmable device that, when executed, causes operations comprising:

obtaining current hardware performance data, including hardware performance time-varying counter data, for a hardware device executing a first process associated with recorded hardware performance data representative of the first process'"'"' normal behavior;

selecting one or more features based on respective computed one or more scores representative of a quantitative measure of how effective one or more features associated with the current hardware performance data can discriminate between hardware performance data obtained during clean execution of a victim process and hardware performance data obtained during infected execution of the victim process,

wherein the quantitative measures are computed for both an exploitation stage and a take-over stage of a multi-stage malware infection that hijacks control of the victim process, and

wherein the quantitative measures taken at both the exploitation stage and the take-over stage enable the determination of which features are most useful in differentiating clean execution for the victim process from infected execution of the victim process;

obtaining performance data from the current hardware performance data only for the selected one or more features;

aggregating the current hardware performance data according to a sampling duration;

transforming the aggregated hardware performance data based on one or more transform functions, the transforming the aggregated hardware performance data comprising deriving a normalized hardware performance value, normalized_i, for an event i, from hardware performance data value, raw_i, for the event i, according to;

12. The computer readable media of claim 11, wherein determining whether the anomalous process is affecting the performance of the first process comprises:

applying one or more machine-learning procedures, trained using the recorded hardware performance data representative of the normal behavior of the first process, to the aggregated current hardware performance data to determine whether the aggregated current hardware performance data for the hardware device executing the first process deviates from the recorded hardware performance data for the first process.

13. An apparatus for unsupervised anomaly-based malware detection using hardware features, the apparatus comprising:

a hardware device executing a first process;

an antivirus engine in communication with the hardware device, the antivirus engine configured to;

obtain current hardware performance data, including hardware performance counter data, for the hardware device executing a first process associated with recorded hardware performance data representative of the first process'"'"' normal behavior;

select one or more features based on respective computed one or more scores representative of a quantitative measure of how effective one or more features associated with the current hardware performance data can discriminate between hardware performance data obtained during clean execution of a victim process and hardware performance data obtained during infected execution of the victim process,

wherein the quantitative measures are computed for both an exploitation stage and a take-over stage of a multi-stage malware infection that hijacks control of the victim process, and

wherein the quantitative measures taken at both the exploitation stage and the take-over stage enable the determination of which features are most useful in differentiating clean execution for the victim process from infected execution of the victim process;

obtain performance data from the current hardware performance data only for the selected one or more features;

aggregate the current hardware performance data according to a sampling duration;

transform the aggregated hardware performance data based on one or more transform functions, the transforming the aggregated hardware performance data comprising deriving a normalized hardware performance value, normalized_i for an event i, from hardware performance data value, raw_i, for the event i, according to;