System and method of detecting time-delayed malicious traffic

• US 8,635,696 B1
• Filed: 06/28/2013
• Issued: 01/21/2014
• Est. Priority Date: 04/01/2004
• Status: Active Grant

• Current Assignee: Fireeye Security Holdings US LLC
• Original Assignee: FireEye, Inc.
• Inventors: Aziz, Ashar
• Primary Examiner(s): Chea, Philip
• Assistant Examiner(s): DOLLY, KENDALL LYNN
• Application Number:
  US13/931,631
• Time in Patent Office: 207 Days
• Field of Search: None
• US Class Current: 726/23
• CPC Class Codes: G06F 2009/45587 Isolation or security of virtual machine instances
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 63/0245 Filtering by information in the payload
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
• CPC Class Codes: H04L 63/1458 Denial of Service
• CPC Class Codes: H04L 63/1491 using deception as countermeasure, e.g. honeypots, honeynets, decoys or entrapment

Abstract

A system for detecting a computer worm comprises a traffic analysis device in communication with a network device. The traffic analysis device can analyze network traffic received over a communication network and duplicate at least select network communications within the network traffic having characteristics associated with one or more computer worms. The network device comprises a controller in communication with one or more virtual machines that are configured to receive the duplicated network communications from the traffic analysis device. The network device may (i) monitor a behavior of a first virtual machine of the one or more virtual machines in response to processing of the duplicated network communications within the first virtual machine, (ii) identify an anomalous behavior as an unexpected occurrence in the monitored behavior, and (iii) determine, based on the identified anomalous behavior, the presence of the one or more computer worms in the duplicated network communications.

Claims

1. A system comprising:

a traffic device configured to receive network traffic over a communication network; and

a network device in communication with the traffic device, the network device comprises a controller in communication with one or more virtual machines that is configured to (i) receive the network traffic from the traffic device, (ii) monitor a behavior of a first virtual machine of the one or more virtual machines in response to processing of the network traffic within the first virtual machine, (iii) identify at least one anomalous behavior as an unexpected occurrence in the monitored behavior by accelerating activities caused by the network traffic to reduce time for detecting time-delayed malicious traffic, and (iv) determine, based on the identified anomalous behavior, the presence of the time-delayed malicious traffic in the network traffic,

wherein the controller accelerating the activities by at least intercepting one or more time-sensitive system calls and modifying one or more responses to the one or more of the system calls so as to accelerate the activities in the first virtual machine caused by the network traffic.

2. The system of claim 1, wherein the one or more time-sensitive system calls are generated by a software program in response to the network traffic, the software program being executed by the first virtual machine.

3. The system of claim 2, wherein the one or more time-sensitive system calls by the network traffic is a time-of-day system call, and the response to the time-of-day system call by the first virtual machine is to specify at least one of a future time and future date.

4. The system of claim 1, wherein the controller is further configured to (i) intercept one or more time-sensitive system calls generated by a software program in response to the network traffic, the software program being executed by the first virtual machine, and (ii) modify the intercepted one or more time-sensitive system calls so as to accelerate the activities in the first virtual machine caused by the network traffic.

5. The system of claim 4, wherein the one or more time-sensitive system calls is a sleep system call having a time parameter, which is modified so as to accelerate the activities in the first virtual machine.

6. The system of claim 1, wherein the controller is further configured to (i) identify a time consuming program loop executing in the first virtual machine, and (ii) accelerate execution of the time consuming program loop in the first virtual machine.

7. The system of claim 6, wherein the controller is configured to accelerate execution of the time consuming program loop by increasing a priority of execution of the time consuming program loop in the first virtual machine so to accelerate the activities caused by the network traffic in the first virtual machine.

8. The system of claim 1, wherein the traffic device is further configured to filter the received network traffic by passing the received network traffic to the network device that contains at least one characteristic associated with time-delayed malicious traffic.

9. The system of claim 8, wherein the traffic device is further configured to pass the received network traffic that satisfies a heuristic threshold using heuristic analysis.

10. The system of claim 1, wherein the controller is further configured as an intrusion detection system which is configured to generate an intrusion alert in response to detecting an anomalous behavior.

11. The system of claim 10, wherein the controller is further configured to generate a recovery script for the time-delayed malicious traffic to disable the malicious traffic routed over the communication network.

12. The system of claim 1, wherein the malicious traffic includes one or more computer worms.

13. The system of claim 1, wherein the traffic device is physically separate from the network device.

14. The system of claim 1, wherein the traffic device comprises a hardware device physically coupled to the communication network.

15. The system of claim 1, wherein the network device is configured to monitor the behavior of the first virtual machine by being further configured to orchestrate a plurality of network activities during processing of the received network traffic within the first virtual machine.

16. A computer implemented method comprising:

monitoring, by a network device, operatively coupled with a controller, a behavior of network traffic within a first virtual machine of one or more virtual machines, wherein the first virtual machine is configured to accelerate activities caused by the network traffic during processing thereof to reduce time for detecting time-delayed malicious traffic;

identifying at least one anomalous behavior as an unexpected occurrence in the monitored behavior; and

determining the presence of the time-delayed malicious traffic based on the anomalous behavior,

wherein accelerating of the activities by the first virtual machine includes intercepting one or more time-sensitive system calls generated by a software program in response to the network traffic, the software program being executed by the first virtual machine, and modifying the one or more responses to the one or more of the system calls so as to accelerate the activities in the first virtual machine caused by the network traffic.

17. The method of claim 16, wherein the one or more time-sensitive system calls by the network traffic is a time-of-day system call, and the response to the time-of-day system call by the first virtual machine is to specify at least one of a future time and future date.

18. The computer implemented method of claim 16, wherein prior to monitoring the behavior of the network traffic, the method further comprising:

analyzing, by a traffic device, network traffic from a communication network having one or more characteristics associated with the time-delayed malicious traffic; and

submitting, by the traffic device, the network traffic for subsequent analysis to at least the first virtual machine of the one or more virtual machines when the network traffic from the communication network is suspected of having characteristics associated with the time-delayed malicious traffic.

19. The computer implemented method of claim 18, wherein submitting select portions of the network traffic from the communication network subsequently as the network traffic when the network traffic from the communication network satisfies a heuristic threshold as determined using heuristics analysis.

20. The computer implemented method of claim 16, further comprising generating an intrusion alert in response to detecting the anomalous behavior.

21. The method of claim 20, further including generating a recovery script for the time-delayed malicious traffic to disable the malicious traffic routed over the communication network.

22. The computer implemented method of claim 16, wherein the malicious traffic includes one or more computer worms.

23. The computer implemented method of claim 16, further comprising filtering by a traffic device a portion of the network traffic having the one or more characteristics associated with malicious network traffic, the portion comprising the network traffic submitted subsequently to the first virtual machine of the one or more virtual machines.

24. The computer implemented method of claim 16, further comprising orchestrating network activities in the at least one of the plurality of virtual machines;

and wherein monitoring behaviors of the at least one of the plurality of virtual machines comprises monitoring a behavior in response to the orchestrated network activities.

25. A computer implemented method comprising:

monitoring, by a network device, operatively coupled with a controller, a behavior of network traffic within a first virtual machine of one or more virtual machines, wherein the first virtual machine is configured to accelerate activities caused by the network traffic during processing thereof to reduce time for detecting time-delayed malicious traffic;

identifying at least one anomalous behavior as an unexpected occurrence in the monitored behavior; and

determining the presence of the time-delayed malicious traffic based on the anomalous behavior,

wherein accelerating of the activities by the first virtual machine includes (i) intercepting one or more time-sensitive system calls generated by a software program in response to network traffic, the software program being executed by the first virtual machine, and (ii) modifying the one or more of the system calls so to accelerate the activities in the first virtual machine caused by the network traffic.

26. The computer implemented method of claim 25, wherein the one or more time-sensitive system calls is a sleep system call, and modifying the one or more system calls comprises modifying a parameter of the sleep system call by the first virtual machine so as to accelerate the activities in the first virtual machine.

27. The computer implemented method of claim 25, wherein prior to monitoring the behavior of the network traffic, the method further comprising:

analyzing, by a traffic device, network traffic from a communication network having one or more characteristics associated with the time-delayed malicious traffic; and

submitting, by the traffic device, the network traffic for subsequent analysis to at least the first virtual machine of the one or more virtual machines when the network traffic from the communication network is suspected of having characteristics associated with the time-delayed malicious traffic.

28. The computer implemented method of claim 27, wherein submitting select portions of the network traffic from the communication network subsequently as the network traffic when the network traffic from the communication network satisfies a heuristic threshold as determined using heuristics analysis.

29. The computer implemented method of claim 25, further comprising generating an intrusion alert in response to detecting the anomalous behavior.

30. The method of claim 29, further including generating a recovery script for the time-delayed malicious traffic to disable the malicious traffic routed over the communication network.

31. The computer implemented method of claim 25, wherein the malicious traffic includes one or more computer worms.

32. The computer implemented method of claim 25, further comprising filtering by a traffic device a portion of the network traffic having the one or more characteristics associated with malicious network traffic, the portion comprising the network traffic submitted subsequently to the first virtual machine of the one or more virtual machines.

33. The computer implemented method of claim 25, further comprising orchestrating network activities in the at least one of the plurality of virtual machines;

and wherein monitoring behaviors of the at least one of the plurality of virtual machines comprises monitoring a behavior in response to the orchestrated network activities.

34. A computer implemented method comprising:

monitoring, by a network device, operatively coupled with a controller, a behavior of network traffic within a first virtual machine of one or more virtual machines, wherein the first virtual machine is configured to accelerate activities caused by the network traffic during processing thereof to reduce time for detecting time-delayed malicious traffic;

identifying at least one anomalous behavior as an unexpected occurrence in the monitored behavior; and

determining the presence of the time-delayed malicious traffic based on the anomalous behavior,

wherein accelerating of the activities by the first virtual machine comprises (i) identifying a time consuming program loop executing in the first virtual machine, and (ii) accelerating execution of the time consuming program loop in the first virtual machine.

35. The computer implemented method of claim 34, wherein accelerating execution of the time consuming program loop comprises increasing a priority of execution of the time consuming program loop in the first virtual machine so to accelerate the activities caused by the network traffic in the first virtual machine.

36. A computer implemented method comprising:

analyzing, by a traffic device, network traffic received over a communication network;

submitting, by the traffic device, at least select network traffic within the communication network having one or more characteristics associated with a time-delayed malicious traffic for a subsequent analysis; and

performing the subsequent analysis in a network device includes, accelerating activities caused by the network traffic so as to reduce time for detecting time-delayed malicious traffic in at least one of a plurality of virtual machines, identifying one or more anomalous behaviors as an unexpected occurrence, and determining, based on the identified anomalous behavior, the presence of the time-delayed malicious traffic in the network traffic,

wherein the accelerating of the activities in the at least one of the plurality of virtual machines includes at least one of

(A) intercepting one or more time-sensitive system calls generated by a software program in response to the network traffic, the software program being executed by the at least one of the plurality of virtual machines, and either (i) modifying the one or more responses to the one or more of the system calls so as to accelerate the activities in the at least one of the plurality of virtual machines or (ii) modifying the one or more of the system calls so to accelerate the activities in the at least one of the plurality of virtual machines, and

(B) identifying a time consuming program loop executing in the at least one of the plurality of virtual machines, and (ii) accelerating execution of the time consuming program loop in the at least one of the plurality of virtual machines.

37. The computer implemented method of claim 36, wherein performing the subsequent analysis in a network device further includes:

generating a recovery script for the time-delayed malicious traffic, and generating an intrusion alert in response to detecting the anomalous behavior.

38. A system comprising:

a traffic device configured to receive network traffic over a communication network; and

a network device in communication with the traffic device, the network device comprises a controller in communication with one or more virtual machines that is configured to (i) receive the network traffic from the traffic device, (ii) monitor a behavior of a first virtual machine of the one or more virtual machines in response to processing of the network traffic within the first virtual machine, (iii) identify at least one anomalous behavior as an unexpected occurrence in the monitored behavior by accelerating activities caused by the network traffic to reduce time for detecting time-delayed malicious traffic, and (iv) determine, based on the identified anomalous behavior, the presence of the time-delayed malicious traffic in the network traffic,

wherein the controller to accelerate the activities by at least intercepting one or more time-sensitive system calls and modifying the intercepted one or more time-sensitive system calls so as to accelerate the activities in the first virtual machine caused by the network traffic.

39. The system of claim 38, wherein the controller intercepting the one or more time-sensitive system calls that are generated by a software program in response to the network traffic, the software program being executed by the first virtual machine.

40. The system of claim 38, wherein the one or more time-sensitive system calls is a sleep system call having a time parameter, which is modified so as to reduce time for detecting time-delayed malicious traffic.

41. A system comprising:

a traffic device configured to receive network traffic over a communication network; and

a network device in communication with the traffic device, the network device comprises a controller in communication with one or more virtual machines that is configured to (i) receive the network traffic from the traffic device, (ii) monitor a behavior of a first virtual machine of the one or more virtual machines in response to processing of the network traffic within the first virtual machine, (iii) identify at least one anomalous behavior as an unexpected occurrence in the monitored behavior by at least (a) identifying a time consuming program loop executing in the first virtual machine, and (b) accelerating execution of the time consuming program loop in the first virtual machine in order to reduce time for detecting time-delayed malicious traffic, and (iv) determine, based on the identified anomalous behavior, the presence of the time-delayed malicious traffic in the network traffic.

42. The system of claim 41, wherein the controller is configured to accelerate execution of the time consuming program loop by increasing a priority of execution of the time consuming program loop in the first virtual machine.