Technique for malware detection capability comparison of network security devices

• US 10,587,647 B1
• Filed: 11/22/2016
• Issued: 03/10/2020
• Est. Priority Date: 11/22/2016
• Status: Active Grant

• Current Assignee: Fireeye Security Holdings US LLC
• Original Assignee: FireEye, Inc.
• Inventors: Khalid, Yasir, Shahbaz, Nadeem
• Primary Examiner(s): Gracia, Gary S
• Application Number:
  US15/358,688
• Time in Patent Office: 1,204 Days
• Field of Search: 726 11, 726 22, 726 23
• CPC Class Codes: G06F 2009/45587 Isolation or security of virtual machine instances
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 21/56 Computer malware detection or handling, e.g. anti-virus arrangements
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 63/0245 Filtering by information in the payload
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1433 Vulnerability analysis
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms

Abstract

A testing technique tests and compares malware detection capabilities of network security devices, such as those commercially available from a variety of cyber-security vendors. Testing is conducted on test samples in a “blind” fashion, where the security devices do not know beforehand whether the test samples are “live” malware or benign network traffic. The test samples are received from a remote server and potentially represent malicious attacks against a testing network. Notably, for truly blind testing, embodiments of the testing technique employ a mixture of malware and benign test samples, as well as addressing subterfuge, to prevent the security devices from being able to reliably determine maliciousness of the test samples based on a source of any of the samples.

Claims

1. A method comprising:

generating and sending, by a virtualized endpoint that is operating within a test console being part of a testing network, one or more requests to acquire a plurality of samples from a remote server, each request having a uniform resource locator (URL) indicating a domain;

modifying, at the test console including a hardware processor and a memory each generated request to re-direct the request to the remote server and obscure address information associated with the request to render ineffective malware detection through address blacklisting by one or more units under test (UUTs) while maintaining the URL of the request as an alias destination domain;

receiving a response to each request by the virtualized endpoint, wherein a copy of the response being redirected to each of the one or more UUTs and the response including one or more samples, each sample including an object;

processing the object for each of the one or more samples at the virtualized endpoint and at each of the one or more UUTs to detect whether the object is either malicious or benign; and

generating a report to compare and contrast detection efficacy of the UUTs in correctly identifying each of the objects.

2. The method of claim 1 wherein the alias destination domain corresponds to a domain that is known to be associated with a source of malware.

3. The method of claim 1 wherein the remote server is communicatively coupled to the endpoint over a public network.

4. The method of claim 1 wherein the modifying of each generated request comprises:

modifying a header of each request to include an internet protocol (IP) address of the remote server.

5. The method of claim 1 wherein the plurality of samples includes a mixture of objects identified as benign and objects identified as malicious;

and wherein maintaining the URL of the request as the alias destination domain further renders malware detection by the one or more UUTs by source address blacklisting ineffective.

6. The method of claim 1 further comprising:

receiving the response by a switch;

sending, by the switch, the response to the endpoint and a copy of the response to each of the one or more UUTs.

7. The method of claim 1 further comprising:

running the endpoint in a first virtual machine on the test console operating as a test computer; and

running a proxy in a second virtual machine on the test console, the proxy modifying each generated request.

8. The method of claim 7 further comprising:

determining whether the endpoint in the first virtual machine is compromised by using indicators of compromise that are one of received from the remote server and generated by an agent executing on the endpoint.

9. The method of claim 1 further comprising:

filtering the request using a rules-based gateway to protect a production network coupled to the testing network.

10. The method of claim 1, further comprising:

operating a gateway to control outbound communication triggered by processing the object by allowing only select communications to be sent over a public network from the endpoint.

11. The method of claim 1 further comprising:

authenticating the endpoint at the server as entitled to receive the plurality of samples indicated in the one or more requests, and wherein a sample database being stored at the remote server to maintain the plurality of samples.

12. The method of claim 1 further comprising:

using one or more portions of the URL as an index to obtain the one or more samples from the server in response to the request.

13. The method of claim 1 wherein the object is detected to be malicious as being part of a multi-flow attack comprising a plurality of flows, and the report indicates whether each of the UUTs detects each of the plurality of flows of the multi-flow attack.

14. The method of claim 1 further comprising:

providing a test computer to run a test application, and the endpoint comprises a virtualized endpoint instantiated by the test application, and displaying a comparison of the one or more UUTs on a computer display comprising the report.

15. The method of claim 4 wherein the IP address of the remote server is a public IP address where the remote server is connected to the endpoint over a public network and resides on the public network.

16. The method of claim 4 wherein the IP address of the remote server is a private IP address where the remote server is connected to the endpoint over and resides on a private network.

17. The method of claim 1 further comprising:

receiving indicators of compromise (IOCs) at a test application running on the test console of the testing network, the received IOCs serving as a baseline of first behaviors associated with the object processed by the endpoint; and

comparing the received IOCs with IOCs having second behaviors monitored during processing of the object in the endpoint to determine whether the object constitutes a malicious object.

18. The method of claim 17 wherein the endpoint corresponds to the virtualized endpoint including a detection agent configured to monitor for the second behaviors during processing of the object.

19. The method of claim 1 further comprising:

receiving indicators of compromise (IOCs) at a test application running on a test console of the testing network, the received IOC'"'"'s serving as a baseline of first behaviors monitored during processing of the object in the endpoint; and

comparing the received IOCs with IOCs having second behaviors detected by each of the one or more UUTs to determine the efficacy of each of the one or more UUTs in detecting whether one or more objects provided in each copy of the response is malicious or benign.

20. A system comprising:

a server storing a plurality of samples including objects identified as benign and objects identified as malicious; and

a test computer coupled to the server via a network, the test computer having a first processor, the test computer configured to;

send one or more requests over the network to acquire the plurality of samples, each request of the one or more requests includes a uniform resource locator (URL),

modify each request of the one or more requests to re-direct the request to the server by at least obscuring address information associated with the request to render ineffective malware detection through address blacklisting by one or more security devices,

receive a response to each request from the server with a copy of the response being redirected to the one or more security devices, the response including one or more samples, each sample having an object for processing by the test computer,

process the object to detect whether the object is either malicious or benign, and

generate a report of detection of the object for each of the plurality of samples as either malicious or benign.

21. The system of claim 20 wherein the test computer is further configured to:

modify a header of each request to include an internet protocol (IP) address of the server.

22. The system of claim 20 wherein the plurality of samples includes a mixture of objects identified as benign and objects identified as malicious that render malware detection by address blacklisting ineffective for one or more units under test (UUTs), each of the one or more UUTs being configured to receive a copy of the response to each request from the server.

23. The system of claim 20 wherein the network is a testing network isolated from a private network.

24. The system of claim 20 wherein the first processor of the test computer is adapted to execute software in one or more virtual machines.

25. The system of claim 20 wherein the server further comprises:

a second processor adapted to execute an authenticating proxy configured to authenticate the test computer as entitled to receive the one or more test samples indicated in the one or more requests, and wherein the sample database is a secure database.

26. The system of claim 20 wherein the server further comprises:

a second processor adapted to execute a content engine configured to use one or more portions of the URL as an index to obtain a sample from the database stored at the server.

27. The system of claim 20 wherein the malware objects includes advance malware developed to avoid detection.

28. The system of claim 20 wherein the network is a testing network isolated from a production network, and further comprising:

a switch communicatively coupled to the one or more UUTs, the switch being configured to generate the copy of the response and provide the copy of the response to each of the one or more UUT.

29. The system of claim 24 wherein the test computer is further configured to:

determine whether a virtualized endpoint in a first virtual machine of the one or more virtual machines is compromised by using indicators of compromise received from the server.

30. The system of claim 20, wherein each request of the one or more requests is re-directed while maintaining the URL of the request as an alias destination domain.

31. A non-transitory computer readable media containing instructions for execution on a processor for a method comprising:

modifying a destination internet protocol (IP) address of each request of one or more requests to re-direct the request to the remote server and obscure address information associated with the request to render ineffective malware detection through address blacklisting by one or more units under test (UUTs), wherein each request of the one or more requests initiated by a virtualized endpoint to acquire a plurality of samples stored at a remote server, each request having a uniform resource locator (URL); and

receiving a response to each request at the virtualized endpoint, the response including one or more samples, each sample having an object for processing by the virtualized endpoint, the object being either malicious or benign, wherein

processing the object at the virtualized endpoint to detect whether the object is either malicious or benign, wherein

a copy of the response is provide to the one or more UUTs, wherein a mixture of objects identified as benign and objects identified as malicious provided as part of the response renders malware detection by address blacklisting ineffective for the one or more UUTs; and

a report of detection of the object for each of the plurality of samples identified as malicious or benign by each UUT is generated.

32. The non-transitory computer readable media of claim 31 wherein the remote server is communicatively coupled to the virtualized endpoint over a public network.

33. The non-transitory computer readable media of claim 31 wherein the plurality of samples includes a mixture of objects identified as benign and objects identified as malicious.