Distributed malware detection system and submission workflow thereof

• US 10,616,266 B1
• Filed: 09/30/2016
• Issued: 04/07/2020
• Est. Priority Date: 03/25/2016
• Status: Active Grant

• Current Assignee: Musarubra US LLC (Musarubra US SellCo LLC)
• Original Assignee: FireEye, Inc. (Alphabet Inc.)
• Inventors: Otvagin, Alexander
• Primary Examiner(s): Dada, Beemnet W
• Application Number:
  US15/283,206
• Time in Patent Office: 1,285 Days
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
• CPC Class Codes: H04L 63/18 using different networks or channels, e.g. using out of band channels cryptographic mechanisms or cryptographic arrangements for key distribution involving distinctive intermediate devices or communication paths H04L9/0827; cryptographic mechanisms or cryptographic arrangements for authentication using a plurality of channels H04L9/3215

Abstract

A submission process for a malware detection system including one or more sensors and a cluster including one or more computing nodes is described. The process includes the sensor that determines whether a prior malware analysis has been conducted on any previously submitted object matching the object under analysis. If not, the process determines whether the object is suspicious, namely a first probability of the first object being associated with malware. If suspicious, metadata associated with the suspicious object is sent to an analysis coordinator of a first computing node of the cluster. The metadata is used in determining whether a prior malware analysis has been previously conducted within the cluster on any object that matches the suspicious object. The metadata is also used in fetching, by an object analyzer of the same or a different computing node of the cluster, the suspicious object from the sensor for malware analysis.

Claims

1. A computerized method performed by a distributed malware detection system, the method comprising:

intercepting, by a sensor operating as a network device, traffic in a first network;

performing, by the sensor, a preliminary analysis on each of a plurality of objects extracted from the traffic to identify one or more suspicious objects included in the traffic and store metadata associated with the one or more suspicious objects, the one or more suspicious objects include a first suspicious object;

sending, by the sensor, over a second network, at least the first suspicious object and metadata associated with the first suspicious object to an analysis coordinator of a first computing node of a plurality of computing nodes forming a cluster, the cluster includes the plurality of computing nodes interconnected by a third network;

performing, by an object analyzer of a second computing node of the plurality of computing nodes deployed as part of the cluster, a malware analysis on the first suspicious object of the one or more suspicious objects received by the cluster, the second computing node being different from the first computing node;

sending, over the second network, results of the malware analysis to the sensor; and

sending, by the sensor to a management system, security information associated with at least the first suspicious object in the event the malware analysis indicates the first suspicious object is associated with a cybersecurity attack.

2. The method of claim 1, wherein the first network is a different network than the second network and the third network.

3. The method of claim 1, wherein the security information for the first suspicious object includes a hash value of the first suspicious object, information identifying that the first suspicious object is associated with a cybersecurity attack, or information that identifies a severity of the malware associated with the first suspicious object.

4. The method of claim 1, wherein each of the plurality of computing nodes includes one or more hardware processors.

5. The method of claim 1,

wherein the cluster includes the plurality of computing nodes interconnected by the third network, each of the plurality of computing nodes including an analysis coordinator communicatively coupled with the second network and the third network, and the object analyzer communicatively coupled with the analysis coordinator, and

wherein the performing of the malware analysis includes performing, on the first suspicious object, the malware analysis by the object analyzer of the second computing node of the plurality of computing nodes, the second computing node being communicatively coupled with the analysis coordinator of the first computing node.

6. The method of claim 5, wherein the third network is different than the second network.

7. The method of claim 1, wherein the first network comprises a private network in which the sensor is operationally located, and the sending of at least the first suspicious object comprises sending the first suspicious object over a public network that comprises the second network in secure communications.

8. The method of claim 1, wherein the cluster includes the plurality of computing nodes interconnected over the third network, each of the plurality of computing nodes including an analysis coordinator and an object analyzer.

9. The method of claim 1, wherein the sending, by the sensor, over the second network, at least the first suspicious object and the metadata associated with the first suspicious object to the cluster comprises sending, by the sensor, the metadata of the first suspicious object to the analysis coordinator without sending the first suspicious object until requested by the cluster.

10. The method of claim 9, further comprising:

storing, by the analysis coordinator, the metadata associated with the first suspicious object including an object identifier (ID) and a sensor ID,

using, by the cluster, the object ID to identify the first suspicious object for tracking the malware analysis and obtaining the first suspicious object if necessary from the sensor; and

using, by the cluster, the sensor ID to identify the sensor with whom to communicate to obtain the first suspicious object if necessary and to send to the sensor, over the second network, results of the malware analysis of the first suspicious object.

11. The method of claim 9, further comprising:

performing, by the object analyzer, an analysis on the metadata associated with the first object, the analysis comprises determining whether an analysis has previously been performed on an object matching the first suspicious object.

17. The method of claim 1, wherein the sending of at least the first suspicious object and the metadata associated with the first suspicious object to the cluster comprises sending suspicious objects, including at least the first suspicious object, over a first communication path within the cluster and sending the metadata associated with the suspicious objects, including at least the metadata associated with the first suspicious object, over a second communication path within the cluster, the second communication path being different from the first communication path.

12. A computerized method performed by a distributed malware detection system, the method comprising:

intercepting, by a sensor operating as a network device, traffic in a first network;

performing, by the sensor, a preliminary analysis on each of a plurality of objects extracted from the traffic to identify suspicious objects included in the traffic and store metadata associated with the suspicious objects, wherein the performing of the preliminary analysis on each of the plurality of objects comprises (i) determining whether a first object of the plurality of objects has at least a first probability of being associated with malware, and (ii) determining whether the first object has at least a second probability greater than the first probability that the first object is associated with malware in response to determining that the first object corresponds to a second object of the plurality of objects that has been previously determined to be associated with malware;

sending, by the sensor, over a second network, the suspicious objects and metadata associated with the suspicious objects to a cluster, the cluster includes the plurality of computing nodes interconnected by a third network and each computing device includes one or more processors;

performing, by an object analyzer deployed as part of the cluster, a malware analysis on the suspicious objects received by the cluster;

sending, over the second network, results of the malware analysis to the sensor; and

sending, by the sensor to a management system, security information associated a malware analysis for the second object as the security information associated with the malware analysis for the first object.

13. A computerized method performed by a distributed malware detection system, the method comprising:

intercepting, by a sensor operating as a network device, traffic in a first network;

performing, by the sensor, a preliminary analysis on each of a plurality of objects extracted from the traffic to identify suspicious objects included in the traffic and store metadata associated with the suspicious objects;

sending, by the sensor, over a second network, the suspicious objects and the metadata associated with the suspicious objects to a cluster, the cluster includes the plurality of computing nodes interconnected by a third network, wherein the sending of the suspicious objects and the metadata associated with the suspicious objects further comprises;

receiving metadata associated with a first suspicious object of the suspicious objects by an analysis coordinator of a first computing node of the cluster, and

queuing, by the analysis coordinator, a portion of the received metadata for subsequent retrieval by the object analyzer, the object analyzer being operationally located at either (i) the first computing node of the cluster or (ii) a second computing node of the cluster different from the first computing node;

performing, by an object analyzer deployed as part of the cluster, a malware analysis on the suspicious objects received by the cluster;

sending, over the second network, results of the malware analysis to the sensor; and

sending, by the sensor to a management system, security information associated with the suspicious objects in the event the malware analysis indicates any of the suspicious objects is associated with malware.

14. The method of claim 13, wherein the sending of the suspicious objects and metadata associated with the suspicious objects to the cluster further comprising:

accessing the metadata within an entry of the queue by the object analyzer and using the metadata to retrieve the first suspicious object from the sensor, wherein the object analyzer accessing the metadata upon the object analyzer determining to be qualified to conduct the malware analysis on the first suspicious object.

15. The method of claim 14, wherein the object analyzer determined to be qualified to conduct the malware analysis in response to the object analyzer determining to support a predetermined level of processing capacity and support an object type pertaining to the first suspicious object.

16. The method of claim 13, wherein the sending of the suspicious objects and metadata associated with the suspicious objects to the cluster further comprising:

receiving, by the sensor, a request for the first suspicious object from either (i) the object analyzer of the first computing node of the cluster over the first network when the portion of the metadata is retrieved by the object analyzer of the first computing node or (ii) the second computing node of the cluster when the portion of the metadata is retrieved by the second computing node; and

sending the first suspicious object from the sensor to either (i) the object analyzer of the first computing node of the cluster when the request for the first suspicious object is received from the object analyzer of the first computing node or (ii) the object analyzer of the second computing node of the cluster when the request for the first suspicious object is received from the second computing node.

18. A distributed malware detection system comprising:

one or more sensors, each sensor of the one or more sensors and includes processor and a storage medium and is operationally located to intercept traffic in an associated network or sub-network, wherein each sensor of the one or more sensors to perform a preliminary analysis on a plurality of objects extracted from the traffic of the associated network to identify suspicious objects included in the traffic and store metadata associated with the suspicious objects; and

a cluster including a plurality of computing nodes, each computing node of the plurality of computing nodes includes an analysis coordinator and an object analyzer, the plurality of computing nodes includes at least a first computing node operating as a broker computing node in which a first analysis coordinator of the first computing node is placed in an active state and at least a second computing node operating as an analytic computing node in which an analysis coordinator of the second computing node is placed in an inactive state,

wherein the first analysis coordinator of the first computing node of the plurality of computing node is communicatively coupled with the one or more sensors to receive, over a second network, at least the metadata associated with the suspicious objects, and

wherein a first object analyzer of either (i) the first computing node or (ii) a second computing node of the plurality of computing nodes to perform malware analysis on the suspicious objects received by the cluster from the one or more sensors and to return results of the malware analyses to corresponding sensors of the one or more sensors that provided the suspicious objects.

19. The distributed malware detection system of claim 18, wherein the one or more sensors comprise a plurality of sensors.

20. The distributed malware detection system of claim 18, wherein each of plurality of computing nodes forming the cluster comprises the analysis coordinator communicatively coupled to the broker computing node.

21. The distributed malware detection system of claim 18, wherein a sensor of the one or more sensors is configured to transmit, over the second network, metadata associated with a first suspicious object to the first analysis coordinator of the cluster without sending the first suspicious object until requested by the cluster.

22. The distributed malware detection system of claim 21,

wherein the first analysis coordinator to store the metadata associated with the first suspicious object including an object identifier (ID) and a sensor ID, and

wherein the first object analyzer to use (i) the object ID to identify the first suspicious object for the malware analysis and obtain the first suspicious object if necessary from the sensor and (ii) the sensor ID to identify the sensor with whom to communicate to obtain the first suspicious object if necessary and to send to the sensor, over the second network, the results of the malware analysis of the first suspicious object.

23. The distributed malware detection system of claim 21, wherein the first object analyzer to perform an analysis on the metadata associated with the first suspicious object, the analysis comprises determining whether an analysis has previously been performed on an object matching the first suspicious object.

24. The distributed malware detection system of claim 18, wherein each of the one or more sensors to perform the preliminary analysis on the plurality of objects by at least (i) determining whether a first object of the plurality of objects has at least a first probability of being associated with malware, and (ii) determining whether the first object has at least a second probability greater than the first probability that the first object is associated with malware in response to determining that the first object corresponds to a second object of the plurality of objects that has been previously determined to be associated with malware.

25. The distributed malware detection system of claim 18, wherein the first analysis coordinator to receive at least metadata associated a first suspicious object of the suspicious objects from a first sensor of the one or more sensors and store a portion of the metadata associated with the first suspicious object within a queue for subsequent retrieval by the first object analyzer.

26. The distributed malware detection system of claim 25, wherein the first object analyzer to access the portion of the metadata within an entry of the queue and use the portion of the metadata to retrieve the first suspicious object from the first sensor, wherein the first object analyzer to access the portion of the metadata associated with the first suspicious object upon the first object analyzer determining to be qualified to conduct the malware analysis on the first suspicious object.

27. The distributed malware detection system of claim 26, wherein the first object analyzer determined to be qualified to conduct the malware analysis in response to the first object analyzer determining to support a predetermined level of processing capacity and support an object type pertaining to the first suspicious object.

28. The distributed malware detection system of claim 25, wherein the first sensor to transmit the metadata associated the first suspicious object over a first communication path within the cluster and transmit the first suspicious object over a second communication path within the cluster, the second communication path being different from the first communication path.