Systematic mining of associated server herds for uncovering malware and attack campaigns

• US 9,088,598 B1
• Filed: 11/14/2013
• Issued: 07/21/2015
• Est. Priority Date: 11/14/2013
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: Narus, Inc. (Gen Digital Inc.)
• Inventors: Saha, Sabyasachi, Lee, Sung-Ju, Zhang, Jialong, Gu, Guofei, Nardelli, Bruno
• Primary Examiner(s): Powers, William
• Application Number:
  US14/080,225
• Time in Patent Office: 614 Days
• Field of Search: 726/22
• US Class Current: 1/1
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00

Abstract

A method for detecting malicious servers. The method includes analyzing network traffic data to generate a main similarity measure and a secondary similarity measure for each server pair found in the network traffic data, extracting a main subset and a secondary subset of servers based on the main similarity measure and the secondary similarity measure, identifying a server that belongs to the main subset and the secondary subset, and determining a suspicious score of the server based on at least a first similarity density measure of the main subset, a second similarity density measure of the secondary subset, and a commonality measure of the main subset and the secondary subset.

Claims

1. A method for detecting malicious servers, comprising:

analyzing, by a processor of a computer system, network traffic data to generate a main similarity measure for each server pair of a plurality of servers found in the network traffic data, wherein the main similarity measure represents first similarity between two servers of the server pair based on a plurality of clients found in the network traffic data that communicate to both of the two servers using at least a portion of the network traffic data;

extracting, by the processor and based on a first pre-determined algorithm, a main subset representing a portion of the plurality of servers based on the main similarity measure;

analyzing, by the processor, the network traffic data to generate a plurality of secondary similarity measures for each server pair of the plurality of servers, wherein a secondary similarity measure of the plurality of secondary similarity measures represents second similarity between the two servers of the server pair based on a pre-determined criterion;

extracting, by the processor and based on a second pre-determined algorithm, a secondary subset representing another portion of the plurality of servers based on the secondary similarity measure;

identifying a server of the plurality of servers that belongs to the main subset and the secondary subset; and

determining, by the processor, a suspicious score of the server based on at least a first similarity density measure of the main subset, a second similarity density measure of the secondary subset, and a commonality measure of the main subset and the secondary subset.

2. The method of claim 1, further comprising:

determining, based at least on the suspicious score, a joint set of the main subset and the secondary subset as an associated group of the malicious servers collectively engaging in a malicious HTTP activity,

wherein the network traffic data comprises HTTP traffic data.

3. The method of claim 2, further comprising:

extracting, based on the second pre-determined algorithm, another secondary subset of the plurality of servers based on one of the plurality of secondary similarity measures, wherein the one of the plurality of secondary similarity measures is same as or separate from the secondary similarity measure;

determining another joint set of the main subset and the another secondary subset as another associated group of the malicious servers collectively engaging in another malicious HTTP activity;

merging, subsequent to determining the joint set as the associated group of the malicious servers and determining the another joint set as the another associated group of the malicious servers, the joint set and the another joint to generate an aggregate joint set; and

determining, in response to the merging, the malicious HTTP activity and another malicious HTTP activity as portions of a single malicious attack campaign.

4. The method of claim 2, further comprising:

identifying the server as further belonging to an additional number of secondary subsets each extracted based on one of the plurality of secondary similarity measures that is separate from the secondary similarity measure,

wherein determining the suspicious score of the server is further based on the second similarity density measure of each of the additional number of secondary subsets and the commonality measure of the main subset and each of the additional number of secondary subsets, and

wherein the suspicious score is proportional to a count of the additional number of secondary subsets.

5. The method of claim 1, wherein the pre-determined criterion for generating the secondary similarity measure comprises at least one selected from a group consisting of file similarity, IP similarity, and domain registration similarity between the two servers.

6. The method of claim 1, wherein extracting the main subset of the plurality of servers comprises:

representing the plurality of servers as nodes in a graph having edges each weighted by the main similarity measure of a corresponding server pair; and

partitioning the graph into a plurality of clusters by maximizing a modularity of the plurality of clusters,

wherein the main subset is one of the plurality of clusters, and

wherein the first similarity density measure of the main subset comprises a count of the edges in the graph divided by a maximum number of possible edges in the graph.

7. The method of claim 1, wherein extracting the secondary subset of the plurality of servers comprises:

representing the plurality of servers as nodes in a graph having edges each weighted by the secondary similarity measure of a corresponding server pair; and

partitioning the graph into a plurality of clusters by maximizing a modularity of the plurality of clusters,

wherein the secondary subset is one of the plurality of clusters, and

wherein the second similarity density measure of the secondary subset comprises a count of the edges in the graph divided by a maximum number of possible edges in the graph.

8. A system for detecting malicious servers, comprising:

a processor of a computer system;

memory comprising instructions executable by the processor, wherein the instructions comprises;

a main similarity analyzer configured to;

analyze network traffic data to generate a main similarity measure for each server pair of a plurality of servers found in the network traffic data, wherein the main similarity measure represents first similarity between two servers of the server pair based on a plurality of clients found in the network traffic data that communicate with both of the two servers using at least a portion of the network traffic data; and

extract, based on a first pre-determined algorithm, a main subset representing a portion of the plurality of servers based on the main similarity measure;

a secondary similarity analyzer configured to;

analyze the network traffic data to generate a plurality of secondary similarity measures for each server pair of the plurality of servers, wherein a secondary similarity measure of the plurality of secondary similarity measures represents second similarity between the two servers of the server pair based on a pre-determined criterion; and

extract, based on a second pre-determined algorithm, a secondary subset representing another portion of the plurality of servers based on the secondary similarity measure; and

a correlation analyzer configured to;

identify a server of the plurality of servers that belongs to the main subset and the secondary subset; and

determine a suspicious score of the server based on at least a first similarity density measure of the main subset, a second similarity density measure of the secondary subset, and a commonality measure of the main subset and the secondary subset; and

a repository configured to store the main subset, the secondary subset, the first similarity density measure of the main subset, the second similarity density measure of the secondary subset, and the commonality measure of the main subset and the secondary subset.

9. The system of claim 8, further comprising an attack campaign inference engine configured to:

determine, based at least on the suspicious score, a joint set of the main subset and the secondary subset as an associated group of the malicious servers collectively engaging in a malicious HTTP activity,

wherein the network traffic data comprises HTTP traffic data.

10. The system of claim 9,

wherein the secondary similarity analyzer is further configured to:

extract, based on the second pre-determined algorithm, another secondary subset of the plurality of servers based on one of the plurality of secondary similarity measures, wherein the one of the plurality of secondary similarity measures is same as or separate from the secondary similarity measure, and

wherein the correlation analyzer is further configured to;

determine another joint set of the main subset and the another secondary subset as another associated group of the malicious servers collectively engaging in another malicious HTTP activity;

merge, subsequent to determining the joint set as the associated group of the malicious servers and determining the another joint set as the another associated group of the malicious servers, the joint set and the another joint to generate an aggregate joint set; and

determine, in response to the merging, the malicious HTTP activity and another malicious HTTP activity as portions of a single malicious attack campaign.

11. The system of claim 9, wherein the correlation analyzer is further configured to:

identify the server as further belonging to an additional number of secondary subsets each extracted based on one of the plurality of secondary similarity measures that is separate from the secondary similarity measure,

wherein determining the suspicious score of the server is further based on the second similarity density measure of each of the additional number of secondary subsets and the commonality measure of the main subset and each of the additional number of secondary subsets, and

wherein the suspicious score is proportional to a count of the additional number of secondary subsets.

12. The system of claim 8, wherein the pre-determined criterion for generating the secondary similarity measure comprises at least one selected from a group consisting of file similarity, IP similarity, and domain registration similarity between the two servers.

13. The system of claim 8, wherein extracting the main subset of the plurality of servers comprises:

representing the plurality of servers as nodes in a graph having edges each weighted by the main similarity measure of a corresponding server pair; and

partitioning the graph into a plurality of clusters by maximizing a modularity of the plurality of clusters,

wherein the main subset is one of the plurality of clusters, and

wherein the first similarity density measure of the main subset comprises a count of the edges in the graph divided by a maximum number of possible edges in the graph.

14. The system of claim 8, wherein extracting the secondary subset of the plurality of servers comprises:

representing the plurality of servers as nodes in a graph having edges each weighted by the secondary similarity measure of a corresponding server pair; and

partitioning the graph into a plurality of clusters by maximizing a modularity of the plurality of clusters,

wherein the secondary subset is one of the plurality of clusters, and

wherein the second similarity density measure of the secondary subset comprises a count of the edges in the graph divided by a maximum number of possible edges in the graph.

15. A non-transitory computer readable medium embodying instructions for detecting malicious servers, the instructions when executed by a processor comprising functionality for:

analyzing network traffic data to generate a main similarity measure for each server pair of a plurality of servers found in the network traffic data, wherein the main similarity measure represents first similarity between two servers of the server pair based on a plurality of clients found in the network traffic data that communicate to both of the two servers using at least a portion of the network traffic data;

extracting, based on a first pre-determined algorithm, a main subset representing a portion of the plurality of servers based on the main similarity measure;

analyzing the network traffic data to generate a plurality of secondary similarity measures for each server pair of the plurality of servers, wherein a secondary similarity measure of the plurality of secondary similarity measures represents second similarity between the two servers of the server pair based on a pre-determined criterion;

extracting, based on a second pre-determined algorithm, a secondary subset representing another portion of the plurality of servers based on the secondary similarity measure;

identifying a server of the plurality of servers that belongs to the main subset and the secondary subset; and

determining a suspicious score of the server based on at least a first similarity density measure of the main subset, a second similarity density measure of the secondary subset, and a commonality measure of the main subset and the secondary subset.

16. The non-transitory computer readable medium of claim 15, the instructions when executed by the processor further comprising functionality for:

determining, based at least on the suspicious score, a joint set of the main subset and the secondary subset as an associated group of the malicious servers collectively engaging in a malicious HTTP activity,

wherein the network traffic data comprises HTTP traffic data.

17. The non-transitory computer readable medium of claim 16, the instructions when executed by the processor further comprising functionality for:

extracting, based on the second pre-determined algorithm, another secondary subset of the plurality of servers based on one of the plurality of secondary similarity measures, wherein the one of the plurality of secondary similarity measures is same as or separate from the secondary similarity measure;

determining another joint set of the main subset and the another secondary subset as another associated group of the malicious servers collectively engaging in another malicious HTTP activity;

merging, subsequent to determining the joint set as the associated group of the malicious servers and determining the another joint set as the another associated group of the malicious servers, the joint set and the another joint to generate an aggregate joint set; and

determining, in response to the merging, the malicious HTTP activity and another malicious HTTP activity as portions of a single malicious attack campaign.

18. The non-transitory computer readable medium of claim 16, the instructions when executed by the processor further comprising functionality for:

identifying the server as further belonging to an additional number of secondary subsets each extracted based on one of the plurality of secondary similarity measures that is separate from the secondary similarity measure,

wherein determining the suspicious score of the server is further based on the second similarity density measure of each of the additional number of secondary subsets and the commonality measure of the main subset and each of the additional number of secondary subsets, and

wherein the suspicious score is proportional to a count of the additional number of secondary subsets.

19. The non-transitory computer readable medium of claim 15, wherein the pre-determined criterion for generating the secondary similarity measure comprises at least one selected from a group consisting of file similarity, IP similarity, and domain registration similarity between the two servers.

20. The non-transitory computer readable medium of claim 15, wherein extracting the main subset of the plurality of servers comprises:

representing the plurality of servers as nodes in a graph having edges each weighted by the main similarity measure of a corresponding server pair; and

partitioning the graph into a plurality of clusters by maximizing a modularity of the plurality of clusters,

wherein the main subset is one of the plurality of clusters, and

wherein the first similarity density measure of the main subset comprises a count of the edges in the graph divided by a maximum number of possible edges in the graph.