Systems and methods for identifying similar hosts

• US 9,836,512 B1
• Filed: 02/23/2017
• Issued: 12/05/2017
• Est. Priority Date: 05/11/2016
• Status: Active Grant

• Current Assignee: Acalvio Technologies, Inc.
• Original Assignee: Acalvio Technologies, Inc.
• Inventors: Singh, Satnam, Kosgi, Santosh, Gopalakrishna, Rajendra
• Primary Examiner(s): Corrielus, Jean M
• Application Number:
  US15/440,451
• Publication Number:
  US 20170329783A1
• Time in Patent Office: 285 Days
• Field of Search: 726 22, 726 23, 726 24, 726 25, 709224
• CPC Class Codes: G06F 16/2255 Hash tables
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 21/566 Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities
• CPC Class Codes: G06F 21/577 Assessing vulnerabilities and evaluating computer system security
• CPC Class Codes: G06F 2221/034 Test or assess a computer or a system
• CPC Class Codes: G06F 2221/2101 Auditing as a secondary aspect
• CPC Class Codes: G06F 2221/2127 Bluffing
• CPC Class Codes: H04L 63/1433 Vulnerability analysis
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/1491 using deception as countermeasure, e.g. honeypots, honeynets, decoys or entrapment

Abstract

Systems and methods for identifying potentially compromised devices using attributes of a known compromised device may be provided. In one embodiment, an attribute set can be constructed for the compromised hosts using data from these logs. Weights can be assigned to each attribute in the attribute set initially, and further weights can be learned using audits by a user. This attribute set can be used in the disclosed systems and methods for identifying hosts that are similar to compromised hosts. The similar items can be used as hosts for deception mechanisms, can be taken off the network as being likely compromised or likely to become compromised, or quarantined.

Claims

1. A computer-implemented method comprising:

determining a query item, wherein the query item is associated with a compromised host of a plurality of hosts;

selecting an attribute associated with the query item;

assigning an attribute weight to the attribute;

identifying a query attribute value associated with the attribute and the query item;

weighting the query attribute value using the attribute weight;

determining a first distance between the weighted query attribute value and a random value;

identifying a candidate item, wherein the candidate item includes a host of the plurality of hosts;

identifying a candidate attribute value associated with the attribute and the candidate item;

weighting the candidate attribute value using the attribute weight;

determining a second distance between the weighted candidate attribute value and the random value;

determining a third distance between the first distance and the second distance; and

characterizing the candidate item as a similar item to the query item when the third distance is within a threshold value, wherein when input corresponding to feedback is received, a similarity value for the similar item is generated using the feedback, and the attribute weight is automatically optimized using the similarity value.

2. The computer-implemented method of claim 1, further comprising:

receiving feedback confirming similarity of the similar item to the query item;

generating the similarity value for the similar item and the query item using the feedback; and

optimizing the attribute weight using the similarity value.

3. The computer-implemented method of claim 1, wherein determining a query item comprises:

identifying compromised hosts from the plurality of hosts;

determining a cluster for the compromised hosts, wherein the cluster includes a cluster centroid, and wherein the cluster includes similar compromised hosts;

computing a cluster quality parameter for the cluster, wherein the cluster quality parameter is based on a scatter of the cluster;

weighting the cluster centroid with the cluster quality parameter to form a population centroid of the compromised hosts; and

characterizing the population centroid as the query item.

4. The computer-implemented method of claim 1, wherein determining a query item comprises:

identifying compromised hosts from the plurality of hosts machine;

determining clusters for the compromised hosts, wherein each cluster includes a cluster centroid, and wherein each cluster includes similar compromised hosts;

computing a cluster quality parameter for each cluster, wherein each cluster quality parameter is based on a scatter of a corresponding cluster;

weighting the cluster centroid of each cluster with a corresponding cluster quality parameter;

summing the weighted cluster centroids to form a population centroid of the compromised hosts; and

characterizing the population centroid as the query item.

5. The computer-implemented method of claim 1, wherein the first distance and the second distance are Euclidian distances.

6. The computer-implemented method of claim 1, wherein determining the first distance includes computing a first hash function and determining the second distance includes computing a second hash function.

7. The computer-implemented method of claim 6, further comprising:

generating buckets of hash functions including the first hash function and the second hash function; and

determining that the first hash function and the second hash function are in a same bucket.

8. The computer-implemented method of claim 1, wherein the query item includes a plurality of attributes.

9. The computer-implemented method of claim 1, wherein identifying the query attribute value includes receiving the query attribute value from a logging agent.

10. The computer-implemented method of claim 1, wherein identifying the candidate attribute value includes receiving the candidate attribute value from a logging agent.

11. The computer-implemented method of claim 1, further comprising:

normalizing the query attribute value and the candidate attribute value.

12. A network device comprising:

one or more processors; and

a non-transitory computer-readable medium containing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including;

determining a query item, wherein the query item is associated with a compromised host of a plurality of hosts;

selecting an attribute associated with the query item;

assigning an attribute weight to the attribute;

identifying a query attribute value associated with the attribute and the query item;

weighting the query attribute value using the attribute weight;

determining a first distance between the weighted query attribute value and a random value;

identifying a candidate item, wherein the candidate item includes a host of the plurality of hosts;

identifying a candidate attribute value associated with the attribute and the candidate item;

weighting the candidate attribute value using the attribute weight;

determining a second distance between the weighted candidate attribute value and the random value;

determining a third distance between the first distance and the second distance; and

characterizing the candidate item as a similar item to the query item when the third distance is within a threshold value, wherein when input corresponding to feedback is received, a similarity value for the similar item is generated using the feedback, and the attribute weight is automatically optimized using the similarity value.

13. The network device of claim 12, wherein the operations further include:

receiving feedback confirming similarity of the similar item to the query item;

generating the similarity value for the similar item and the query item using the feedback; and

optimizing the attribute weight using the similarity value.

14. The network device of claim 12, wherein determining a query item comprises:

identifying compromised hosts from the plurality of hosts;

determining a cluster for the compromised hosts, wherein the cluster includes a cluster centroid, and wherein the cluster includes similar compromised hosts;

computing a cluster quality parameter for the cluster, wherein the cluster quality parameter is based on a scatter of the cluster;

weighting the cluster centroid with the cluster quality parameter to form a population centroid of the compromised hosts; and

characterizing the population centroid as the query item.

15. The network device of claim 12, wherein determining a query item comprises:

identifying compromised hosts from the plurality of hosts;

determining clusters for the compromised hosts, wherein each cluster includes a cluster centroid, and wherein each cluster includes similar compromised hosts;

computing a cluster quality parameter for each cluster, wherein each cluster quality parameter is based on a scatter of a corresponding cluster;

weighting the cluster centroid of each cluster with a corresponding cluster quality parameter;

summing the weighted cluster centroids to form a population centroid of the compromised hosts; and

characterizing the population centroid as the query item.

16. The network device of claim 12, wherein the first distance and the second distance are Euclidian distances.

17. The network device of claim 12, wherein determining the first distance includes computing a first hash function and determining the second distance includes computing a second hash function.

18. The network device of claim 17, wherein the operations further include:

generating buckets of hash functions including the first hash function and the second hash function; and

determining that the first hash function and the second hash function are in a same bucket.

19. The network device of claim 12, wherein the query item includes a plurality of attributes.

20. The network device of claim 12, wherein identifying the query attribute value includes receiving the query attribute value from a logging agent.

21. The network device of claim 12, wherein identifying the candidate attribute value includes receiving the candidate attribute value from a logging agent.

22. The network device of claim 12, wherein the operations further include:

normalizing the query attribute value and the candidate attribute value.

23. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium of a network device, including instructions that, when executed by the one or more processors, cause the one or more processors to:

determine a query item, wherein the query item is associated with a compromised host of a plurality of hosts;

select an attribute associated with the query item;

assign an attribute weight to the attribute;

identify a query attribute value associated with the attribute and the query item;

weight the query attribute value using the attribute weight;

determine a first distance between the weighted query attribute value and a random value;

identify a candidate item, wherein the candidate item includes a host of the plurality of hosts;

identify a candidate attribute value associated with the attribute and the candidate item;

weight the candidate attribute value using the attribute weight;

determine a second distance between the weighted candidate attribute value and the random value;

determine a third distance between the first distance and the second distance; and

characterize the candidate item as a similar item to the query item when the third distance is within a threshold value, wherein when input corresponding to feedback is received, a similarity value for the similar item is generated using the feedback, and the attribute weight is automatically optimized using the similarity value.

24. The computer-program product of claim 23, wherein the instructions further cause the one or more processors to:

receive feedback confirming similarity of the similar item to the query item;

generate the similarity value for the similar item and the query item using the feedback; and

optimize the attribute weight using the similarity value.

25. The computer-program product of claim 23, wherein the instruction to determine a query item comprises:

identify compromised hosts from the plurality of hosts;

determine a cluster for the compromised hosts, wherein the cluster includes a cluster centroid, and wherein the cluster includes similar compromised hosts;

compute a cluster quality parameter for the cluster, wherein the cluster quality parameter is based on a scatter of the cluster;

weight the cluster centroid with the cluster quality parameter to form a population centroid of the compromised hosts; and

characterize the population centroid as the query item.

26. The computer-program product of claim 23, wherein the instruction to determine a query item comprises:

identify compromised hosts from the plurality of hosts;

determine clusters for the compromised hosts, wherein each cluster includes a cluster centroid, and wherein each cluster includes similar compromised hosts;

compute a cluster quality parameter for each cluster, wherein each cluster quality parameter is based on a scatter of a corresponding cluster;

weight the cluster centroid of each cluster with a corresponding cluster quality parameter;

sum the weighted cluster centroids to form a population centroid of the compromised hosts; and

characterize the population centroid as the query item.

27. The computer-program product of claim 23, wherein the first distance and the second distance are Euclidian distances.

28. The computer-program product of claim 23, wherein determining the first distance includes computing a first hash function and determining the second distance includes computing a second hash function.

29. The computer-program product of claim 28, wherein the instructions further cause the one or more processors to:

generate buckets of hash functions including the first hash function and the second hash function; and

determine that the first hash function and the second hash function are in a same bucket.

30. The computer-program product of claim 23, wherein the query item includes a plurality of attributes.