Techniques for botnet detection and member identification

• US 10,673,719 B2
• Filed: 02/24/2017
• Issued: 06/02/2020
• Est. Priority Date: 02/25/2016
• Status: Active Grant

• Current Assignee: Imperva Incorporated (Thales SA)
• Original Assignee: Imperva Incorporated (Thales SA)
• Inventors: Niv, Nitzan
• Primary Examiner(s): Zarrineh, Shahriar
• Application Number:
  US15/442,582
• Publication Number:
  US 20170251005A1
• Time in Patent Office: 1,194 Days
• Field of Search: 726 23
• CPC Class Codes: H04L 2463/144 Detection or countermeasures against botnets
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 61/5007 Internet protocol [IP] addresses
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

A botnet identification module identifies members of one or more botnets based upon network traffic destined to one or more servers over time, and provides sets of botnet sources to a traffic monitoring module. Each set of botnet sources includes a plurality of source identifiers of end stations acting as part of a corresponding botnet. A traffic monitoring module receives the sets of botnet sources from the botnet identification module, and upon a receipt of traffic identified as malicious that was sent by a source identified within one of the sets of botnet sources, activates a protection mechanism with regard to all traffic from all of the sources identified by the one of the sets of botnet sources for an amount of time.

Claims

1. A method in a botnet identification module (BIM) that is implemented by an electronic device and that is for identifying a subset of a plurality of end stations that collectively act as a suspected botnet, the method comprising:

obtaining, by the BIM, traffic data from one or more traffic monitoring modules (TMMs) implemented by one or more electronic devices, wherein the traffic data includes or is based upon a plurality of request messages that were originated by ones of the plurality of end stations and that were destined to one or more servers, wherein each of the one or more TMMs is deployed in front of at least one of the one or more servers in that the TMM receives all network traffic originated by the plurality of end stations that is destined for the at least one server;

generating, by the BIM based upon the obtained traffic data, a set of identifiers corresponding to the subset of the plurality of end stations that are determined by the BIM to be collectively acting as the suspected botnet in that they have transmitted request messages, destined for one or more of the one or more servers, that collectively or individually satisfy one or more security rules which, when satisfied, indicate a malicious attack, wherein the set of identifiers comprises a plurality of identifiers, wherein said generating the set of identifiers comprises;

identifying, from the traffic data, a subset of the plurality of request messages that are malicious,

generating an attacking-clusters graph based on the subset of request messages that is identified as being malicious, wherein the attacking-clusters graph includes vertices representing source clusters, wherein the attacking-clusters graph further includes edges between vertices in adjacent levels of the attacking-clusters graph, wherein an edge between vertices indicates that source clusters represented by those vertices share one or more common sources, wherein the vertices in the attacking-clusters graph are each assigned a weight representing a number of sources in a source cluster represented by that vertex, and wherein the edges in the attacking-clusters graph are each assigned a weight representing a number of common sources between source clusters represented by vertices connected by that edge, and

determining, based upon analyzing the attacking-clusters graph, that the subset of end stations have collectively performed the malicious attack for at least a threshold number of consecutive time periods and that at least a threshold number of the subset of end stations have been involved in the malicious attack for each of the threshold number of consecutive time periods, wherein the analyzing includes finding paths in the attacking-clusters graph that only contain edges with a threshold weight, that only pass through vertices with a threshold weight, and that have a length longer than a threshold length; and

transmitting, by the BIM, the set of identifiers to the one or more TMMs to cause the one or more TMMs to utilize the set of identifiers while analyzing subsequent request messages destined to the one or more servers to detect an attack from the suspected botnet and to protect the one or more servers from the attack.

2. The method of claim 1, wherein said generating the attacking-clusters graph comprises:

generating a plurality of event data structures corresponding to the subset of request messages, wherein each of the plurality of event data structures includes a source identifier of a source of the corresponding request message and a destination identifier of a destination of the corresponding request message;

identifying a plurality of groupings of the generated event data structures, wherein each of the plurality of groupings corresponds to a time period of a plurality of different time periods and includes those of the generated event data structures corresponding to a request message that was received at a corresponding one of the one or more TMMs within the time period; and

identifying, for each of the different time periods, groupings of source identifiers of the request messages corresponding to the event data structures of the time period,

wherein generating the set of identifiers is based upon analyzing the groupings of source identifiers of each of the different time periods.

3. The method of claim 2, wherein said identifying the groupings of source identifiers for each of the different time periods comprises:

calculating a similarity score between each pair of source identifiers of those of the event data structures of the time period that have a same attack type, wherein the similarity score is based upon the destination identifiers of those event data structures that include one of the pair of source identifiers; and

clustering, based upon the similarity scores, the source identifiers of those of the event data structures of the time period that have the same attack type.

4. The method of claim 2, wherein said generating the set of identifiers further comprises:

removing, from the set of identifiers, any identifier existing within a set of whitelisted source identifiers.

5. The method of claim 1, further comprising:

determining, by the BIM based at least in part upon the obtained traffic data, an attack duration of the suspected botnet, wherein the attack duration is an average attack duration or a maximum attack duration observed over a plurality of attacks observed from the suspected botnet; and

transmitting the attack duration of the suspected botnet to the one or more TMMs to cause the one or more TMMs to utilize the attack duration while protecting the one or more servers from the suspected botnet.

6. The method of claim 1, wherein each identifier of the set of identifiers comprises an Internet Protocol (IP) address.

7. A non-transitory computer readable storage medium having instructions which, when executed by one or more processors of an electronic device, cause the electronic device to implement a botnet identification module (BIM) that performs operations for identifying a subset of a plurality of end stations that collectively act as a suspected botnet, the operations comprising:

obtaining traffic data from one or more traffic monitoring modules (TMMs) implemented by one or more electronic devices, wherein the traffic data includes or is based upon a plurality of request messages that were originated by ones of the plurality of end stations and that were destined to one or more servers, wherein each of the one or more TMMs is deployed in front of at least one of the one or more servers in that the TMM receives all network traffic originated by the plurality of end stations that is destined for the at least one server;

generating, based upon the obtained traffic data, a set of identifiers corresponding to the subset of the plurality of end stations that are determined by the BIM to be collectively acting as the suspected botnet in that they have transmitted request messages, destined for one or more of the one or more servers, that collectively or individually satisfy one or more security rules which, when satisfied, indicate a malicious attack, wherein the set of identifiers comprises a plurality of identifiers, wherein said generating the set of identifiers comprises;

identifying, from the traffic data, a subset of the plurality of request messages that are malicious,

generating an attacking-clusters graph based on the subset of request messages that is identified as being malicious, wherein the attacking-clusters graph includes vertices representing source clusters, wherein the attacking-clusters graph further includes edges between vertices in adjacent levels of the attacking-clusters graph, wherein an edge between vertices indicates that source clusters represented by those vertices share one or more common sources, wherein the vertices in the attacking-clusters graph are each assigned a weight representing a number of sources in a source cluster represented by that vertex, and wherein the edges in the attacking-clusters graph are each assigned a weight representing a number of common sources between source clusters represented by vertices connected by that edge, and

determining, based upon analyzing the attacking-clusters graph, that the subset of end stations have collectively performed the malicious attack for at least a threshold number of consecutive time periods and that at least a threshold number of the subset of end stations have been involved in the malicious attack for each of the threshold number of consecutive time periods; and

transmitting the set of identifiers to the one or more TMMs to cause the one or more TMMs to utilize the set of identifiers while analyzing subsequent request messages destined to the one or more servers to detect an attack from the suspected botnet and to protect the one or more servers from the attack.

8. The non-transitory computer readable storage medium of claim 7, wherein said generating the attacking-clusters graph comprises:

generating a plurality of event data structures corresponding to the subset of request messages, wherein each of the plurality of event data structures includes a source identifier of a source of the corresponding request message and a destination identifier of a destination of the corresponding request message;

identifying a plurality of groupings of the generated event data structures, wherein each of the plurality of groupings corresponds to a time period of a plurality of different time periods and includes those of the generated event data structures corresponding to a request message that was received at a corresponding one of the one or more TMMs within the time period; and

identifying, for each of the different time periods, groupings of source identifiers of the request messages corresponding to the event data structures of the time period,

wherein generating the set of identifiers is based upon analyzing the groupings of source identifiers of each of the different time periods.

9. The non-transitory computer readable storage medium of claim 8, wherein said identifying the groupings of source identifiers for each of the different time periods comprises:

calculating a similarity score between each pair of source identifiers of those of the event data structures of the time period that have a same attack type, wherein the similarity score is based upon the destination identifiers of those event data structures that include one of the pair of source identifiers; and

clustering, based upon the similarity scores, the source identifiers of those of the event data structures of the time period that have the same attack type.

10. The non-transitory computer readable storage medium of claim 8, wherein said generating the set of identifiers further comprises:

removing, from the set of identifiers, any identifier existing within a set of whitelisted source identifiers.

11. The non-transitory computer readable storage medium of claim 7, further comprising:

determining, based at least in part upon the obtained traffic data, an attack duration of the suspected botnet, wherein the attack duration is an average attack duration or a maximum attack duration observed over a plurality of attacks observed from the suspected botnet; and

transmitting the attack duration of the suspected botnet to the one or more TMMs to cause the one or more TMMs to utilize the attack duration while protecting the one or more servers from the suspected botnet.

12. The non-transitory computer readable storage medium of claim 7, wherein each identifier of the set of identifiers comprises an Internet Protocol (IP) address.

13. An electronic device, comprising:

one or more processors; and

one or more non-transitory computer readable storage media having instructions which, when executed by the one or more processors, cause the electronic device to implement a botnet identification module (BIM) to identify a subset of a plurality of end stations that collectively act as a suspected botnet, the BIM to;

obtain traffic data from one or more traffic monitoring modules (TMMs) implemented by one or more electronic devices, wherein the traffic data includes or is based upon a plurality of request messages that were originated by ones of the plurality of end stations and that were destined to one or more servers, wherein each of the one or more TMMs is deployed in front of at least one of the one or more servers in that the TMM receives all network traffic originated by the plurality of end stations that is destined for the at least one server;

generate, based upon the obtained traffic data, a set of identifiers corresponding to the subset of the plurality of end stations that are determined by the BIM to be collectively acting as the suspected botnet in that they have transmitted request messages, destined for one or more of the one or more servers, that collectively or individually satisfy one or more security rules which, when satisfied, indicate a malicious attack, wherein the set of identifiers comprises a plurality of identifiers, wherein to generate the set of identifiers the BIM is to;

identify, from the traffic data, a subset of the plurality of request messages that are malicious,

generate an attacking-clusters graph based on the subset of request messages that is identified as being malicious, wherein the attacking-clusters graph includes vertices representing source clusters, wherein the attacking-clusters graph further includes edges between vertices in adjacent levels of the attacking-clusters graph, wherein an edge between vertices indicates that source clusters represented by those vertices share one or more common sources, wherein the vertices in the attacking-clusters graph are each assigned a weight representing a number of sources in a source cluster represented by that vertex, and wherein the edges in the attacking-clusters graph are each assigned a weight representing a number of common sources between source clusters represented by vertices connected by that edge, and

determine, based upon analyzing the attacking-clusters graph, that the subset of end stations have collectively performed the malicious attack for at least a threshold amount of time which includes a threshold number of consecutive time periods and that at least a threshold number of the subset of end stations have been involved in the malicious attack for each of the threshold number of time periods within the threshold amount of time, wherein the analyzing includes finding paths in the attacking-clusters graph that only contain edges with a threshold weight, that only pass through vertices with a threshold weight, and that have a length longer than a threshold length; and

transmit the set of identifiers to the one or more TMMs to cause the one or more TMMs to utilize the set of identifiers while analyzing subsequent request messages destined to the one or more servers to detect an attack from the suspected botnet and to protect the one or more servers from the attack.

14. The electronic device of claim 13, wherein the BIM, to determine that the subset of end stations have collectively performed the malicious attack for at least the threshold amount of time and that at least the threshold number of the subset of end stations have been involved in the malicious attack for each of the threshold number of time periods within the threshold amount of time, is to:

generate a plurality of event data structures corresponding to the subset of request messages, wherein each of the plurality of event data structures includes a source identifier of a source of the corresponding request message and a destination identifier of a destination of the corresponding request message;

identify a plurality of groupings of the generated event data structures, wherein each of the plurality of groupings corresponds to a time period of a plurality of different time periods and includes those of the generated event data structures corresponding to a request message that was received at a corresponding one of the one or more TMMs within the time period; and

identify, for each of the different time periods, groupings of source identifiers of the request messages corresponding to the event data structures of the time period,

wherein said generation of the set of identifiers is based upon analyzing the groupings of source identifiers of each of the different time periods.

15. The electronic device of claim 14, wherein the BIM, to identify the groupings of source identifiers for each of the different time periods, is to:

calculate a similarity score between each pair of source identifiers of those of the event data structures of the time period that have a same attack type, wherein the similarity score is based upon the destination identifiers of those event data structures that include one of the pair of source identifiers; and

cluster, based upon the similarity scores, the source identifiers of those of the event data structures of the time period that have the same attack type.

16. The electronic device of claim 14, wherein the BIM, to generate the set of identifiers, is to further:

remove, from the set of identifiers, any identifier existing within a set of whitelisted source identifiers.

17. The electronic device of claim 13, wherein the BIM is further to:

determine, based at least in part upon the obtained traffic data, an attack duration of the suspected botnet, wherein the attack duration is an average attack duration or a maximum attack duration observed over a plurality of attacks observed from the suspected botnet; and

transmit the attack duration of the suspected botnet to the one or more TMMs to cause the one or more TMMs to utilize the attack duration while protecting the one or more servers from the suspected botnet.