Verifying network attack detector effectiveness
First Claim
Patent Images
1. A method, comprising:
- identifying, by a coordinator device in a network, a type of network attack;
determining, by the coordinator device, a verification schedule during which an attack classifier executed by a device in the network is to be tested;
coordinating, by the coordinator device, a validation test for the attack classifier for execution during the verification schedule and for the identified type of network attack, wherein the validation test includes instructing the device in the network to;
classify a set of network traffic that includes traffic observed by the device and attack traffic specified by the coordinator device;
generate classification results based on the classified set of network traffic; and
provide the classification results to the coordinator device,wherein the attack traffic and the observed traffic are received from one or more other devices in the network, and wherein the coordinator device instructs the one or more other devices to send the attack traffic at a low priority;
receiving, at the coordinator device, results of the validation test from the device; and
evaluating, by the coordinator device, a performance of the attack classifier based on the results of the validation test, wherein evaluating includes determining when the performance of the attack classifier is above a specified performance threshold to determine whether the attack classifier is still adequately able to detect an attack.
0 Assignments
0 Petitions
Accused Products
Abstract
In one embodiment, a device receives a classifier tracking request from a coordinator device that specifies a classifier verification time period. During the classifier verification time period, the device classifies a set of network traffic that includes traffic observed by the device and attack traffic specified by the coordinator device. The device generates classification results based on the classified set of network traffic and provides the classification results to the coordinator device.
35 Citations
20 Claims
-
1. A method, comprising:
-
identifying, by a coordinator device in a network, a type of network attack; determining, by the coordinator device, a verification schedule during which an attack classifier executed by a device in the network is to be tested; coordinating, by the coordinator device, a validation test for the attack classifier for execution during the verification schedule and for the identified type of network attack, wherein the validation test includes instructing the device in the network to; classify a set of network traffic that includes traffic observed by the device and attack traffic specified by the coordinator device; generate classification results based on the classified set of network traffic; and provide the classification results to the coordinator device, wherein the attack traffic and the observed traffic are received from one or more other devices in the network, and wherein the coordinator device instructs the one or more other devices to send the attack traffic at a low priority; receiving, at the coordinator device, results of the validation test from the device; and evaluating, by the coordinator device, a performance of the attack classifier based on the results of the validation test, wherein evaluating includes determining when the performance of the attack classifier is above a specified performance threshold to determine whether the attack classifier is still adequately able to detect an attack.
-
-
2. The method as in claim 1, wherein coordinating the validation test comprises:
instructing the device to evaluate a particular set of attack traffic during the validation test by mixing the attack traffic with traffic observed by the device.
-
3. The method as in claim 1, wherein coordinating the validation test comprises:
instructing a set of one or more network nodes to send attack traffic to the device during the validation test.
-
4. The method as in claim 3, wherein the instruction causes the set of one or more network nodes to flag the attack traffic as testing traffic.
-
5. The method as in claim 3, wherein the set of one or more network nodes deprioritize the attack traffic with respect to other network traffic.
-
6. The method as in claim 1, wherein the set of one or more network nodes are instructed to use a different routing topology than a normal routing topology during the validation test.
-
7. The method as in claim 1, wherein the verification schedule is authorized by a network policy engine.
-
8. An apparatus, comprising:
-
one or more network interfaces to communicate with a network; a processor coupled to the network interfaces and configured to execute one or more processes; and a memory configured to store a process executable by the processor, the process when executed operable to; identify a type of network attack; determine a verification schedule during which an attack classifier executed by a device in the network is to be tested; coordinate a validation test for the attack classifier for execution during the verification schedule and for the identified type of network attack, wherein the validation test includes instructing the device in the network to; classify a set of network traffic that includes traffic observed by the device and attack traffic specified by the coordinator device; generate classification results based on the classified set of network traffic; and provide the classification results to the coordinator device, wherein the attack traffic and the observed traffic are received from one or more other devices in the network, and wherein the coordinator device instructs the one or more other devices to send the attack traffic at a low priority; receive results of the validation test from the device; and evaluate a performance of the attack classifier based on the results of the validation test, wherein evaluating includes determining when the performance of the attack classifier is above a specified performance threshold to determine whether the attack classifier is still adequately able to detect an attack.
-
-
9. The apparatus as in claim 8, wherein the validation test is coordinated by:
instructing the device to evaluate a particular set of attack traffic during the validation test by mixing the attack traffic with traffic observed by the device.
-
10. The apparatus as in claim 8, wherein the validation test is coordinate by:
instructing a set of one or more network nodes to send attack traffic to the device during the validation test.
-
11. The apparatus as in claim 10, wherein the instruction causes the set of one or more network nodes to flag the attack traffic as testing traffic.
-
12. The apparatus as in claim 10, wherein the set of one or more network nodes deprioritize the attack traffic with respect to other network traffic.
-
13. The apparatus as in claim 8, wherein the set of one or more network nodes are instructed to use a different routing topology than a normal routing topology during the validation test.
-
14. The apparatus as in claim 8, wherein the verification schedule is authorized by a network policy engine.
-
15. A tangible non-transitory computer readable medium storing program instructions that cause a computer to execute a process, the process comprising:
-
identifying, as a coordinator device in a network, a type of network attack; determining, as the coordinator device, a verification schedule during which an attack classifier executed by a device in the network is to be tested; coordinating, as the coordinator device, detection validation test for the attack classifier for execution during the verification schedule and for the identified type of network attack, wherein the validation test includes instructing the device in the network to; classify a set of network traffic that includes traffic observed by the device and attack traffic specified by the coordinator device; generate classification results based on the classified set of network traffic; and provide the classification results to the coordinator device, wherein the attack traffic and the observed traffic are received from one or more other devices in the network, and wherein the coordinator device instructs the one or more other devices to send the attack traffic at a low priority; receiving, as the coordinator device, results of the validation test from the device; and evaluating, as the coordinator device, a performance of the attack classifier based on the results of the validation test, wherein evaluating includes determining when the performance of the attack classifier is above a specified performance threshold to determine whether the attack classifier is still adequately able to detect an attack.
-
-
16. The tangible non-transitory computer readable medium as in claim 15, wherein coordinating the validation test comprises:
instructing the device to evaluate a particular set of attack traffic during the validation test by mixing the attack traffic with traffic observed by the device.
-
17. The tangible non-transitory computer readable medium as in claim 15, wherein coordinating the validation test comprises:
instructing a set of one or more network nodes to send attack traffic to the device during the validation test.
-
18. The tangible non-transitory computer readable medium as in claim 17, wherein the instruction causes the set of one or more network nodes to flag the attack traffic as testing traffic.
-
19. The tangible non-transitory computer readable medium as in claim 17, wherein the set of one or more network nodes deprioritize the attack traffic with respect to other network traffic.
-
20. The tangible non-transitory computer readable medium as in claim 15, wherein the set of one or more network nodes are instructed to use a different routing topology than a normal routing topology during the validation test.
Specification