Scheduling a network attack to train a machine learning model

• US 10,356,111 B2
• Filed: 01/27/2014
• Issued: 07/16/2019
• Est. Priority Date: 01/06/2014
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Di Pietro, Andrea, Vasseur, Jean-Philippe, Cruz Mota, Javier
• Primary Examiner(s): Chen, Alan
• Assistant Examiner(s): Figueroa, Kevin W
• Application Number:
  US14/164,467
• Publication Number:
  US 20150195145A1
• Time in Patent Office: 1,996 Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: H04L 41/16 using machine learning or artificial intelligence
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 45/48 Routing tree calculation
• CPC Class Codes: H04L 47/127 by using congestion prediction
• CPC Class Codes: H04L 47/2466 using signalling traffic
• CPC Class Codes: H04L 47/41 by acting on aggregated flows or links
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

In one embodiment, a device evaluates a set of training data for a machine learning model to identify a missing feature subset in a feature space of the set of training data. The device identifies a plurality of network nodes eligible to initiate an attack on a network to generate the missing feature subset. One or more attack nodes are selected from among the plurality of network nodes. An attack routine is provided to the one or more attack nodes to cause the one or more attack nodes to initiate the attack. An indication that the attack has completed is then received from the one or more attack nodes.

Claims

1. A method, comprising:

evaluating, by a device, a set of training data for a machine learning model to identify a missing feature subset in a feature space of the set of training data;

identifying, by the device, a plurality of network nodes eligible to initiate an attack on a network to generate the missing feature subset at the machine learning model;

selecting, by the device from among the plurality of network nodes, one or more attack nodes based on the identified missing feature subset and on a traffic matrix associated with the plurality of network nodes, wherin the selected one or more attack nodes are based at least in part on a score of the identified missing feature subset, and the score is a value inversely proportional to a density of observations at a given point;

in response to selecting the one or more attack nodes, generating, by the device an attack routine to be sent to the selected one or more attack nodes that will generate the missing feature subset in the feature space of the set of training data;

in response to identifying the missing feature subset and selecting the one or more attack nodes that will cause the learning machine to generate the missing feature subset, transmitting, by the device, the attack routing to the one or more attack nodes;

instructing, by the device, the one or more attack nodes initiate the attack and generate the missing feature subset in the feature space of the set of training data upon receiving the attack routine; and

receiving, at the device from the one or more attack nodes, an indication that the attack has completed.

2. The method as in claim 1, wherein the machine learning model is an artificial neural network (ANN).

3. The method as in claim 1, further comprising:

sending a notification to a particular attack node that the particular attack node has been selected as an attacker; and

receiving, from the particular attack node, a confirmation that the particular attack node is capable of performing the attack.

4. The method as in claim 1, further comprising:

sending a notification to the one or more attack nodes that indicates a type and scheduled start time for the attack.

5. The method as in claim 1, wherein the attack routine is configured to cause the one or more attack nodes to perform a mixed mode of operation in which a particular attack node performs the attack for some communications and operates normally for other communications.

6. The method as in claim 1, further comprising:

generating the set of training data by randomly selecting attack nodes from among the plurality of network nodes to initiate network attacks.

7. The method as in claim 6, further comprising:

receiving the set of training data from the plurality of network nodes;

maintaining a first histogram of features from network nodes when unaffected by a network attack;

maintaining a second histogram of features from network nodes when affected by a network attack; and

using the first and second histograms to identify the missing feature subset.

8. The method as in claim 1, wherein the attack nodes are selected based on computing resource available at the attack nodes.

9. The method as in claim 1, further comprising:

determining that a particular network node in the plurality will be affected by two or more attacks that are scheduled to occur at the same time; and

scheduling execution times for the two or more attacks so as not to overlap.

10. The method as in claim 9, further comprising:

determining a set of one or more network nodes to be attacked in order to generate the missing feature subset;

providing the set of one or more network nodes to be attacked to a network device configured to initiate network attacks, wherein the network device configured to initiate network attacks compares the set of one or more network nodes to be attacked to a set of one or more network nodes to be attacked via a different attack.

11. The method as in claim 9, further comprising:

identifying the particular network device that will be affected by the two or more attacks by broadcasting information regarding the attacks to the plurality of network nodes.

12. The method as in claim 9, further comprising:

receiving network topology data from the plurality of network nodes; and

using the network topology data to identify the particular network device that will be affected by the two or more attacks.

13. The method as in claim 11, wherein the network nodes in the plurality span two or more computer networks.

14. The method as in claim 9, further comprising:

receiving a set of attack nodes as a unicast message, wherein the particular network node that will be affected by the two or more network attacks is determined by comparing the received set of attack nodes to the selected one or more attack nodes.

15. The method as in claim 9, wherein a particular attack is scheduled to execute first based on an attack priority value associated with the particular attack.

16. An apparatus, comprising:

one or more network interfaces to communicate in a computer network;

a processor coupled to the network interfaces and configured to execute one or more processes; and

a memory configured to store a process executable by the processor, the process when executed operable to;

evaluate a set of training data for a machine learning model to identify a missing a feature subset in a feature space of the set of training data;

identify a plurality of network nodes eligible to initiate an attack on a network to generate the missing feature subset;

select, from among the plurality of network nodes, one or more attack nodes based on the identified missing feature subset and on a traffic matrix associated with the plurality of network nodes, wherein the selected one or more attack nodes are based at least in part on a score of the identified missing feature subset, and the score is a value inversely proportional to a density of observations at a given point;

in response to selecting the one or more attack nodes, generate an attack routine to be sent to the selected one or more attack nodes that will generate the missing feature subset in the feature space of the set of training data;

in response to identification of the missing feature subset and selection of the one or more attack nodes that will cause the learning machine to generate the missing feature subset, transmit the attack routing to the one or more attack nodes;

instruct the one or more attack nodes to initiate the attack and generate the missing feature subset in the feature space of the set of training data upon receiving the attack routine from the apparatus; and

receive, from the one or more attack nodes, an indication that the attack has completed.

17. The apparatus as in claim 16, wherein the process when executed is further operable to:

generate the set of training data by randomly selecting attack nodes from among the plurality of network nodes to initiate network attacks;

receive the set of training data from the plurality of network nodes;

maintain a first histogram of features from network nodes when unaffected by a network attack;

maintain a second histogram of features from network nodes when affected by a network attack; and

use the first and second histograms to identify the missing feature subset.

18. The apparatus as in claim 16, wherein the process when executed is further operable to:

determine that a particular network node in the plurality will be affected by two or more attacks scheduled to occur at the same time; and

schedule execution of the two or more attacks not to overlap.

19. The apparatus as in claim 18, wherein the process when executed is further operable to:

determine a set of one or more network nodes to be attacked in order to generate the missing feature subset;

provide the set of one or more network nodes to be attacked to a network device configured to initiate network attacks, wherein the network device configured to initiate network attacks compare the set of one or more network nodes to be attacked to a set of one or more network nodes to be attacked via a different attack.

20. A tangible, non-transitory, computer-readable media having software encoded thereon, the software when executed by a processor operable to:

evaluate a set of training data for a machine learning model to identify a missing feature subset in a feature space of the set of training data;

identify a plurality of network nodes eligible to initiate an attack on a network to generate the missing feature subset;

select, from among the plurality of network nodes, one or more attack nodes based on the identified missing feature subset and on a traffic matrix associated with the plurality of network nodes, wherein the selected one or more attack nodes are based at least in part on a score of the identified missing feature subset, and the score is a value inversely proportional to a density of observations at a given point;

in response to selecting the one or more attack nodes, generate an attack routine to be sent to the selected one or more attack nodes that will generate the missing feature subset in the feature space of the set of training data;

in response to identification of the missing feature subset and selection of the one or more attack nodes that will cause the learning machine to generate the missing feature subset in the feature space of the set of training data;

in response to identification of the missing feature subset and selection of the one or more attack nodes that will cause the learning machine to generate the missing feature subset, transmit the attack routine to the one or more attack nodes;

instruct the one or more attack nodes to initiate the attack and generate the missing feature subset in the feature space of the set of training data upon receiving the attack routine; and

receive, from the one or more attack nodes, and indication that the attack has completed.