Assessing effectiveness of cybersecurity technologies

• US 10,270,798 B2
• Filed: 10/06/2016
• Issued: 04/23/2019
• Est. Priority Date: 10/08/2015
• Status: Active Grant

• Current Assignee: NS Holdings LLC
• Original Assignee: Siege Technologies, LLC
• Inventors: Zaffarano, Kara, Taylor, Joshua, Hamilton, Samuel
• Primary Examiner(s): Revak, Christopher A
• Application Number:
  US15/286,990
• Publication Number:
  US 20170104780A1
• Time in Patent Office: 929 Days
• Field of Search: None
• CPC Class Codes: H04L 63/1433 Vulnerability analysis
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

A method for assessing effectiveness of one or more cybersecurity technologies in a computer network includes testing each of two or more component stages of an attack model at a first computer network element twice. A first one of the tests is conducted with a first one of the cybersecurity technologies operable to protect the first computer network element, and a second one of the tests is conducted with the first cybersecurity technology not operable to protect the first computer network element. For each one of the twice-tested component stages, comparing results from the first test and the second test, wherein the comparison yields or leads to information helpful in assessing effectiveness of the first cybersecurity technology on each respective one of the twice-tested component stages at the computer network element.

Claims

1. A method for assessing effectiveness of one or more cybersecurity technologies in a computer network, the method comprising:

generating two or more component stages of an attack model based on decomposing one or more attacks;

separately testing each of the two or more component stages at a first computer network element twice,

wherein a first one of the tests is conducted with a first one of the cybersecurity technologies operable to protect the first computer network element,

wherein a second one of the tests is conducted with the first cybersecurity technology not operable to protect the first computer network element, and

wherein the two tests include generating results based on measuring a cost and a benefit associated with operating the first one of the cybersecurity technologies; and

characterizing the effectiveness of the first cybersecurity technology for each one of the twice-tested component stages, based on comparing results from the first test and the second test.

2. The method of claim 1, wherein the component stages of the attack model are selected from the group consisting of reconnaissance, weaponization, delivery, exploitation, installation, command and control, and action on target.

3. The method of claim 1, wherein testing the two or more component stages of the attack model at the computer network element twice, comprises testing all of the component stages of the attack model at the computer network element twice, wherein the component stages of the attack model include reconnaissance, weaponization, delivery, exploitation, installation, command and control, and action on target.

4. The method of claim 1, further comprising:

subsequently testing each of the two or more component stages of the attack model at the first computer network element twice,

wherein a first one of the subsequent tests is conducted with a second one of the cybersecurity technologies operable to protect the first computer network element, and

wherein a second one of the subsequent tests is conducted with the second cybersecurity technology not operable to protect the first computer network element; and

for each one of the subsequently twice-tested component stages, comparing results from the first and second subsequent tests,

wherein the comparison yields or leads to information helpful in assessing effectiveness of the second cybersecurity technology on each respective one of the subsequently twice-tested component stages at the computer network element.

5. The method of claim 4, further comprising:

comparing results of the testing that involved the first cybersecurity technology to the results of the testing that involved the second cybersecurity technology,

wherein comparing the results yields or leads to information helpful in assessing effectiveness of the first cybersecurity system relative to the second cybersecurity technology.

6. The method of claim 1, wherein the testing includes measuring one or more of mission productivity, attack productivity, mission success, attack success, mission confidentiality, attack confidentiality, mission integrity and attack integrity.

7. The method of claim 6, wherein:

the mission productivity relates to a rate at which mission tasks are complete,

the attack productivity relates to a rate at which mission tasks are complete,

the mission success relates to an amount of attempted mission tasks that are successfully completed,

the attack success relates to how successful an attacker may be while attempting to attack the computer network,

the mission confidentiality relates to how much mission information is exposed to the attacker or can be intercepted,

the attack confidentiality relates to how much attacker activity may be visible to detection mechanisms,

the mission integrity relates to how much mission information is transmitted without modification or corruption, and

the attack integrity relates to accuracy of information viewed by the attacker.

8. The method of claim 1, wherein the testing comprises:

measuring statistical interaction effects between decomposed mission, attack and/or defense components;

weighing importance of each effect; and

analyzing weighted importance of missing or uncertain mission, attack and/or defense components.

9. The method of claim 1, wherein the computer network is a computer-implemented virtual model of an actual or planned computer network and the testing is performed in a virtual environment that includes the virtual model of the computer network.

10. The method of claim 9, further comprising:

performing the testing at a plurality of component stages of the attack model at a plurality of different computer network elements simultaneously.

11. The method of claim 1, wherein the computer network is a real world computer network, and wherein the testing comprises instrumenting one or more points in the network.

12. The method of claim 1, further comprising, for each respective one of the tested component stages, considering a plurality of different typical tasks that might lead to an undesirable compromise of network security.

13. The method of claim 1, further comprising, for each respective one of the tested component stages, providing information relevant to effectiveness of the first cybersecurity technology, in terms of one or more of the following:

detection, mitigation and effect on network overhead.

14. The method of claim 1, further comprising:

saving data from each testing performed in computer-based database;

predicting, with a computer-based prediction engine, effectiveness of the first, or another, cybersecurity technology in an untested network based on the data saved in the computer-based database.

15. The method of claim 14, further comprising:

using machine learning at the computer-based prediction engine based on data in the computer-based database to improve prediction capabilities as more data is added to the computer-based database.

16. The method of claim 1, wherein at least one of the tests is conducted on a downstream one of the component stages of the attack model at the computer network element, without also testing one or more upstream component stages of the attack model at the computer network element.

17. A method comprising:

defining a set of attack, mission, and defense elements at a computer network element to test;

posing one or more hypotheses regarding one or more of the defined attack, mission, and defense elements;

generating two or more component stages of an attack model based on decomposing one or more attacks;

executing testing of the one or more hypotheses, wherein executing the testing comprises;

testing each of the two or more component stages a first computer network element twice,

wherein a first one of the tests is conducted with a first one of the defensive cybersecurity technologies operable to protect the first computer network element,

wherein a second one of the tests is conducted with the first defensive cybersecurity technology not operable to protect the first computer network element, and

wherein the two tests include generating results based on measuring a cost and a benefit associated with operating the first one of the defensive cybersecurity technologies; and

analyzing the first computer network element,

wherein analyzing the first computer network element comprises characterizing an effectiveness of the first one of the defensive cybersecurity technologies for each one of the twice-tested component stages based on comparing the results from the first test and the second test; and

identifying one or more missing or uncertain elements.

18. The method of claim 17, wherein the component stages of the attack model are selected from the group consisting of reconnaissance, weaponization, delivery, exploitation, installation, command and control, and action on target.

19. A system comprising:

a computer-based processor and

a computer-based memory coupled to the computer-based processor and having stored thereon instructions executable by the computer-based processor to cause the computer-based processor to facilitate assessing effectiveness of one or more defensive cybersecurity technologies in a computer network,

wherein assessing effectiveness comprises;

generating two or more component stages of an attack model based on decomposing one or more attacks;

testing each of the two or more component stages of an attack model at a first computer network element twice,

wherein a first one of the tests is conducted with a first one of the defensive cybersecurity technologies operable to protect the first computer network element,

wherein a second one of the tests is conducted with the first defensive cybersecurity technology not operable to protect the first computer network and

wherein the two tests include generating results based on measuring a cost and a benefit associated with operating the first defensive cybersecurity technology; and

for each one of the twice-tested component stages, characterizing with the computer-based processor, an effectiveness of the first one of the defensive cybersecurity technologies based on comparing the results from the first test and the second test.

20. The system of claim 19, wherein the component stages of the attack model are selected from the group consisting of reconnaissance, weaponization, delivery, exploitation, installation, command and control, and action on target.