METHODS AND SYSTEMS FOR REDUCING FALSE POSITIVE FINDINGS

• US 20200134194A1
• Filed: 08/26/2019
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Youngberg, Adam, Filbey, David, Fernando, Kishore Prabakaran
• Granted Patent: US 10,929,543 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/3608 using formal methods, e.g. model checking, abstract interpretation theorem proving G06N5/013
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 21/563 by source code analysis
• CPC Class Codes: G06F 21/577 Assessing vulnerabilities and evaluating computer system security
• CPC Class Codes: G06F 2221/033 Test or assess software

Abstract

A system for validating software security analysis findings includes a non-transitory computer readable medium and a processor. The non-transitory computer readable medium stores a source truth dataset including criteria for validating characteristics of findings. The processor receives a finding from a software security analysis tool that performs scan on application code. The processor identifies a characteristic from the finding. The processor selects a criterion from the non-transitory computer readable medium for validating the identified characteristic. The processor determines a validity score for the finding based on whether the selected criterion is met. The processor determines whether the finding is false positive by comparing the validity score to a predetermined validity threshold. If the finding is true positive, a graphical user interface displays the finding.

Claims

1. A system for validating software security analysis findings, comprising:

one or more processors;

a graphical user interface (GUI); and

a memory in communication with the one or more processors and the GUI, the memory storing instructions that, when executed by the one or more processors, are configured to;

receive, from a first software security analysis tool, a first finding from a first scan of an application code, the first finding including one or more software issues within the application code;

retrieve a source truth dataset including a plurality of criteria for validating the first finding;

determine a first validity score for the first finding based on whether a first criterion is met, the first validity score indicating the accuracy of the first finding;

determine, by comparing the first validity score to a predetermined validity threshold, a value of the first finding; and

send, to the GUI, a first signal to cause the GUI to display the first finding and the value of the first finding.

2. The system of claim 1, wherein determining the first validity score further comprises:

identifying a first characteristic from the first finding; and

identifying, from amongst the plurality of criteria, the first criterion for validating the first characteristic.

3. The system of claim 2, wherein the value of the first finding is a false positive, and wherein the instructions, when executed by the one or more processors are further configured to:

dissociate the first criterion from the first characteristic in the source truth dataset;

generate a new criterion; and

associate the new criterion with the first characteristic in the source truth dataset.

4. The system of claim 3, wherein the instructions, when executed by the one or more processors are further configured to:

receive, from a second software security analysis tool, a second finding from a second scan of the application code, the second finding including one or more software issues within the application code;

identify a second characteristic from the second finding;

determine that the second characteristic is the same as the first characteristic;

identify the new criterion for validating the second characteristic from the plurality of criteria;

determine a second validity score for the second finding based on whether the new criterion is met, wherein the second validity score indicates the accuracy of the second finding;

determine, by comparing the second validity score to the predetermined validity threshold, a value of the second finding; and

send, to the GUI, a second signal to cause the GUI to display the second finding and the value of the second finding.

5. The system of claim 1, wherein the value of the first finding indicates a false positive, and wherein the instructions, when executed by the one or more processors, are further configured to:

receive, via the GUI, an input from a user regarding the validity of the first finding; and

update the source truth dataset based on the input.

6. The system of claim 1, wherein the instructions, when executed by the one or more processors, are further configured to:

identify a first set of one or more findings stored in the memory, each finding in the first set overlapping with the first finding;

determine a number of findings in the first set; and

wherein determining the value of the first finding further comprises;

comparing the number of findings in the first set with a predetermined limit; and

when the number of findings in the first set exceeds the predetermined limit;

determining that the first finding is a false positive;

or

when the number of findings in the first set does not exceed the predetermined limit;

determining that the first finding is a true positive.

7. The system of claim 1, wherein the instructions, when executed by the one or more processors, are further configured to:

retrieve, from the memory, a confidence score associated with the first software security analysis tool; and

wherein determining the value of the first finding is further based on the confidence score associated with the first software security analysis tool.

8. The system of claim 1, wherein the source truth dataset includes information received from a user regarding one or more previous findings.

9. A system for validating software security analysis findings, comprising:

one or more processors;

a graphical user interface (GUI); and

a memory in communication with the one or more processors and the GUI, the memory storing instructions that, when executed by the one or more processors, are configured to;

receive, from a first software security analysis tool, a first finding from a first scan of an application code, the first finding including one or more software issues within the application code;

retrieve, a source truth dataset, the source truth dataset including a plurality of criteria for validating one or more characteristics;

determine a first validity factor by determining whether a first criterion is met;

determine a second validity factor by retrieving a confidence score associated with the first software security analysis tool;

determine a first validity score for the first finding based on at least one of the first validity factor and the second validity factor, the first validity score indicating the accuracy of the first finding;

determine, by comparing the first validity score to a predetermined validity threshold, a value of the first finding; and

send, to the GUI, a first signal to cause the GUI to display the first finding and the value of the first finding.

10. The system of claim 9, wherein determining the first validity factor further comprises:

identifying a first characteristic from the first finding; and

identify, from amongst the plurality of criteria, the first criterion for validating the first characteristic.

11. The system of claim 10, wherein the value of the first finding is false positive, and wherein the instructions, when executed by the one or more processors are further configured to:

dissociate the first criterion from the first characteristic in the source truth dataset;

generate a new criterion; and

associate the new criterion with the first characteristic in the source truth dataset.

12. The system of claim 11, wherein the instructions, when executed by the one or more processors, are further configured to:

receive, from a second software security analysis tool, a second finding from a second scan of the application code, the second finding including one or more software issues within the application code;

identify a second characteristic from the second finding;

determine that the second characteristic is the same as the first characteristic;

identify the new criterion for validating the second characteristic from the plurality of criteria;

determine a third validity factor by determining whether the new criterion is met;

determine a fourth validity factor by retrieving, from the memory, a confidence score associated with the second software security analysis tool;

determine a second validity score for the second finding based on at least one of the third validity factor and the fourth validity factor, wherein the second validity score indicates the accuracy of the second finding;

determine, by comparing the second validity score to the predetermined validity threshold, a value of the second finding; and

send, to the GUI, a second signal to cause the GUI to display the second finding and the value of the second finding.

13. The system of claim 9, wherein the first software security analysis tool belongs to one of the following categories:

a first category of at least one software security analysis tool for performing Static Application Security Testing (SAST);

a second category of at least one software security analysis tool for performing Dynamic Application Security Testing (DAST);

a third category of at least one software security analysis tool for performing Open Source Analysis (OSA); and

a fourth category of at least one software security analysis tool for performing Interactive Application Security Testing (IAST).

14. The system of claim 13, wherein each category has a respective confidence score and a predetermined validity threshold.

15. The system of claim 14, wherein the confidence score for each category is different.

16. The system of claim 14, wherein the predetermined validity threshold associated with different categories are different.

17. The system of claim 9, wherein the instructions, when executed by the one or more processors, are further configured to:

identify a first set of one or more findings stored in the memory, each finding of the one or more findings overlapping with the first finding;

determine a number of findings in the first set; and

wherein determining the value of the first finding further comprises;

comparing the number of findings in the first set with a predetermined limit; and

when the number of findings in the first set exceeds the predetermined limit;

determining that the value of the first finding is false positive;

or

when the number of findings in the first set does not exceed the predetermined limit;

determining that the value of the first finding is true positive.

18. A method for validating software security analysis findings, comprising:

receiving, by a processor and from a first software security analysis tool, a first finding from a first scan of an application, the first finding including one or more software issues within the application code;

retrieving, from memory, a source truth dataset, the source truth dataset including a plurality of criteria for validating one or more characteristics;

determining, by the processor, a first validity factor by determining whether a first criterion is met;

determining, by the processor, a second validity factor by retrieving, from the memory, a confidence score associated with the first software security analysis tool;

identifying, by the processor, a first set of one or more findings stored in the memory, each finding of the one or more findings overlapping with the first finding;

determining, by the processor, a number of findings in the first set;

determining, by the processor, a third validity factor based on the number of findings in the first set;

determining, by the processor, a first validity score for the first finding based on at least one of the first validity factor, the second validity factor, or the third validity factor;

determining, by the processor, by comparing the first validity score to a predetermined validity threshold, a value of the first finding; and

sending, to a graphical user interface (GUI), a first signal to cause the GUI to display the first finding and the value of the first finding.

19. The method of claim 18, wherein the value of the first finding is false positive, the method further comprising:

receiving, via the GUI, an input from a user regarding the validity of the first finding; and

updating, by the processor, the source truth dataset based on the input.

20. The method of claim 18, wherein determining the first validity factor further comprises:

identifying, by the processor, a first characteristic from the first finding; and

identifying, by the processor and from amongst the plurality of criteria, the first criterion for validating the first characteristic.

21. The method of claim 20, wherein the value of the first finding is false positive, the method further comprising:

dissociating the first criterion from the first characteristic in the source truth dataset;

generating a new criterion; and

associating the new criterion with the first characteristic in the source truth dataset.

22. The method of claim 21, further comprising:

receiving, by the processor and from a second software security analysis tool, a second finding from a second scan of the application code, the second finding including one or more software issues within the application code;

identifying, by the processor, a second characteristic from the second finding;

determining, by the processor, that the second characteristic is the same as the first characteristic;

identifying, by the processor, the new criterion for validating the second characteristic from the plurality of criteria;

determining, by the processor, a third validity factor by determining whether the new criterion is met;

determining, by the processor, a fourth validity factor by retrieving, from the memory, a confidence score associated with the second software security analysis tool;

determining, by the processor, a second validity score for the second finding based on at least one of the third validity factor and the fourth validity factor, wherein the second validity score indicates the accuracy of the second finding;

determining, by comparing the second validity score to the predetermined validity threshold, a value of the second finding; and

sending, to the GUI, a second signal to cause the GUI to display the second finding and the value of the second finding.