Methods and systems for reducing false positive findings

• US 10,395,041 B1
• Filed: 10/31/2018
• Issued: 08/27/2019
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Youngberg, Adam, Filbey, David, Fernando, Kishore Prabakaran
• Primary Examiner(s): Kanaan, Simon P
• Application Number:
  US16/177,236
• Time in Patent Office: 300 Days
• CPC Class Codes: G06F 11/3608 using formal methods, e.g. model checking, abstract interpretation theorem proving G06N5/013
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 21/563 by source code analysis
• CPC Class Codes: G06F 21/577 Assessing vulnerabilities and evaluating computer system security
• CPC Class Codes: G06F 2221/033 Test or assess software

Abstract

A system for validating software security analysis findings includes a non-transitory computer readable medium and a processor. The non-transitory computer readable medium stores a source truth dataset including criteria for validating characteristics of findings. The processor receives a finding from a software security analysis tool that performs scan on application code. The processor identifies a characteristic from the finding. The processor selects a criterion from the non-transitory computer readable medium for validating the identified characteristic. The processor determines a validity score for the finding based on whether the selected criterion is met. The processor determines whether the finding is false positive by comparing the validity score to a predetermined validity threshold. If the finding is true positive, a graphical user interface displays the finding.

Claims

1. A system for validating software security analysis findings, comprising:

a non-transitory computer readable medium for storing;

a plurality of confidence scores designating confidence levels of a plurality of software security analysis tools of different categories;

findings generated by each software security analysis tool; and

a source truth dataset including criteria for validating characteristics of software security analysis findings; and

a processor configured to;

receive a first finding from a first software security analysis tool that performs a scan of application code;

identify a first characteristic from the first finding;

select, from the non-transitory computer readable medium, a criterion for validating the first characteristic;

determine a first validity factor by determining whether the selected criterion is met;

determine a second validity factor by retrieving, from the non-transitory computer readable medium, a confidence score associated with the first software security analysis tool;

determine a third validity factor by determining a number of findings stored in the non-transitory computer readable medium that overlap with the first finding;

determine a validity score for the first finding based on at least one of the first validity factor, the second validity factor and the third validity factor;

determine whether the first finding is false positive by comparing the validity score to a predetermined validity threshold; and

display the first finding on a graphical user interface when the first finding is true positive.

2. The system of claim 1, wherein the processor is configured to:

update the source truth dataset stored in the non-transitory computer readable medium based on whether the first finding is false positive.

3. The system of claim 1, wherein the processor is configured to exclude the first finding from being displayed on the graphical user interface when the first finding is false positive.

4. The system of claim 1, wherein the processor is configured to:

receive, via the graphical user interface, a disposition of the first finding entered by a user; and

update the source truth dataset stored in the non-transitory computer readable medium based on the disposition.

5. The system of claim 4, wherein the processor is configured to exclude the first finding from being displayed on the graphical user interface when the disposition indicates that the first finding is false positive.

6. The system of claim 1, wherein the source truth dataset includes information of disposition of findings entered by a user.

7. The system of claim 1, wherein the processor is configured to update a scan rule of the first software security analysis tool to omit scanning the first characteristic when a plurality of findings having the first characteristic are false positive.

8. The system of claim 1, wherein each software security analysis tool belongs to one of the following categories:

a first category of at least one software security analysis tool for performing Static Application Security Testing (SAST);

a second category of at least one software security analysis tool for performing Dynamic Application Security Testing (DAST);

a third category of at least one software security analysis tool for performing Open Source Analysis (OSA); and

a fourth category of at least one software security analysis tool for performing Interactive Application Security Testing (IAST).

9. The system of claim 8, wherein a confidence score associated with a software security analysis tool of the first category is lower than a confidence score associated with a software security analysis tool of the third category.

10. The system of claim 8, wherein a confidence score associated with a software security analysis tool of the second category is lower than a confidence score associated with a software security analysis tool of the third category.

11. The system of claim 8, wherein each category is associated with a predetermined validity threshold.

12. The system of claim 11, wherein the predetermined validity thresholds associated with different categories are different.

13. The system of claim 1, wherein the selected criterion includes a requirement of presence of a second finding with a second characteristic with respect to the application code.

14. The system of claim 13, wherein the processor determines the first validity factor by determining whether the second finding with the second characteristic is present in any finding generated by any software security analysis tool that performs a scan on the application code.

15. The system of claim 13, wherein the processor determines the first validity factor by determining whether the second finding with the second characteristic is present in any finding generated by the first software security analysis tool that performs the scan of the application code.

16. The system of claim 1, wherein the non-transitory computer readable medium stores characteristic confidence scores indicating confidence levels of each software security analysis tool to correctly identify each characteristic of findings.

17. The system of claim 16, wherein the processor is configured to determine the validity score for the first finding based on a characteristic confidence score that indicates a confidence level of the first software security analysis tool to correctly identify the first characteristic.

18. The system of claim 1, wherein the third validity factor indicates an increase of likelihood of validity when there exists a plurality of findings produced by other software security analysis tools that overlap with the first finding.

19. A method for validating software security analysis findings, comprising:

storing, by a non-transitory computer readable medium, a plurality of confidence scores designating confidence levels of a plurality of software security analysis tools of different categories;

storing, by the non-transitory computer readable medium, findings generated by each software security analysis tool; and

storing, by the non-transitory computer readable medium, a source truth dataset including criteria for validating characteristics of software security analysis findings;

receiving, by a processor, a first finding from a first software security analysis tool that performs a scan of application code;

identifying, by the processor, a first characteristic from the first finding;

selecting, by the processor, from the non-transitory computer readable medium, a criterion for validating the first characteristic;

determining, by the processor, a first validity factor by determining whether the selected criterion is met;

determining, by the processor, a second validity factor by retrieving, from the non-transitory computer readable medium, the confidence score associated with the first software security analysis tool;

determining, by the processor, a third validity factor by determining a number of findings stored in the non-transitory computer readable medium that overlap with the first finding;

determining, by the processor, a validity score for the first finding based on at least one of the first validity factor, the second validity factor or the third validity factor;

determining, by the processor, whether the first finding is false positive by comparing the validity score to a predetermined validity threshold; and

displaying, by the processor, the first finding on a graphical user interface when the first finding is true positive.

20. The method of claim 19, further comprising:

updating the source truth dataset stored in the non-transitory computer readable medium based on whether the first finding is false positive.

21. The method of claim 19, further comprising:

receiving, via the graphical user interface, a disposition of the first finding entered by a user;

updating the source truth dataset stored in the non-transitory computer readable medium based on the disposition, and

excluding the first finding from being displayed on the graphical user interface when the disposition indicates that the first finding is false positive.

22. The method of claim 19, wherein each software security analysis tool belongs to one of the following categories:

a first category of at least one software security analysis tool for performing Static Application Security Testing (SAST);

a second category of at least one software security analysis tool for performing Dynamic Application Security Testing (DAST);

a third category of at least one software security analysis tool for performing Open Source Analysis (OSA); and

a fourth category of at least one software security analysis tool for performing Interactive Application Security Testing (IAST).

23. The method of claim 19, wherein the third validity factor indicates an increase of likelihood of validity when there exists a plurality of findings produced by other software security analysis tools that overlap with the first finding.