Computer system to identify anomalies based on computer-generated results

• US 10,084,805 B2
• Filed: 02/15/2018
• Issued: 09/25/2018
• Est. Priority Date: 02/20/2017
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Nadolski, William Robert, Chapman-McQuiston, Emily Louise, King, Julius Alton, Nino, Mauricio Alvarez
• Primary Examiner(s): Patel, Haresh N
• Application Number:
  US15/897,263
• Publication Number:
  US 20180241764A1
• Time in Patent Office: 222 Days
• Field of Search: 726 23
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/043 based on fuzzy logic, fuzzy membership or fuzzy inference, e.g. adaptive neuro-fuzzy inference systems [ANFIS]
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

One or more embodiments may include techniques to identify anomalies based on computer-generated results. Moreover, embodiments may include applying scenario rules to data to detect scenario violations and grouping the scenario violations into scenario clusters based on similar behavior performed by entities indicated by similarity metrics. embodiments include determining predictive ability values for each of the scenario clusters, ranking the scenario clusters based on the predictive ability values, and removing scenario clusters having predictive ability values below a threshold. In embodiments combinations of scenario clusters may be generated from the set of scenario clusters and the combinations of scenario clusters may be evaluated for effectiveness. Embodiments include generating scores for entities of the combinations of scenario clusters deemed effective, and provide results indicating whether one or more of the entities committed an anomaly based on the scores for each of the entities.

Claims

1. An apparatus, comprising:

processing circuitry; and

memory to store instructions that, when executed by the processing circuitry, cause the processing circuitry to;

obtain scenario rules and data representing actions performed by entities;

apply the scenario rules to a subset of the data to detect scenario violations based on the actions performed by the entities, the subset of the data associated with the entities of a particular entity type;

group scenario violations into scenario clusters, each scenario cluster comprising one or more scenario violations associated with similar behavior performed by the entities indicated by similarity metrics, and each of the scenario clusters is one of a set of scenario clusters;

determine predictive ability values for each of the scenario clusters, the predictive ability values to indicate relative significance between each of the scenario clusters to predict a target behavior;

rank the scenario clusters based on the predictive ability values and remove scenario clusters from the set of scenario clusters having predictive ability values below a predictive threshold;

generate combinations of scenario clusters from the set of scenario clusters, each of the combinations of scenario clusters including two or more scenario clusters;

determine an effectiveness factor for each of the combinations of scenario clusters, each of the effectiveness factors based on a number of entities committing the targeted behavior as a percentage of all the entities that committed at least one scenario violation for a particular combination of scenario clusters of the combinations of scenario clusters;

generate scores for each of the entities of the particular entity type using the combinations of scenario clusters having the effectiveness factor at or above an effectiveness threshold; and

provide results to a system to enable presentation on a display device, the results indicating one or more of the entities that committed the targeted behavior based on the scores for each of the entities.

2. The apparatus of claim 1, wherein the similarity metrics to indicate correlation distances for the scenario violations, the scenario violations having a similarity metric below an eigen value threshold are grouped into a same scenario cluster.

3. The apparatus of claim 1, the processing circuitry to:

obtain the scenario rules and the data from one or more data system coupled via one or more network connections; and

apply the scenario rules to the actions by comparing each indication of an action to each scenario rule to determine whether an entity committed a scenario violation, wherein the scenario rules define activity to detect the targeted behavior.

4. The apparatus of claim 3, the processing circuitry to generate indicators for entities committing scenario violations based on applying the scenario rules, each indicator to indicate an entity committed a scenario violation.

5. The apparatus of claim 4, the processing circuitry to:

pass each indicator for each of the scenario clusters through a statistical model to determine the predictive ability values for the scenario clusters, each indicator to indicate whether a scenario cluster is triggered to predict the targeted behavior,

rank each of the scenario clusters based on the predictive ability values by relative significance, and

remove scenario clusters having predictive ability values below the predictive threshold.

6. The apparatus of claim 1, the processing circuitry to exhaustively generate the combinations of scenario clusters, each of the combinations comprising one or more scenario clusters.

7. The apparatus of claim 1, the processing circuitry to determine one or more combinations of scenario clusters having overlapping scenario clusters, and to discard combinations of scenario clusters that are entirely overlapped by two or more other scenario clusters or entirely overlapped by another combination of scenario cluster.

8. The apparatus of claim 1, wherein the effectiveness threshold is a minimum number of scenario clusters required for a combination of scenario clusters, and the processing circuitry to determine a number of scenario clusters in each of the combinations of scenario clusters, and discard each of the combinations of scenario clusters having the number of scenario clusters below the minimum number of scenario clusters.

9. The apparatus of claim 1, wherein the effectiveness threshold is a minimum number of violating entities required for a combination of scenario clusters, and the processing circuitry to determine a number of violating entities for each of the combination of scenario clusters, and discard each of the combinations of scenario clusters having the number of violating entities below the minimum number of violating entities.

10. The apparatus of claim 1, the processing circuitry to iteratively generate scores for every entity of every entity type, during each iteration the processing circuitry to:

determine another subset of the data associated with entities of another entity type;

apply the scenario rules to the other subset of the data to determine new scenario violations for the entities of the other entity type;

generate new scenario clusters by grouping the new scenario violations based on similarity metrics;

rank the new scenario clusters based on predictive ability values for the new scenario clusters and remove scenario clusters from a set of new scenario clusters having predictive ability values below the predictive threshold;

generate new combinations of scenario clusters, each new combination of scenario clusters to include one or more new scenario clusters;

determine effectiveness factors the new combinations of scenario clusters, each of the effectiveness factors based on a number of entities of the other entity type committing the targeted behavior as a percentage of all the entities of the other entity type that committed at least one scenario violation for a particular combination of scenario clusters of the new combinations of scenario clusters;

generate scores for each of the entities of the other entity type using the new combinations of scenario clusters having the effectiveness factor at or above an effectiveness threshold; and

perform another iteration until scores are generated for every entity of every entity type.

11. A computer-implemented, comprising:

obtaining scenario rules and data representing actions performed by entities;

applying the scenario rules to a subset of the data to detect scenario violations based on the actions performed by the entities, the subset of the data associated with the entities of a particular entity type;

grouping scenario violations into scenario clusters, each scenario cluster comprising one or more scenario violations associated with similar behavior performed by the entities indicated by similarity metrics, and each of the scenario clusters is one of a set of scenario clusters;

determining predictive ability values for each of the scenario clusters, the predictive ability values to indicate relative significance between each of the scenario clusters to predict a target behavior;

ranking the scenario clusters based on the predictive ability values and removing scenario clusters from the set of scenario clusters having predictive ability values below a predictive threshold;

generating combinations of scenario clusters from the set of scenario clusters, each of the combinations of scenario clusters including two or more scenario clusters;

determining an effectiveness factor for each of the combinations of scenario clusters, each of the effectiveness factors based on a number of entities committing the targeted behavior as a percentage of all the entities that committed at least one scenario violation for a particular combination of scenario clusters of the combinations of scenario clusters;

generating scores for each of the entities of the particular entity type using the combinations of scenario clusters having the effectiveness factor at or above an effectiveness threshold; and

providing results to a system to enable presentation on a display device, the results indicating one or more of the entities that committed the targeted behavior based on the scores for each of the entities.

12. The computer-implemented method of claim 11, wherein the similarity metrics to indicate correlation distances for the scenario violations, the scenario violations having a similarity metric below an eigen value threshold are grouped into a same scenario cluster.

13. The computer-implemented method of claim 11, comprising:

obtaining the scenario rules and the data from one or more data system coupled via one or more network connections; and

applying the scenario rules to the actions by comparing each indication of an action to each scenario rule to determine whether an entity committed a scenario violation, wherein the scenario rules define activity to detect the targeted behavior.

14. The computer-implemented method of claim 13, comprising generating indicators for entities committing scenario violations based on applying the scenario rules, each indicator to indicate an entity committed a scenario violation.

15. The computer-implemented method of claim 14, comprising:

passing each indicator for each of the scenario clusters through a statistical model to determine the predictive ability values for the scenario clusters, each indicator to indicate whether a scenario cluster is triggered to predict the targeted behavior,

ranking each of the scenario clusters based on the predictive ability values by relative significance, and

removing scenario clusters having predictive ability values below the predictive threshold.

16. The computer-implemented method of claim 11, comprising exhaustively generating the combinations of scenario clusters, each of the combinations comprising one or more scenario clusters.

17. The computer-implemented method of claim 11, comprising determining one or more combinations of scenario clusters having overlapping scenario clusters, and to discard combinations of scenario clusters that are entirely overlapped by two or more other scenario clusters or entirely overlapped by another combination of scenario cluster.

18. The computer-implemented method of claim 11, comprising determining a number of scenario clusters in each of the combinations of scenario clusters, and discarding each of the combinations of scenario clusters having the number of scenario clusters below a minimum number of scenario clusters, wherein the effectiveness threshold is the minimum number of scenario clusters required for a combination of scenario clusters.

19. The computer-implemented method of claim 11, comprising determining a number of violating entities for each of the combination of scenario clusters, and discarding each of the combinations of scenario clusters having the number of violating entities below a minimum number of violating entities, wherein the effectiveness threshold is the minimum number of violating entities required for a combination of scenario clusters.

20. The computer-implemented method of claim 11, comprising iteratively generating scores for every entity of every entity type, during each iteration:

determining another subset of the data associated with entities of another entity type;

applying the scenario rules to the other subset of the data to determine new scenario violations for the entities of the other entity type;

generating new scenario clusters by grouping the new scenario violations based on similarity metrics;

ranking the new scenario clusters based on predictive ability values for the new scenario clusters and remove scenario clusters from a set of new scenario clusters having predictive ability values below the predictive threshold;

generating new combinations of scenario clusters, each new combination of scenario clusters to include one or more new scenario clusters;

determining effectiveness factors the new combinations of scenario clusters, each of the effectiveness factors based on a number of entities of the other entity type committing the targeted behavior as a percentage of all the entities of the other entity type that committed at least one scenario violation for a particular combination of scenario clusters of the new combinations of scenario clusters;

generating scores for each of the entities of the other entity type using the new combinations of scenario clusters having the effectiveness factor at or above an effectiveness threshold; and

performing another iteration until scores are generated for every entity of every entity type.

21. At least one non-transitory computer-readable storage medium comprising instructions that when executed cause processing circuitry to:

obtain scenario rules and data representing actions performed by entities;

apply the scenario rules to a subset of the data to detect scenario violations based on the actions performed by the entities, the subset of the data associated with the entities of a particular entity type;

group scenario violations into scenario clusters, each scenario cluster comprising one or more scenario violations associated with similar behavior performed by the entities indicated by similarity metrics, and each of the scenario clusters is one of a set of scenario clusters;

determine predictive ability values for each of the scenario clusters, the predictive ability values to indicate relative significance between each of the scenario clusters to predict a target behavior;

rank the scenario clusters based on the predictive ability values and removing scenario clusters from the set of scenario clusters having predictive ability values below a predictive threshold;

generate combinations of scenario clusters from the set of scenario clusters, each of the combinations of scenario clusters including two or more scenario clusters;

determine an effectiveness factor for each of the combinations of scenario clusters, each of the effectiveness factors based on a number of entities committing the targeted behavior as a percentage of all the entities that committed at least one scenario violation for a particular combination of scenario clusters of the combinations of scenario clusters;

generate scores for each of the entities of the particular entity type using the combinations of scenario clusters having the effectiveness factor at or above an effectiveness threshold; and

provide results to a system to enable presentation on a display device, the results indicating one or more of the entities that committed the targeted behavior based on the scores for each of the entities.

22. The non-transitory computer-readable storage medium of claim 21, wherein the similarity metrics to indicate correlation distances for the scenario violations, the scenario violations having a similarity metric below an eigen value threshold are grouped into a same scenario cluster.

23. The non-transitory computer-readable storage medium of claim 21, comprising instructions that when executed cause the processing circuitry to:

obtain the scenario rules and the data from one or more data system coupled via one or more network connections; and

apply the scenario rules to the actions by comparing each indication of an action to each scenario rule to determine whether an entity committed a scenario violation, wherein the scenario rules define activity to detect the targeted behavior.

24. The non-transitory computer-readable storage medium of claim 23, comprising instructions that when executed cause the processing circuitry to generate indicators for entities committing scenario violations based on applying the scenario rules, each indicator to indicate an entity committed a scenario violation.

25. The non-transitory computer-readable storage medium of claim 24, comprising instructions that when executed cause the processing circuitry to:

pass each indicator for each of the scenario clusters through a statistical model to determine the predictive ability values for the scenario clusters, each indicator to indicate whether a scenario cluster is triggered to predict the targeted behavior,

rank each of the scenario clusters based on the predictive ability values by relative significance, and

remove scenario clusters having predictive ability values below the predictive threshold.

26. The non-transitory computer-readable storage medium of claim 21, comprising instructions that when executed cause the processing circuitry to exhaustively generate the combinations of scenario clusters, each of the combinations comprising one or more scenario clusters.

27. The non-transitory computer-readable storage medium of claim 21, comprising instructions that when executed cause the processing circuitry to determine one or more combinations of scenario clusters having overlapping scenario clusters, and to discard combinations of scenario clusters that are entirely overlapped by two or more other scenario clusters or entirely overlapped by another combination of scenario cluster.

28. The non-transitory computer-readable storage medium of claim 21, comprising instructions that when executed cause the processing circuitry to determine a number of scenario clusters in each of the combinations of scenario clusters, and discarding each of the combinations of scenario clusters having the number of scenario clusters below a minimum number of scenario clusters, wherein the effectiveness threshold is the minimum number of scenario clusters required for a combination of scenario clusters.

29. The non-transitory computer-readable storage medium of claim 21, comprising instructions that when executed cause the processing circuitry to determine a number of violating entities for each of the combination of scenario clusters, and discarding each of the combinations of scenario clusters having the number of violating entities below a minimum number of violating entities, wherein the effectiveness threshold is the minimum number of violating entities required for a combination of scenario clusters.

30. The non-transitory computer-readable storage medium of claim 21, comprising instructions that when executed cause the processing circuitry to iteratively generate scores for every entity of every entity type, during each iteration the processing circuitry to:

determine another subset of the data associated with entities of another entity type;

apply the scenario rules to the other subset of the data to determine new scenario violations for the entities of the other entity type;

generate new scenario clusters by grouping the new scenario violations based on similarity metrics;

rank the new scenario clusters based on predictive ability values for the new scenario clusters and remove scenario clusters from a set of new scenario clusters having predictive ability values below the predictive threshold;

generate new combinations of scenario clusters, each new combination of scenario clusters to include one or more new scenario clusters;

determine effectiveness factors the new combinations of scenario clusters, each of the effectiveness factors based on a number of entities of the other entity type committing the targeted behavior as a percentage of all the entities of the other entity type that committed at least one scenario violation for a particular combination of scenario clusters of the new combinations of scenario clusters;

generate scores for each of the entities of the other entity type using the new combinations of scenario clusters having the effectiveness factor at or above an effectiveness threshold; and

perform another iteration until scores are generated for every entity of every entity type.