Parameter adjustment for pattern discovery

• US 10,027,686 B2
• Filed: 05/30/2012
• Issued: 07/17/2018
• Est. Priority Date: 05/30/2012
• Status: Active Grant

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: entIT Software LLC (Open Text Corporation)
• Inventors: Zhao, Zhipeng, Wang, Yanlin, Singla, Anurag
• Primary Examiner(s): Gracia, Gary S
• Application Number:
  US14/398,017
• Publication Number:
  US 20150106922A1
• Time in Patent Office: 2,239 Days
• Field of Search: 726 22
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00

Abstract

Pattern discovery performed on event data may include selecting an initial set of parameters for the pattern discovery. The parameters may specify conditions for identifying a pattern in the event data. A pattern discovery run is executed on the event data based on the initial set of parameters, and a parameter may be adjusted based on the output of the pattern discovery run.

Claims

1. A method comprising:

receiving event data collected by agents from sources over a communication network, the sources comprising network security devices;

providing a set of parameters to a pattern identifier engine, wherein the set of parameters specify conditions for identifying patterns in the event data;

executing, by the pattern identifier engine executed on a hardware processor of a manager system, a pattern discovery comprising identifying, by the pattern identifier engine, the patterns in the event data if the event data satisfies the conditions specified by the set of parameters;

determining whether the pattern discovery failed to complete within a predetermined period of time;

in response to determining that the pattern discovery failed to complete within the predetermined period of time, iteratively performing further pattern discovery until a criterion is satisfied by;

adjusting a parameter of the set of parameters to reduce use of system resources of the manager system for a subsequent pattern discovery run, the adjusting producing a respective adjusted set of parameters;

providing the respective adjusted set of parameters to the pattern identifier engine and executing, by the pattern identifier engine, the subsequent pattern discovery run to identify patterns in the event data if the event data satisfies conditions specified by the respective adjusted set of parameters; and

executing an action in response to the identified patterns produced by the further pattern discovery, the action comprising one or more of mitigating an attack and displaying the event data for analysis by a network administrator.

2. The method of claim 1, wherein the set of parameters comprises at least one of:

a pattern length parameter that identifies a minimum number of different activities that are performed in a sequence for the activities to be considered a pattern, a repeatability parameter that identifies a minimum number of times the different activities are repeated to be considered a pattern, or a time duration parameter that identifies a time duration of the event data that is considered for the pattern discovery.

3. The method of claim 2, comprising:

determining whether a number of the patterns identified from the event data as a result of executing the pattern discovery is less than a threshold; and

in response to determining that the number of the patterns identified from the event data is less than the threshold, adjusting the parameter to increase the number of the patterns identified from the event data.

4. The method of claim 3, wherein adjusting the parameter comprises at least one of:

increasing the pattern length parameter, increasing the repeatability parameter, or decreasing the time duration parameter.

5. The method of claim 2, comprising:

determining whether a number of the patterns identified from the event data as a result of executing the pattern discovery is greater than a threshold; and

in response to determining that the number of the patterns identified from the event data is greater than the threshold, adjusting the parameter to decrease the number of the patterns identified from the event data.

6. The method of claim 5, wherein adjusting the parameter comprises at least one of:

decreasing the pattern length parameter, decreasing the repeatability parameter, or increasing the time duration parameter.

7. The method of claim 1, comprising:

selecting a set of fields based on field statistics;

including the set of fields and the set of parameters in a pattern discovery profile; and

executing the pattern discovery based on the pattern discovery profile.

8. The method of claim 7, wherein executing the pattern discovery comprises:

determining whether the event data includes the set of fields that satisfy the conditions specified by the set of parameters to identify the patterns in the event data matching the pattern discovery profile.

9. The method of claim 1, wherein an output of the pattern discovery comprises a result set of the patterns satisfying the conditions that are specified by the set of parameters, wherein the conditions are associated with a set of fields for the event data.

10. The method of claim 1, further comprising:

identifying presence of a malicious program based on the patterns in the event data identified by the further pattern discovery.

11. A network security event processing system comprising:

a hardware processor; and

a non-transitory storage medium storing instructions executable on the hardware processor to;

receive event data collected by agents from sources over a communication network, the sources comprising network security devices;

execute, by a pattern identifier engine, a pattern discovery run to detect patterns in the event data based on a set of parameters that specify conditions for identifying the patterns, wherein the patterns are detected if the event data satisfies the conditions specified by the set of parameters, the event data describing activities for devices connected to the communication network;

determine whether the pattern discovery run failed to complete within a predetermined period of time;

in response to determining that the pattern discovery run failed to complete within the predetermined period of time, iteratively perform further pattern discovery until a criterion is satisfied by;

adjusting a parameter from the set of parameters to reduce use of system resources of the network security event processing system for a subsequent pattern discovery run, the adjusting producing an adjusted set of parameters;

providing the adjusted set of parameters to the pattern identifier engine and execute, by the pattern identifier engine, the subsequent pattern discovery run to identify patterns in the event data if the event data satisfies conditions specified by the adjusted set of parameters; and

execute an action in response to the identified patterns produced by the further pattern discovery, the action comprising one or more of mitigating an attack and displaying the event data for analysis by a network administrator.

12. The network security event processing system of claim 11, wherein the instructions are executable on the hardware processor to:

determine whether a number of the patterns identified from the event data as a result of executing the pattern discovery run is less than a threshold; and

in response to determining that the number of the patterns identified from the event data is less than the threshold, adjust the parameter to increase the number of the patterns identified from the event data.

13. The network security event processing system of claim 11, wherein the instructions are executable on the hardware processor to:

determine whether a number of the patterns identified from the event data as a result of executing the pattern discovery run is greater than a threshold; and

in response to determining that the number of the patterns identified from the event data is greater than the threshold, adjust the parameter to decrease the number of the patterns identified from the event data.

14. The network security event processing system of claim 11, wherein the instructions are executable on the hardware processor to:

identify presence of a virus or worm based on the patterns in the event data identified by the further pattern discovery.

15. A non-transitory computer readable medium including machine readable instructions that when executed cause a manager system to:

receive event data collected by agents from sources over a communication network, the sources comprising network security devices;

provide a set of parameters to a pattern identifier engine, wherein the set of parameters specify conditions for identifying patterns in the event data;

execute, by the pattern identifier engine, a pattern discovery comprising identifying the patterns in the event data if the event data satisfies the conditions specified by the set of parameters;

determine whether the pattern discovery failed to complete within a predetermined period of time;

in response to determining that the pattern discovery failed to complete within the predetermined period of time, iteratively perform further pattern discovery until a criterion is satisfied by;

adjusting a parameter of the initial set of parameters to reduce use of system resources of the manager system for a subsequent pattern discovery run, the adjusting producing an adjusted set of parameters; and

providing the adjusted set of parameters to the pattern identifier engine and execute, by the pattern identifier engine, the subsequent pattern discovery run to identify patterns in the event data if the event data satisfies conditions specified by the adjusted set of parameters; and

execute an action in response to the identified patterns produced by the further pattern discovery, the action comprising one or more of mitigating an attack and displaying the event data for analysis by a network administrator.

16. The non-transitory computer readable medium of claim 15, wherein the instructions when executed cause the manager system to:

determine whether a number of the patterns identified from the event data as result of executing the pattern discovery is less than a threshold; and

in response to determining that the number of the patterns identified from the event data is less than the threshold, adjust the parameter to increase the number of the patterns identified from the event data.

17. The non-transitory computer readable medium of claim 15, wherein the instructions when executed cause the manager system to:

determine whether a number of the patterns identified from the event data as result of executing the pattern discovery is greater than a threshold; and

in response to determining that the number of the patterns identified from the event data is greater than the threshold, adjust the parameter to decrease the number of the patterns identified from the event data.

18. The non-transitory computer readable medium of claim 15, wherein the instructions when executed cause the system to:

identify presence of a virus or worm based on the patterns in the event data identified by the further pattern discovery.