Automatic baselining of anomalous event activity in time series data

• US 9,471,778 B1
• Filed: 11/30/2015
• Issued: 10/18/2016
• Est. Priority Date: 11/30/2015
• Status: Active Grant

• Current Assignee: HCL Technologies Limited
• Original Assignee: International Business Machines Corporation
• Inventors: Seo, Hyun Kyu, Zenz, Gideon, Williams, Ronald B.
• Primary Examiner(s): Pyzocha, Michael
• Application Number:
  US14/953,742
• Time in Patent Office: 323 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/34 Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2221/034 Test or assess a computer or a system
• CPC Class Codes: H04L 47/29 using a combination of thresholds
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002

Abstract

Software that automatically creates baselines from time series data of computer system activity, thereby providing immediate value from observed system data. The software performs the following operations: (i) receiving values of one or more attributes of a computing system that correspond to one or more time periods; (ii) determining a first set of statistical thresholds for the received values, wherein the received values include a subset of values that exceed the first set of statistical thresholds; (iii) determining a second set of statistical thresholds for the subset of values that exceed the first set of statistical thresholds; and (iv) determining a baseline pattern for the one or more attributes based, at least in part, on the determined first set of statistical thresholds and the determined second set of statistical thresholds.

Claims

1. A computer-implemented method comprising:

receiving values of one or more attributes of a computing system, wherein the values of the one or more attributes correspond to one or more time periods;

determining a first set of statistical thresholds for the received values of the one or more attributes, wherein the received values of the one or more attributes include one or more values that exceed the first set of statistical thresholds for the received values of the one or more attributes;

determining a second set of statistical thresholds for a first subset of values of the received values of the one or more attributes, wherein each value of the first subset exceeds the first set of statistical thresholds for the received values of the one or more attributes;

determining a baseline pattern for the one or more attributes based, at least in part, on the determined first set of statistical thresholds for the received values of the one or more attributes and the determined second set of statistical thresholds for the first subset of values that exceed the first set of statistical thresholds for the received values of the one or more attributes;

utilizing an anti-gaming mechanism for preventing undetected malicious activity on the computing system, wherein the anti-gaming mechanism randomly determines a start time of one or more additional time periods to prevent potential attackers of the computing system from utilizing knowledge of the first set of statistical thresholds, the second set of statistical thresholds, and/or the baseline pattern to avoid detection of malicious activity;

receiving additional values of the one or more attributes of the computing system, wherein the additional values of the one or more attributes correspond to the one or more additional time periods; and

in response to identifying anomalous values in the received additional values based on the determined baseline pattern, sending an alert to a user of the computing system that a potential intrusion in the computing system has occurred.

2. The method of claim 1, further comprising modifying the determined baseline pattern based on the received additional values of the one or more attributes.

3. The method of claim 1, further comprising:

determining a burst period representing a minimum amount of time for which values must exceed the first set of statistical thresholds to be considered statistically significant; and

prior to determining the second set of statistical thresholds for the first subset of values that exceed the first set of statistical thresholds, filtering the first subset of values that exceed the first set of statistical thresholds to remove a second subset of the first subset of values, wherein the values of the second subset correspond to time periods that are shorter than the determined burst period.

4. The method of claim 1, wherein identifying anomalous values is further based on a determination that at least one of the one or more additional time periods is similar to at least one of the one or more time periods.

5. The method of claim 4, wherein the at least one of the one or more additional time periods that is determined to be similar to the at least one of the one or more time periods corresponds to the same time of day as the similar at least one of the one or more time periods, but on a different day.

6. The method of claim 4, further comprising optimizing runtime storage by making the received values of the one or more attributes available for deletion from runtime storage after the determining of the baseline pattern.

7. The method of claim 4, further comprising optimizing runtime storage by making a first set of portions of the determined baseline pattern available for deletion from runtime storage, wherein the portions of the first set correspond to time periods other than the at least one of the one or more additional time periods determined to be similar to the at least one of the one or more time periods.

8. A computer program product comprising a computer readable storage medium having stored thereon:

program instructions programmed to receive values of one or more attributes of a computing system, wherein the values of the one or more attributes correspond to one or more time periods;

program instructions programmed to determine a first set of statistical thresholds for the received values of the one or more attributes, wherein the received values of the one or more attributes include one or more values that exceed the first set of statistical thresholds for the received values of the one or more attributes;

program instructions programmed to determine a second set of statistical thresholds for a first subset of values of the received values of the one or more attributes, wherein each value of the first subset exceeds the first set of statistical thresholds for the received values of the one or more attributes;

program instructions programmed to determine a baseline pattern for the one or more attributes based, at least in part, on the determined first set of statistical thresholds for the received values of the one or more attributes and the determined second set of statistical thresholds for the first subset of values that exceed the first set of statistical thresholds for the received values of the one or more attributes;

program instructions programmed to utilize an anti-gaming mechanism for preventing undetected malicious activity on the computing system, wherein the anti-gaming mechanism randomly determines a start time of one or more additional time periods to prevent potential attackers of the computing system from utilizing knowledge of the first set of statistical thresholds, the second set of statistical thresholds, and/or the baseline pattern to avoid detection of malicious activity;

program instructions programmed to receive additional values of the one or more attributes of the computing system, wherein the additional values of the one or more attributes correspond to the one or more additional time periods; and

program instructions programmed to, in response to identifying anomalous values in the received additional values based on the determined baseline pattern, send an alert to a user of the computing system that a potential intrusion in the computing system has occurred.

9. The computer program product of claim 8, further comprising program instructions programmed to modify the determined baseline pattern based on the received additional values of the one or more attributes.

10. The computer program product of claim 8, further comprising:

program instructions programmed to determine a burst period representing a minimum amount of time for which values must exceed the first set of statistical thresholds to be considered statistically significant; and

program instructions programmed to, prior to determining the second set of statistical thresholds for the first subset of values that exceed the first set of statistical thresholds, filter the first subset of values that exceed the first set of statistical thresholds to remove a second subset of the first subset of values, wherein the values of the second subset correspond to time periods that are shorter than the determined burst period.

11. The computer program product of claim 8, wherein identifying anomalous values is further based on a determination that at least one of the one or more additional time periods is similar to at least one of the one or more time periods.

12. The computer program product of claim 11, further comprising program instructions programmed to optimize runtime storage by making a first set of portions of the determined baseline pattern available for deletion from runtime storage, wherein the portions of the first set correspond to time periods other than the at least one of the one or more additional time periods determined to be similar to the at least one of the one or more time periods.

13. A computer system comprising:

a processor(s) set; and

a computer readable storage medium;

wherein;

the processor set is structured, located, connected and/or programmed to run program instructions stored on the computer readable storage medium; and

the program instructions include;

program instructions programmed to receive values of one or more attributes of a computing system, wherein the values of the one or more attributes correspond to one or more time periods;

program instructions programmed to determine a first set of statistical thresholds for the received values of the one or more attributes, wherein the received values of the one or more attributes include one or more values that exceed the first set of statistical thresholds for the received values of the one or more attributes;

program instructions programmed to determine a second set of statistical thresholds for a first subset of values of the received values of the one or more attributes, wherein each value of the first subset exceeds the first set of statistical thresholds for the received values of the one or more attributes;

program instructions programmed to determine a baseline pattern for the one or more attributes based, at least in part, on the determined first set of statistical thresholds for the received values of the one or more attributes and the determined second set of statistical thresholds for the first subset of values that exceed the first set of statistical thresholds for the received values of the one or more attributes;

program instructions programmed to utilize an anti-gaming mechanism for preventing undetected malicious activity on the computing system, wherein the anti-gaming mechanism randomly determines a start time of one or more additional time periods to prevent potential attackers of the computing system from utilizing knowledge of the first set of statistical thresholds, the second set of statistical thresholds, and/or the baseline pattern to avoid detection of malicious activity;

program instructions programmed to receive additional values of the one or more attributes of the computing system, wherein the additional values of the one or more attributes correspond to the one or more additional time periods; and

program instructions programmed to, in response to identifying anomalous values in the received additional values based on the determined baseline pattern, send an alert to a user of the computing system that a potential intrusion in the computing system has occurred.

14. The computer system of claim 13, wherein the program instructions further include program instructions programmed to modify the determined baseline pattern based on the received additional values of the one or more attributes.

15. The computer system of claim 13, wherein the program instructions further include:

program instructions programmed to determine a burst period representing a minimum amount of time for which values must exceed the first set of statistical thresholds to be considered statistically significant; and

program instructions programmed to, prior to determining the second set of statistical thresholds for the first subset of values that exceed the first set of statistical thresholds, filter the first subset of values that exceed the first set of statistical thresholds to remove a second subset of the first subset of values, wherein the values of the second subset correspond to time periods that are shorter than the determined burst period.

16. The computer system of claim 13, wherein identifying anomalous values is further based on a determination that at least one of the one or more additional time periods is similar to at least one of the one or more time periods.

17. The computer system of claim 16, further comprising program instructions programmed to optimize runtime storage by making a first set of portions of the determined baseline pattern available for deletion from runtime storage, wherein the first set of portions of the determined baseline pattern correspond to time periods other than the at least one of the one or more additional time periods determined to be similar to the at least one of the one or more time periods.