STATISTICAL METHOD AND SYSTEM FOR NETWORK ANOMALY DETECTION

• US 20080250497A1
• Filed: 03/31/2008
• Published: 10/09/2008
• Est. Priority Date: 03/30/2007
• Status: Active Grant

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: Netqos Incorporated (Broadcom, Inc.)
• Inventors: Mullarkey, Peter, Johns, Michael C.
• Granted Patent: US 8,601,575 B2
• Time in Patent Office: Days
• US Class Current: 726/22
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/147 for predicting network behaviour
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection

Abstract

An anomaly detection method and system determine network status by monitoring network activity. A statistics based profile for said network over a period is generated to analyze potentially anomalous network activity to determine if said network activity is anomalous by comparing current activity against the profile. Using the profile as a reference, the anomaly detection system and process estimate and prioritize potentially anomalous network activity based on the probability that the behavior is anomalous. The level of severity that the anomaly detection process uses to determine if an alarm is needed is based on comparing user-adjustable thresholds to the current probability. If the threshold has been breached, the user is alerted, subject to other quality checks. After a reporting cycle concludes, the anomaly detection system and process recompiles the statistics based profile to take into account the information observed in the previous reporting cycle.

Claims

1. A method for determining an Internet protocol (IP) network status comprising the steps of:

monitoring an IP communications network, said IP communications network comprising a plurality of associated computer systems, said monitoring step further comprising the steps of;

evaluating logged data communicated on said IP communications network as said logged data is logged; and

detecting at least one data communications event from said logged data to be a potentially anomalous event by nominating said at least one data communications event and performing a time series generation associated with the communication of said at least one data communications event on said IP communications network;

discovering said potentially anomalous data event to be an anomalous event by forming a percentiled data set from said logged data and comparing said at least one data communications event to a threshold level associated with said percentiled data set; and

generating an alert signal in association with said monitoring step if said anomalous event differs from said percentiled data set by a level at least equal to said threshold level.

2. The method of claim 1, further comprising the step of establishing a level of severity associated with the level by which said anomalous event varies from said threshold level.

3. The method of claim 1, further comprising the step of estimating and prioritizing said potentially anomalous network activity based on the probability of said potentially anomalous event comprises an anomalous event.

4. The method of claim 1, further comprising the step of transmitting said alert signal to at least one user of said IP communications network for alerting said at least one user to said potentially anomalous activity.

5. The method of claim 1, further comprising the step of updating a set of previously generated statistics profile of said logged data for relating to said anomalous event.

6. The method of claim 1 where said discovery step further comprises the step of comparing said at least one data communications event to a predicted network event for generating a profile of observed IP communications network activity.

7. The method claim 6 for estimating percentage level of said potentially anomalous network activity comprising the steps of:

recording a predefined number of memory locations on a communications system for recording anomalous network status in said profile;

dividing the number of memory locations into discreet groups;

defining these discreet groups as percentage levels; and

inserting said comparison in appropriate memory locations for a predetermined amount of time.

8. The method claim 7 where said statistic based profile estimates said predicted network activity based as a function of weighted averages.

9. The method claim 7 where said statistic based profile estimates said predicted network activity based as a function of Bayesian algorithms.

10. The method claim 1 comprising the additional step of coordinating with other network monitoring methods to analyze the anomaly upon anomaly detection.

11. A system for determining an Internet protocol (IP) network status, comprising:

a computer system comprising a computer-readable medium, said computer-readable medium further comprising;

a set of instructions and associated circuitry for monitoring an IP communications network, said IP communications network comprising a plurality of associated computer systems, said monitoring instructions further comprising;

a set of instructions and associated circuitry for evaluating logged data communicated on said IP communications network as said logged data is logged; and

a set of instructions and associated circuitry for detecting at least one data communications event from said logged data to be a potentially anomalous event by nominating said at least one data communications event and performing a time series generation associated with the communication of said at least one data communications event on said IP communications network;

a set of instructions and associated circuitry for discovering said potentially anomalous data event to be an anomalous event by forming a percentiled data set from said logged data and comparing said at least one data communications event to a threshold level associated with said percentiled data set; and

said computer system further comprising a set of instructions and associated circuitry for generating an alert signal in association with said monitoring step if said anomalous event differs from said percentiled data set by a level at least equal to said threshold level.

12. The system of claim 11, where said computer readable memory further comprises a set of instructions and associated circuitry for said analyzing potentially anomalous network activity involves comparison of predicted network activity versus observed network activity

13. The system of claim 12, where said computer readable memory further comprises a set of instructions and associated circuitry for generating a value recorded for an observed period.

14. The system of claim 13, where said computer readable memory further comprises a set of instructions and associated circuitry for generating statistics based profile records of said comparison.

15. The system of claim 11, where said computer readable memory further comprises:

a set of instructions and associated circuitry for recording an anomalous network status in said profile a predefined number of memory locations;

a set of instructions and associated circuitry for dividing the number of memory locations into discreet groups;

a set of instructions and associated circuitry for defining these discreet groups as percentage levels; and

a set of instructions and associated circuitry for inserting said comparison in appropriate memory locations for a predetermined amount of time.

16. The system of claim 15, where said computer readable memory further comprises a set of instructions and associated circuitry for generating at least one statistics based profile estimate of said predicted network activity associated with a function of weighted averages.

17. The system of claim 15, where said computer readable memory further comprises a set of instructions and associated circuitry for generating at least one statistics based profile estimate of said predicted network activity based as a function of Bayesian algorithms.

18. The system of claim 15, where said computer readable memory further comprises a set of instructions and associated circuitry for coordinating with other network monitoring methods to analyze the anomaly upon anomaly detection.

19. A computer readable medium, comprising:

a set of instructions and associated circuitry for monitoring an IP communications network, said IP communications network comprising a plurality of associated computer systems, said monitoring instructions further comprising;

a set of instructions and associated circuitry for evaluating logged data communicated on said IP communications network as said logged data is logged;

a set of instructions and associated circuitry for detecting at least one data communications event from said logged data to be a potentially anomalous event by nominating said at least one data communications event and performing a time series generation associated with the communication of said at least one data communications event on said IP communications network;

a set of instructions and associated circuitry for discovering said potentially anomalous data event to be an anomalous event by forming a percentiled data set from said logged data and comparing said at least one data communications event to a threshold level associated with said percentiled data set; and

said computer-readable memory further comprising a set of instructions and associated circuitry for generating an alert signal in association with said monitoring step if said anomalous event differs from said percentiled data set by a level at least equal to said threshold level.

20. The computer readable medium of claim 19 wherein said set of instructions and associated circuitry for analyzing potentially anomalous network activity further comprises a set of instructions and associated circuitry for comparison of predicted network activity versus observed network activity.