Method and system for confident anomaly detection in computer network traffic

• US 9,843,488 B2
• Filed: 02/20/2015
• Issued: 12/12/2017
• Est. Priority Date: 11/07/2011
• Status: Active Grant

• Current Assignee: NetFlow Logic Corporation
• Original Assignee: NetFlow Logic Corporation
• Inventors: Balabine, Igor, Velednitsky, Alexander
• Primary Examiner(s): Smithers, Matthew
• Assistant Examiner(s): Lapian, Alexander
• Application Number:
  US14/627,963
• Publication Number:
  US 20150229661A1
• Time in Patent Office: 1,026 Days
• CPC Class Codes: G06F 21/554 involving event detection and direct action
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1458 Denial of Service

Abstract

The present invention relates to systems and methods for detecting anomalies in computer network traffic with fewer false positives and without the need for time-consuming and unreliable historical baselines. Upon detection, traffic anomalies can be processed to determine valuable network insights, including health of interfaces, devices and network services, as well as to provide timely alerts in the event of attack.

Claims

1. A method for detecting and classifying network traffic anomalies, comprising:

receiving a packet of information related to network traffic;

passing said packet to one or a plurality of network traffic analyzers;

at least some of said network traffic analyzers capable of applying an analytical algorithm to information contained in said packet that is different from the analytical algorithm applied by another of said network traffic analyzers;

receiving results of analysis performed by said analyzers, wherein the results include at least network traffic volume and packet rate;

evaluating results of analysis performed by said analyzers as a collection, by performing a fuzzy classification of the traffic volume and packet rate into linguistic classifications;

computing a first attention level for a given traffic volume and packet rate using Mamdani method;

computing a second attention level for a given traffic volume and packet rate using Sugeno method;

computing an effective attention level by averaging the first attention level and the second attention level, wherein the effective attention level is a measure of an operator'"'"'s attention required at a network node;

computing a network health score by subtracting the effective attention level from one;

determining if the network health score signifies a network traffic anomaly;

emitting an alert if the result of evaluation signifies a network traffic anomaly;

computing a network health trend using the network health score over time; and

improving network functioning using the network health trend to allocate network resources.

2. A method as set forth in claim 1, further comprising the step of performing trend analysis upon the results of said evaluating step to reduce false positives.

3. A method for detecting and classifying network traffic anomalies, comprising:

receiving a stream of packets of information related to network traffic;

passing at least a portion of said stream of information packets to a network traffic analyzer;

applying at least one analytical algorithm to said portion of said stream of information packets to determine at least network traffic volume and packet rate, wherein the at least one analytical algorithm performs a fuzzy classification of the traffic volume and packet rate into linguistic classifications;

computing a first attention level for a given traffic volume and packet rate using Mamdani method;

computing a second attention level for a given traffic volume and packet rate using Sugeno method;

computing an effective attention level by averaging the first attention level and the second attention level, wherein the effective attention level is a measure of an operator'"'"'s attention required at a network node;

computing a network health score by subtracting the effective attention level from one;

determining if said applying step indicates the existence of a network traffic anomaly;

emitting an alert if a network traffic anomaly is detected;

wherein said applying and said determining step are practiced prior to any step of permanently storing said portion of said stream of information packets;

computing a network health trend using the network health score over time; and

improving network functioning using the network health trend to allocate network resources.

4. A method for assessing the condition of an interface of a network device, comprising:

receiving a stream of packets of information related to network traffic passing through said network device interface;

passing at least a portion of said stream of information packets to a network traffic analyzer;

applying at least one analytical algorithm to said portion of said stream of information packets to determine at least network traffic volume and packet rate, wherein the at least one analytical algorithm performs a fuzzy classification of the traffic volume and packet rate into linguistic classifications;

computing a first attention level for a given traffic volume and packet rate using Mamdani method;

computing a second attention level for a given traffic volume and packet rate using Sugeno method;

computing an effective attention level by averaging the first attention level and the second attention level, wherein the effective attention level is a measure of an operator'"'"'s attention required at a network node;

computing a network health score by subtracting the effective attention level from one;

wherein the network health score is a metric for assessing operational condition of said network device interface;

emitting an alert if said computed metric indicates an abnormal operational condition of said network device interface;

wherein said applying and said metric computation are practiced prior to any step of permanently storing said portion of said stream of information packets;

computing a network health trend using the network health score over time; and

improving network functioning using the network health trend to allocate network resources.

5. A method as set forth in claim 4, further comprising the step of performing trend analysis upon the results of said metric computation to predict a risk of failure of said network device interface.

6. A method as set forth in claim 4, further comprising the step of assessing the operational condition of the said network device as a function of assessed operational condition of the interface of said network device;

wherein said assessing step is practiced prior to any step of permanently storing said portion of said stream of information packets.

7. A method as set forth in claim 6, further comprising the step of assessing the operational condition of a network service, comprising:

receiving a stream of packets of information related to network traffic passing through more than one network devices that forward said network traffic to and from said network service;

passing at least a portion of said stream of information packets to a network traffic analyzer;

applying said method to information packets pertaining to at least a portion of said network devices that forward said network traffic to and from said network service;

determining if said applying step indicates an abnormal operational condition of at least one of said network devices;

emitting an alert pertaining to a network service if said applying step indicates an abnormal operational condition of at least one said network device that forwards said network traffic to and from said network service;

wherein said determining and said emitting steps are practiced prior to any step of permanently storing said portion of said stream of information packets.

8. A system for detecting and classifying network traffic anomalies, comprising:

a storage device and processing device of network metadata in syslog format configured to perform the steps of;

receiving a packet of information related to network traffic and selectively passing said packet to one or a plurality of network traffic analysis steps;

at least some of said network traffic analysis steps capable of applying an analytical algorithm to information contained in said packet that is different from the analytical algorithm applied by another of said network traffic analysis steps;

receiving results of analysis performed by said analysis steps, wherein the results include at least network traffic volume and packet rate;

evaluating results of analysis performed by said analysis steps as a collection, by performing a fuzzy classification of the traffic volume and packet rate into linguistic classifications;

computing a first attention level for a given traffic volume and packet rate using Mamdani method;

computing a second attention level for a given traffic volume and packet rate using Sugeno method;

computing an effective attention level by averaging the first attention level and the second attention level, wherein the effective attention level is a measure of an operator'"'"'s attention required at a network node;

computing a network health score by subtracting the effective attention level from one;

determining if the network health score signifies a network traffic anomaly and emitting an alert if the result of evaluation signifies a network traffic anomaly;

computing a network health trend using the network health score over time; and

improving network functioning using the network health trend to allocate network resources.

9. A system as set forth in claim 8, further comprising performing trend analysis upon the results of said evaluating step to reduce false positives.

10. A system for detecting and classifying network traffic anomalies, comprising:

a storage device and processing device of network metadata in syslog format configured to perform the steps of;

receiving a stream of packets of information related to network traffic and passing at least a portion of said stream of information packets to a network traffic analysis step;

said network traffic analysis step capable of applying at least one analytical algorithm to said portion of said stream of information packets to determine at least network traffic volume and packet rate, wherein the at least one analytical algorithm performs a fuzzy classification of the traffic volume and packet rate into linguistic classifications;

computing a first attention level for a given traffic volume and packet rate using Mamdani method;

computing a second attention level for a given traffic volume and packet rate using Sugeno method;

computing an effective attention level by averaging the first attention level and the second attention level, wherein the effective attention level is a measure of an operator'"'"'s attention required at a network node;

computing a network health score by subtracting the effective attention level from one;

determining if the results thereof indicate the existence of a network traffic anomaly and emitting an alert if a network traffic anomaly is detected;

wherein said network traffic analysis step performs at least a portion of applying and determining actions prior to permanently storing said portion of said stream of information packets;

computing a network health trend using the network health score over time; and

improving network functioning using the network health trend to allocate network resources.

11. A system for assessing the condition of an interface of a network device, comprising:

a storage device and processing device of network metadata in syslog format configured to perform the steps of;

receiving a stream of packets of information related to network traffic passing through said network device interface and passing at least a portion of said stream of information packets to a network traffic analysis step;

said network traffic analysis step capable of applying at least one analytical algorithm to said portion of said stream of information packets to determine at least network traffic volume and packet rate, wherein the at least one analytical algorithm performs a fuzzy classification of the traffic volume and packet rate into linguistic classifications;

computing a first attention level for a given traffic volume and packet rate using Mamdani method;

computing a second attention level for a given traffic volume and packet rate using Sugeno method;

computing an effective attention level by averaging the first attention level and the second attention level, wherein the effective attention level is a measure of an operator'"'"'s attention required at a network node;

computing a network health score by subtracting the effective attention level from one, wherein the network health score is a metric for assessing operational condition of said network device interface;

receiving said computed metric and emitting an alert if said computed metric indicates an abnormal operational condition of said network device interface;

wherein said network traffic analysis step performs said metric computation prior to permanently storing said portion of said stream of information packets;

computing a network health trend using the network health score over time; and

improving network functioning using the network health trend to allocate network resources.

12. A system as set forth in claim 11, further comprising performing trend analysis upon the results of said metric computation to predict a risk of failure of said network device interface.

13. A system as set forth in claim 11, further comprising assessing the operational condition of the said network device as a function of assessed operational condition of the interfaces of said network device;

wherein said assessing step is practiced prior to any step of permanently storing said portion of said stream of information packets.

14. A system as set forth in claim 13, further comprising the capability for assessing the operational condition of a network service, comprising:

receiving a stream of packets of information related to network traffic passing through the network devices that forward said network traffic to and from said network service and selectively passing at least a portion of said stream of information packets to a network traffic analysis step;

wherein said information packets pertain to at least a portion of said network devices that forward said network traffic to and from said network service;

said network traffic analysis step capable of determining if said applying step indicates an abnormal operational condition of at least one of said network devices;

emitting an alert pertaining to a network service if said applying step indicates an abnormal operational condition of at least one said network device that forwards said network traffic to and from said network service;

wherein said network traffic analysis step and said emitting is performed prior to permanently storing said portion of said stream of information packets.