Streaming method and system for processing network metadata
First Claim
1. A method of processing network metadata generated on a network transmitting network traffic using one or more network protocols, the network including devices at least some of which receive network traffic through an ingress interface and transmit network traffic through an egress interface, the method comprising the steps of:
- receiving network metadata from a plurality of sources in a data processing system, in at least one data format;
determining the type or character of said network metadata;
processing said network metadata by applying at least one policy governing network metadata processing, wherein said at least one policy includes the steps of;
comparing the source of incoming network traffic to a predefined list of monitored off-limit devices on said network;
if the destination IP address is on a predefined list of off-limit devices, storing the source IP/port, as well as the destination IP/port in a potential alert list, along with the number of bytes and packets reported in the ingress NetFlow record;
examining output records to determine if the source IP/port and the destination IP/port match an entry in the potential alert list;
if a match is found, treating such match as an indication that an internal host replied to an outside peer request; and
generating an alert message in a timely manner to inform of a potential botnet infection; and
converting at least a portion of said network metadata into one or more different data formats that are used in said data processing system for other system metadata, in response, at least in part, to the results of said determining step.
1 Assignment
0 Petitions
Accused Products
Abstract
A method and system for processing network metadata is described. Network metadata may be processed by dynamically instantiated executable software modules which make policy-based decisions about the character of the network metadata and about presentation of the network metadata to consumers of the information carried by the network metadata. The network metadata may be type classified and each subclass within a type may be mapped to a definition by a unique fingerprint value. The fingerprint value may be used for matching the network metadata subclasses against relevant policies and transformation rules. For template-based network metadata such as NetFlow v9, an embodiment of the invention can constantly monitor network traffic for unknown templates, capture template definitions, and informs administrators about templates for which custom policies and conversion rules do not exist. Conversion modules can efficiently convert selected types and/or subclasses of network metadata into alternative metadata formats.
-
Citations
20 Claims
-
1. A method of processing network metadata generated on a network transmitting network traffic using one or more network protocols, the network including devices at least some of which receive network traffic through an ingress interface and transmit network traffic through an egress interface, the method comprising the steps of:
-
receiving network metadata from a plurality of sources in a data processing system, in at least one data format; determining the type or character of said network metadata; processing said network metadata by applying at least one policy governing network metadata processing, wherein said at least one policy includes the steps of; comparing the source of incoming network traffic to a predefined list of monitored off-limit devices on said network; if the destination IP address is on a predefined list of off-limit devices, storing the source IP/port, as well as the destination IP/port in a potential alert list, along with the number of bytes and packets reported in the ingress NetFlow record; examining output records to determine if the source IP/port and the destination IP/port match an entry in the potential alert list; if a match is found, treating such match as an indication that an internal host replied to an outside peer request; and generating an alert message in a timely manner to inform of a potential botnet infection; and converting at least a portion of said network metadata into one or more different data formats that are used in said data processing system for other system metadata, in response, at least in part, to the results of said determining step. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A method of processing network metadata generated on a network transmitting network traffic using one or more network protocols, the network including devices at least some of which receive network traffic through an ingress interface and transmit network traffic through an egress interface, the method comprising the steps of:
-
receiving network metadata from a plurality of sources in a data processing system, in at least one data format; determining the type or character of said network metadata; processing said network metadata by applying at least one policy governing network metadata processing, wherein said at least one policy includes the steps of; collecting information about internal network devices communicating with external network devices;
said information comprising a list of communicating devices'"'"' IP addresses and protocols;applying a streaming cluster analysis method to determine patterns in communications between the communicating devices; identifying abnormal instances of communications between the communicating devices by the presence of small disjoint patterns in the set of all computed patterns; generating an alert message in a timely manner to inform of a potential botnet infection if at least one abnormal communication pattern is identified; and converting at least a portion of said network metadata into one or more different data formats that are used in said data processing system for other system metadata, in response, at least in part, to the results of said determining step.
-
-
20. A method of processing network metadata generated on a network transmitting network traffic using one or more network protocols, the network including devices at least some of which receive network traffic through an ingress interface and transmit network traffic through an egress interface, the method comprising the steps of:
-
receiving network metadata from a plurality of sources in a data processing system, in at least one data format; determining the type or character of said network metadata; processing said network by applying at least one policy governing network metadata processing, wherein said at least one policy includes the steps of; comparing the source of incoming network traffic to a predefined list of monitored devices on said network; if the source IP address is on a predefined list of devices, storing the source IP in an in-memory database along with the number of bytes and packets reported in the ingress NetFlow record; at predefined intervals examining the in-memory database and identifying devices which exceeded a threshold set forth by a network manager; generating an alert message in a timely manner to inform of a traffic spike; and converting at least a portion of said network metadata into one or more different data formats that are used in said data processing system for other system metadata, in response, at least in part, to the results of said determining step.
-
Specification