Streaming Method and System for Processing Network Metadata
First Claim
1. A method of processing network metadata generated on a network transmitting network traffic using one or more network protocols, the network including devices at least some of which receive network traffic through an ingress interface and transmit network traffic through an egress interface, the method comprising the steps of:
- receiving network metadata from a plurality of sources in a data processing system, in at least one data format;
determining the type or character of said network metadata;
processing said network metadata to extract useful information therefrom; and
converting at least a portion of said network metadata into one or more different data formats that are used in said data processing system for other system metadata, in response, at least in part, to the results of said determining step.
1 Assignment
0 Petitions
Accused Products
Abstract
A method and system for processing network metadata is described. Network metadata may be processed by dynamically instantiated executable software modules which make policy-based decisions about the character of the network metadata and about presentation of the network metadata to consumers of the information carried by the network metadata. The network metadata may be type classified and each subclass within a type may be mapped to a definition by a unique fingerprint value. The fingerprint value may be used for matching the network metadata subclasses against relevant policies and transformation rules. For template-based network metadata such as NetFlow v9, an embodiment of the invention can constantly monitor network traffic for unknown templates, capture template definitions, and informs administrators about templates for which custom policies and conversion rules do not exist. Conversion modules can efficiently convert selected types and/or subclasses of network metadata into alternative metadata formats.
-
Citations
80 Claims
-
1. A method of processing network metadata generated on a network transmitting network traffic using one or more network protocols, the network including devices at least some of which receive network traffic through an ingress interface and transmit network traffic through an egress interface, the method comprising the steps of:
-
receiving network metadata from a plurality of sources in a data processing system, in at least one data format; determining the type or character of said network metadata; processing said network metadata to extract useful information therefrom; and converting at least a portion of said network metadata into one or more different data formats that are used in said data processing system for other system metadata, in response, at least in part, to the results of said determining step. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61)
-
-
62. A method of classifying network metadata comprising the steps of:
-
computing an invariant network metadata definition unique identifier; associating at least one processing rule with said network metadata unique identifier; storing said network metadata definition unique identifier in a first information container; storing said at least one processing rule in said first information container; associating said at least one processing rule with said network metadata definition unique identifier in said first information container; receiving a first packet of information containing a network metadata definition and a first transient network metadata identifier from a uniquely identified source; locating invariant network metadata definition in said first packet; computing an invariant network metadata definition unique identifier corresponding to the network metadata definition in said first packet; locating said network metadata definition unique identifier in the first information container; storing said first transient network metadata identifier and said source identifier in a second information container; associating said first transient network metadata identifier in the second information container with said network metadata definition unique identifier in the first information container; receiving a second packet of information containing network metadata, said first transient network metadata identifier and said source identifier; locating said first transient network metadata identifier and said source identifier in the second information container; using said association in the second information container to locate network metadata definition unique identifier in the first information container; using said association of network metadata definition unique identifier in the first information container to locate said processing rule; and executing said processing rule to classify network metadata. - View Dependent Claims (63, 64, 65, 66, 67, 68, 69, 70)
-
-
71. A method of selectively reducing the volume of network metadata supplied to a SIEM system on a network, comprising the steps of:
-
identifying network metadata packets that contain at least partially redundant metadata; and processing said network metadata packets that contain at least partially redundant metadata to aggregate said packets into a smaller number of packets while preserving the metadata of interest.
-
-
72. A system for processing network metadata comprising the steps of:
-
a network node adapted for receiving network metadata from a plurality of sources in at least one data format; a processing module for determining the type or character of said network metadata; a processing module for processing said network metadata to extract useful information therefrom; and a processing module for converting at least a portion of said network metadata into one or more different data formats in response, at least in part, to the results of said determining step.
-
-
73. A system for classifying network metadata comprising the steps of:
-
a processor for computing an invariant network metadata definition unique identifier; a processor for associating at least one processing rule with said network metadata unique identifier; a processor for storing said network metadata definition unique identifier in a first information container; a processor for storing said at least one processing rule in said first information container; a processor for associating said at least one processing rule with said network metadata definition unique identifier in said first information container; a processor for receiving a first packet of information containing a network metadata definition and a transient network metadata identifier; a processor for locating invariant network metadata definition in said first packet; a processor for computing an invariant network metadata definition unique identifier corresponding to the network metadata definition in said first packet; a processor for locating said network metadata definition unique identifier in the first information container; a processor for storing said transient network metadata identifier in a second information container; a processor for associating said transient network metadata identifier in the second information container with said network metadata definition unique identifier in the first information container; a processor for receiving a second packet of information containing network metadata and said transient network metadata identifier; a processor for locating said transient network metadata identifier in the second information container; a processor for using said association in the second information container to locate network metadata definition unique identifier in the first information container; a processor for using said association of network metadata definition unique identifier in the first information container to locate said processing rule; and a processor for executing said processing rule to classify network metadata.
-
-
74. A networked system comprising:
-
a local network comprising a plurality of devices on said network that generate a first collection of network metadata in syslog format; a network monitoring facility that is adapted to receive syslog data; a plurality of devices on said network that generate a second collection of network metadata in a format other than that of syslog data; a first network metadata processing facility capable of converting at least portions of said second collection of network metadata into syslog format; and communication media for supplying the converted output of said first network metadata processing facility as input to said network monitoring facility. - View Dependent Claims (75, 76, 77, 78, 79)
-
-
80. A method of enhancing reliability of transmission of NetFlow messages over a WAN from a branch network to a primary network, comprising the steps of:
-
converting at least a portion of the NetFlow messages generated on said branch network into syslog messages; and utilizing a network protocol more reliable than UDP to transfer said converted syslog messages from said branch network to a server on said primary network.
-
Specification