Streaming method and system for processing network metadata

• US 9,860,154 B2
• Filed: 01/22/2016
• Issued: 01/02/2018
• Est. Priority Date: 11/07/2011
• Status: Active Grant

• Current Assignee: NetFlow Logic Corporation
• Original Assignee: NetFlow Logic Corporation
• Inventors: Balabine, Igor, Velednitsky, Alexander
• Primary Examiner(s): Armouche, Hadi
• Assistant Examiner(s): LAKHIA, VIRAL S
• Application Number:
  US15/004,614
• Publication Number:
  US 20160234094A1
• Time in Patent Office: 711 Days
• Field of Search: 370230, 370241, 726 6- 8, 713150
• CPC Class Codes: G06F 21/55 Detecting local intrusion or implementing counter-measures
• CPC Class Codes: H04L 2463/144 Detection or countermeasures against botnets
• CPC Class Codes: H04L 41/069 using logs of notifications; Post-processing of notifications
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/18 Protocol analysers
• CPC Class Codes: H04L 47/2475 for supporting traffic characterised by the type of applications
• CPC Class Codes: H04L 63/102 Entity profiles
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

An improved method and system for processing network metadata is described. Network metadata may be processed by dynamically instantiated executable software modules which make policy-based decisions about the character of the network metadata and about presentation of the network metadata to consumers of the information carried by the network metadata. The network metadata may be type classified and each subclass within a type may be mapped to a definition by a unique fingerprint value. The fingerprint value may be used for matching the network metadata subclasses against relevant policies and transformation rules. For template-based network metadata such as NetFlow v9, an embodiment of the invention can constantly monitor network traffic for unknown templates, capture template definitions, and informs administrators about templates for which custom policies and conversion rules do not exist. Conversion modules can efficiently convert selected types and/or subclasses of network metadata into alternative metadata formats.

Claims

1. A method of improved management of a software-defined network, said network including a network controller and transmitting network traffic using one or more network protocols, the network including devices at least some of which receive network traffic through an ingress interface and transmit network traffic through an egress interface, the method comprising the steps of:

setting up communication paths with an OpenFlow controller;

receiving network metadata from a plurality of sources in a computing device, in at least one data format;

processing said network metadata, in real time as it is received, while said network metadata is in transition on said network between a network device that generated said network metadata and a device that is able to store said network metadata to retrieve Open Systems Interconnection (OSI) layer 7-information therefrom; and

determining as a result of said metadata processing step, information relating to applications operating on said network;

deriving user identity information from user-identity-aware NetFlow messages;

mapping the information relating to applications operating and the user identity information to a policy provided by a system administrator;

determining a state of the software-defined network;

determine if the applications operating and the user identify satisfy the policy; and

rerouting network traffic around unauthorized network devices by the network controller if the policy is not satisfied, by modifying lower level packet forwarding decisions of the OpenFlow controller with the information relating to applications operating and the user identity information.

2. The method as set forth in claim 1, further comprising the steps of:

determining as a result of said metadata processing step, information relating to the users present on said network.

3. A method of improved management of a cloud-based virtual computing environment, said environment including a cloud operating system and a cloud environment controller and transmitting network traffic using one or more network protocols, the network including devices at least some of which receive network traffic through an ingress interface and transmit network traffic through an egress interface, the method comprising the steps of:

setting up communication paths with an OpenFlow controller;

receiving network metadata, in a computing device, from a plurality of sources in said cloud-based virtual computing environment, in at least one data format;

processing said network metadata, in real time as it is received, while said network metadata is in transition in said environment between a network device that generated said network metadata and a device that is able to store said network metadata to retrieve Open Systems Interconnection (OSI) layer 7 information therefrom; and

determining as a result of said metadata processing step, information relating to applications operating in said environment;

deriving user identity information from user-identity-aware NetFlow messages;

mapping the information relating to applications operating and the user identity information to a policy provided by a system administrator;

determining a state of the cloud-based virtual computing environment;

determine if the applications operating and the user identify satisfy the policy; and

rerouting network traffic around unauthorized network devices by the network controller if the policy is not satisfied, by modifying lower level packet forwarding decisions of the OpenFlow controller with the information relating to applications operating and the user identity information.

4. The method as set forth in claim 3, further comprising the steps of:

determining as a result of said metadata processing step, information relating to the users present in said environment.

5. A system for improved management of a software-defined network, said network including a network controller and transmitting network traffic using one or more network protocols, the network including devices with memory at least some of which receive network traffic through an ingress interface and transmit network traffic through an egress interface and generate network metadata relating to said network traffic, said management system comprising:

an Open Flow controller for setting up communication paths;

at least one ingress interface for receiving network metadata from a plurality of sources in a software-defined network, in at least one data format;

a processing engine within a computing device for processing said network metadata, in real time as it is received, while said network metadata is in transition on said network between a network device that generated said network metadata and a device that is able to store said network metadata to retrieve Open Systems Interconnection (OSI) layer 7 information therefrom;

said processing engine deriving user identity information from user-identity-aware NetFlow messages, determining information relating to applications operating on said network, mapping the information relating to applications operating and user identity information to a policy provided by a system administrator, determining a state of the software-defined network, determine if the applications operating and the user identify satisfy the policy, and rerouting network traffic around unauthorized network devices by the network controller if the policy is not satisfied, by modifying lower level packet forwarding decisions of the Open Flow controller with the information relating to applications operating and the user identity information.

6. The management system as set forth in claim 5, wherein said processing engine determines as a result of said metadata processing step, information relating to the users present on said network.

7. A system for improved management of a cloud-based virtual computing environment, said environment including a cloud operating system and a cloud environment controller that transmits network traffic using one or more network protocols, the network including devices with memory at least some of which receive network traffic through an ingress interface and transmit network traffic through an egress interface, the management system further comprising:

an Open Flow controller for setting up communication paths;

an interface for receiving network metadata from a plurality of sources in said cloud-based virtual computing environment, in at least one data format;

a processing engine within a computing device for processing said network metadata, in real time as it is received, while said network metadata is in transition in said environment between a network device that generated said network metadata and a device that is able to store said network metadata to retrieve Open Systems Interconnection (OSI) layer 7 information therefrom;

said processing engine deriving user identity information from user-identity-aware NetFlow messages, determining, as a result of said metadata processing step, information relating to applications operating to a policy provided by a system administrator, mapping the information relating to applications operating and user identity information to the policy, determining a state of the environment, determine if the applications operating and the user identify satisfy the policy, and rerouting network traffic around unauthorized network devices by the network controller if the policy is not satisfied, by modifying lower level packet forwarding decisions of the OpenFlow controller with the information relating to applications operating and the user identity information.

8. The management system as set forth in claim 7, wherein said processing engine determines as a result of said metadata processing step, information relating to the users present in said environment.