INTELLIGENT FEEDBACK LOOP TO ITERATIVELY REDUCE INCOMING NETWORK DATA FOR ANALYSIS

• US 20140086102A1
• Filed: 09/25/2012
• Published: 03/27/2014
• Est. Priority Date: 09/25/2012
• Status: Active Grant

• Current Assignee: SS8, Inc.
• Original Assignee: SS8, Inc.
• Inventors: Doddapaneni, Ashok Babu
• Granted Patent: US 9,350,762 B2
• Time in Patent Office: Days
• US Class Current: 370/259
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/30 for supporting lawful interception, monitoring or retaining of communications or communication related information circuit switched telephony call monitoring H04M3/2281

Abstract

A method, apparatus and system related to an intelligent feedback loop to iteratively reduce target packet analysis is disclosed. According to one embodiment, a method of a network traffic monitoring system includes processing a flow data received through an aggregation switch of a network traffic monitoring system in a first stage module of the network traffic monitoring system, filtering the flow data to a target data based on a packet classification in the first stage module, determining that a portion of a target data is an extraneous data based on a content filtering algorithm applied in a data processing system of the network traffic monitoring system, and iteratively removing from the target data the extraneous data based on a feedback loop created between the data processing system and the first stage module of the network traffic monitoring system.

Claims

1. A machine-readable medium that provides instructions, that when executed by a machine, enables the machine to perform operations comprising:

determining that a portion of a target data is an extraneous data based on a content filtering algorithm applied in a second stage of a network traffic monitoring system; and

iteratively removing from the target data the extraneous data based on a feedback loop created between the second stage and the first stage of the network traffic monitoring system.

2. The machine readable medium of claim 1 further comprising:

processing a flow data received through an aggregation switch of a network traffic monitoring system in the first stage; and

converting the flow data to a target data based on a packet classification in the first stage.

3. The machine readable medium of claim 2 further comprising:

applying a content filtering algorithm to the target data in the second stage; and

communicating which portion of the target data is extraneous from the second stage to the first stage based on the content filtering algorithm.

4. The machine readable medium of claim 3 further comprising:

applying a static filtering algorithm of the flow data based on an internet protocol analysis and a port analysis in the first stage;

applying a tuple hash algorithm to map the flow data having a variable length into an ordered list of elements having a fixed length of the flow data in the first stage; and

dynamically filtering the flow data in the first stage based on the static filtering algorithm and the tuple hash algorithm, and based on a removal of the extraneous data communicated from a filtering intelligence algorithm of the second stage.

5. The machine readable medium of claim 4 further comprising:

applying a zero-copy driver and a use buffer in at least one of the first stage and the second stage;

reducing processing power and memory usage through the application of the zero-copy driver and the use buffer in the second stage;

buffering the target data in a random access memory in the second stage;

classifying a set of protocols associated with the target data in the second stage;

applying the filtering intelligence algorithm to extract the extraneous data from the target data in the second stage; and

determining a communication mode between the second stage and the dynamic filtering operation of the first stage so that a request to remove the extraneous data based on the filtering intelligence algorithm of the second stage is executable.

6. The machine readable medium of claim 5 further comprising:

extracting a meta data associated with the target data in the second stage;

communicating the extracted meta data to an data retention server;

applying a regex based targeting algorithm to the target data in the second stage to produce a set of regular expressions describing a search pattern; and

communicating the set of regular expressions to a master controller.

7. The machine readable medium of claim 6 further comprising:

analyzing the target data to discover an action of interest in the set of regular expressions associated with a target individual in the second stage,

wherein the action of interest is subject to a governmental permission as to how the action of interest is usable in a lawful data interception system.

8. A network traffic monitoring system comprising:

an aggregation switch to consolidate a flow data;

a first stage module to create a target data from the flow data through an iterative exclusion of an extraneous data; and

a data processing system comprising a processor and a memory to iteratively remove from the target data the extraneous data, and to form a feedback loop between the data processing system and the first stage module of the network traffic monitoring system.

9. The network traffic monitoring system of claim 8 wherein the first stage module:

to process the flow data received through an aggregation switch of the network traffic monitoring system, and

to convert the flow data to a target data based on a packet classification in the first stage.

10. The network traffic monitoring system of claim 9 wherein the data processing system:

to apply a content filtering algorithm to the target data in the data processing system; and

to communicate which portion of the target data is extraneous from the data processing system to the first stage module based on the content filtering algorithm.

11. The network traffic monitoring system of claim 10 wherein the first stage module to:

apply a static filtering algorithm of the flow data based on an internet protocol analysis and a port analysis using the first stage module;

apply a tuple hash algorithm to map the flow data having a variable length into an ordered list of elements having a fixed length of the flow data in the first stage; and

dynamically filter the flow data in the first stage based on the static filtering algorithm and the tuple hash algorithm, and based on a removal of the extraneous data communicated from a filtering intelligence algorithm of the data processing module.

12. The network traffic monitoring system of claim 11 wherein the data processing system:

to apply a zero-copy driver and a use buffer in the data processing system;

to reduce processing power and memory usage through the application of the zero-copy driver and the use buffer in the data processing system;

to buffer the target data in a random access memory in the data processing system;

to classify a set of protocols associated with the target data in the data processing system;

to apply the filtering intelligence algorithm to extract the extraneous data from the target data in the data processing system; and

determine a communication mode between the data processing system and the dynamic filtering operation of the first stage so that a request to remove the extraneous data based on the filtering intelligence algorithm of the data processing system is executable.

13. The network traffic monitoring system of claim 12 wherein the data processing system:

to extract a meta data associated with the target data in the data processing system;

to communicate the extracted meta data to a data retention server;

to apply a regex based targeting algorithm to the target data in the data processing system to produce a set of regular expressions describing a search pattern; and

to communicate the set of regular expressions to a master controller.

14. The network traffic monitoring system of claim 13 wherein the data processing system:

to analyze the target data to discover an action of interest in the set of regular expressions associated with a target individual in the data processing system,

wherein the action of interest is subject to a governmental permission as to how the action of interest is usable in a lawful data interception system.

15. A method of a network traffic monitoring system comprising:

processing a flow data received through an aggregation switch of a network traffic monitoring system in a first stage module of the network traffic monitoring system;

filtering the flow data to a target data based on a packet classification in the first stage module;

determining that a portion of a target data is an extraneous data based on a content filtering algorithm applied in a data processing system of the network traffic monitoring system; and

iteratively removing from the target data the extraneous data based on a feedback loop created between the data processing system and the first stage module of the network traffic monitoring system.

16. The method of a network traffic monitoring system of claim 15 further comprising:

applying a content filtering algorithm to the target data in the data processing system; and

communicating which portion of the target data is extraneous from the data processing system to the first stage module based on the content filtering algorithm.

17. The method of a network traffic monitoring system of claim 16 further comprising:

applying a static filtering algorithm of the flow data based on an internet protocol analysis and a port analysis in the first stage;

applying a tuple hash algorithm to map the flow data having a variable length into an ordered list of elements having a fixed length of the flow data in the first stage; and

dynamically filtering the flow data in the first stage module based on the static filtering algorithm and the tuple hash algorithm, and based on a removal of the extraneous data communicated from a filtering intelligence algorithm of the data processing system.

18. The method of a network traffic monitoring system of claim 17 further comprising:

applying a zero-copy driver and a use buffer in the data processing system;

reducing processing power and memory usage through the application of the zero-copy driver and the use buffer in the data processing system;

buffering the target data in a random access memory in the data processing system;

classifying a set of protocols associated with the target data in the data processing system;

applying the filtering intelligence algorithm to extract the extraneous data from the target data in the data processing system; and

determining a communication mode between the data processing system and the dynamic filtering operation of the first stage module so that a request to remove the extraneous data based on the filtering intelligence algorithm of the data processing system is executable.

19. The method of a network traffic monitoring system of claim 18 further comprising:

extracting a meta data associated with the target data in the data processing system;

communicating the extracted meta data to a data retention server;

applying a regex based targeting algorithm to the target data in the data processing system to produce a set of regular expressions describing a search pattern; and

communicating the set of regular expressions to a master controller.

20. The method of a network traffic monitoring system of claim 19 further comprising:

analyzing the target data to discover an action of interest in the set of regular expressions associated with a target individual in the data processing system,

wherein the action of interest is subject to a governmental permission as to how the action of interest is usable in a lawful data interception system.