HIGH SPEED NETWORK DATA EXTRACTOR
First Claim
1. A system for network data extraction comprising:
- a packet decoding engine including at least one protocol decoder operable to decompose and parse received packets and identify extractable data entities from the received packets;
a data extraction engine operable to extract identified data entities from the packets and arrange the data entities into entity sets along with statistical elements reflective of redundant entity sets; and
an accumulator for storage of extracted entity sets and elimination of redundant entity sets.
2 Assignments
0 Petitions
Accused Products
Abstract
The present invention describes a system and method of extracting and storing data elements from network packets, thus performing the task of data mining. In one embodiment of the present invention incoming packets are decomposed one protocol layer at a time to extract data elements contained in the protocol headers. Layer-specific parsers perform deep packet inspection in order to extract data elements from upper-level protocols. Extracted data is arranged in rows, which are subsequently stored into a memory-based accumulator. After some length of time the accumulator is flushed to disk files. Another process reads the flushed disk files row-by-row, inserting each row into a relational database. Standard SQL operations are performed on the relational database in order to generate and display reports of the collected data.
-
Citations
41 Claims
-
1. A system for network data extraction comprising:
-
a packet decoding engine including at least one protocol decoder operable to decompose and parse received packets and identify extractable data entities from the received packets; a data extraction engine operable to extract identified data entities from the packets and arrange the data entities into entity sets along with statistical elements reflective of redundant entity sets; and an accumulator for storage of extracted entity sets and elimination of redundant entity sets. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A method for network data extraction comprising:
-
receiving packets of data, each packet having at least one protocol layer and at least one payload section; decomposing the received packets to delineate a location of the at least one protocol layer within each packet; parsing the received packets to access data entities in the at least one payload section within the packet; extracting data entities from the at least one protocol layer and payload section within the packet; constructing entity sets from the extracted data entities; inserting at least some of the entity sets into an accumulator; and inserting at least some of the entity sets from the accumulator into a relational database. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. The method of claim 1188, wherein the steps of decomposing and parsing the received packets are executed in parallel on multiple processors or processor cores.
-
20. A system for network data extraction comprising:
-
at least one processor operable to execute computer program instructions; at least one memory operable to store computer program instructions executable by the processor; and computer program instructions stored in the at least one memory and executable to perform the steps of; receiving packets of data, each packet having at least one protocol layer and at least one payload section; decomposing the received packets to delineate a location of the at least one protocol layer within each packet; parsing the received packets to access data entities in the at least one payload section within the packet; extracting data entities from the at least one protocol layer and payload section within the packet; constructing entity sets from the extracted data entities; inserting at least some of the entity sets into an accumulator; and inserting at least some of the entity sets from the accumulator into a relational database. - View Dependent Claims (21, 22, 23, 24, 25, 26, 27, 28, 29, 30)
-
-
31. A computer program product of network data extraction comprising:
-
a computer readable medium; and computer program instructions, recorded on the computer readable medium, executable by a processor, for performing the steps of; receiving packets of data, each packet having at least one protocol layer and at least one payload section; decomposing the received packets to delineate a location of the at least one protocol layer within each packet; parsing the received packets to access data entities in the at least one payload section within the packet; extracting data entities from the at least one protocol layer and payload section within the packet; constructing entity sets from the extracted data entities; inserting at least some of the entity sets into an accumulator; and inserting at least some of the entity sets from the accumulator into a relational database.. - View Dependent Claims (32, 33, 34, 35, 36, 37, 38, 39, 40, 41)
-
Specification