Method and Apparatus for Network Packet Capture Distributed Storage System
First Claim
1. A method for capturing, storing, and retrieving pre-selected packets of data comprising the steps of:
- connecting at least one capture appliance to at least one predetermined data communications path, the capture appliance comprising;
a. a computer system board having a microprocessor and an internal clock;
b. dual port/dual channel volatile storage having sufficient capacity to use a least recently used algorithm to allocate space in the volatile storage;
c. at least one input/output communications adaptor port;
d. at least one non-volatile storage device numerically mapped to defined predetermined storage segments corresponding to a predetermined slot size;
e. a power supply;
f. a controller to map data onto the non-volatile storage device; and
g. an operating system;
promiscuously capturing a predetermined portion of data communicated along the data communications path and replicating the predetermined portion of data in the volatile storage;
aggregating the captured data in the volatile storage into a slot of the predetermined slot size by;
a. creating a slot of predetermined size, comprised of a predetermined number of buffers, each buffer having a predetermined size; and
b. collectively managing the slot based on a least recently used cache that maps the data in the slot to the non-volatile storage to create a cache image of the captured data across contiguous sectors of the non-volatile storage system using at least one high-performance parallel controller fabric to achieve striping and thereby allow the controller simultaneously to write to a plurality of non-volatile storage devices;
annotating the aggregated data based on pre-determined heuristics and non-volatile storage characteristics annotated by fixed time domain and mapped addressed for transfer to non-volatile storage;
storing the annotated, aggregated data in the non-volatile storage using an infinitely journaled, write-once, hierarchical file system to create at least one index of a set of predetermined characteristics that are useable to retrieve the data;
incorporating a means of reconstructing any corrupted data to ensure data accuracy;
retrieving a predetermined portion of captured data from the stored slot by;
a. identifying one or more characteristics of the data to be retrieved;
b. computing the locality of the slot containing the data having the identified characteristics on the non-volatile storage;
c. copying the data having the identified characteristics from the slot to the volatile storage using the least recently used algorithm to allocate space in the volatile storage;
d. packaging and conforming the data having the identified characteristics to be accessible using industry standard access methods; and
e. allowing a user to access and review the data having the identified characteristics.
17 Assignments
0 Petitions
Accused Products
Abstract
This is invention comprises a method an apparatus for Infinite Network Packet Capture System (INPCS). The INPCS is a high performance data capture recorder capable of capturing and archiving all network traffic present on a single network or multiple networks. This device can be attached to Ethernet networks via copper or SX fiber via either a SPAN port (101) router configuration or via an optical splitter (102). By this method, multiple sources or network traffic including gigabit Ethernet switches (102) may provide parallelized data feeds to the capture appliance (104), effectively increasing collective data capture capacity. Multiple captured streams are merged into a consolidated time indexed capture stream to support asymmetrically routed network traffic as well as other merged streams for external consumption.
209 Citations
22 Claims
-
1. A method for capturing, storing, and retrieving pre-selected packets of data comprising the steps of:
-
connecting at least one capture appliance to at least one predetermined data communications path, the capture appliance comprising;
a. a computer system board having a microprocessor and an internal clock;
b. dual port/dual channel volatile storage having sufficient capacity to use a least recently used algorithm to allocate space in the volatile storage;
c. at least one input/output communications adaptor port;
d. at least one non-volatile storage device numerically mapped to defined predetermined storage segments corresponding to a predetermined slot size;
e. a power supply;
f. a controller to map data onto the non-volatile storage device; and
g. an operating system;
promiscuously capturing a predetermined portion of data communicated along the data communications path and replicating the predetermined portion of data in the volatile storage;
aggregating the captured data in the volatile storage into a slot of the predetermined slot size by;
a. creating a slot of predetermined size, comprised of a predetermined number of buffers, each buffer having a predetermined size; and
b. collectively managing the slot based on a least recently used cache that maps the data in the slot to the non-volatile storage to create a cache image of the captured data across contiguous sectors of the non-volatile storage system using at least one high-performance parallel controller fabric to achieve striping and thereby allow the controller simultaneously to write to a plurality of non-volatile storage devices;
annotating the aggregated data based on pre-determined heuristics and non-volatile storage characteristics annotated by fixed time domain and mapped addressed for transfer to non-volatile storage;
storing the annotated, aggregated data in the non-volatile storage using an infinitely journaled, write-once, hierarchical file system to create at least one index of a set of predetermined characteristics that are useable to retrieve the data;
incorporating a means of reconstructing any corrupted data to ensure data accuracy;
retrieving a predetermined portion of captured data from the stored slot by;
a. identifying one or more characteristics of the data to be retrieved;
b. computing the locality of the slot containing the data having the identified characteristics on the non-volatile storage;
c. copying the data having the identified characteristics from the slot to the volatile storage using the least recently used algorithm to allocate space in the volatile storage;
d. packaging and conforming the data having the identified characteristics to be accessible using industry standard access methods; and
e. allowing a user to access and review the data having the identified characteristics.
-
-
2. A method for capturing data packets comprising the steps of:
-
connecting a capture device to a data communications path;
capturing data packets communicated along the data communications path;
persistently storing the captured data from the data packets in a predetermined combination of volatile and non-volatile storage media;
aggregating the persistently stored data packets into a slot of predetermined size;
annotating the aggregated data packets with persistent storage information;
storing the annotated data packets using an infinitely journaled, write-once, hierarchical file system;
incorporating a means of reconstructing any corrupted data to ensure data accuracy of the persistently stored data; and
retrieving a predetermined portion of captured data and persistently stored annotations from the slot. - View Dependent Claims (3, 4, 5, 6, 7, 8, 9)
-
-
10. A method for capturing data packets comprising the steps of:
-
connecting a capture appliance to data communications path;
capturing data communicated along the data communications path;
replicating and persistently annotating the captured data in a predetermined combination of volatile and non-volatile storage;
aggregating the captured data and persistent annotations in the volatile and non-volatile storage into a slot; and
storing the data in a non-volatile storage using an infinitely journaled, write-once, hierarchical file system. - View Dependent Claims (11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22)
-
-
14. (canceled)
Specification