Network event capture and retention system
First Claim
1. A method, comprising:
- collecting and storing a plurality of transmission events as network event data elements in a plurality of data structures, each transmission event being reported by one or more nodes of a network and stored in compressed form in at least one storage site;
extracting said plurality of transmission events stored as network event data elements in said plurality of data structures;
based on a set of predefined network event characteristics and the extracted plurality of transmission events, creating indices which identify data structures and locations of network event data elements within those data structures, wherein each index identifies a data structure and a respective location of a network event data element within that data structure;
receiving a query that requests particular transmission event information;
based on the query, accessing the indices to identify a location of at least one network event data element by apportioning said query into multiple partial queries that request the particular transmission event information and sending the multiple partial queries to different storage sites which store said plurality of data structures; and
in response to the multiple partial queries, receiving query results from the different storage sites and combining the query results to form an analyzable aggregation of transmission event information;
wherein the query results contain particular transmission events in the compressed form to maximize the amount of data conveyed in each disk cycle.
24 Assignments
0 Petitions
Accused Products
Abstract
Methods and apparatus are provided to monitor and analyze activity occurring on a networked computer system. In some embodiments, a method is provided for capturing, in a data structure, at least a portion of a notification describing a network event provided by a node on a computer network, identifying a data element (e.g., an IP address of the node) within the notification, and updating an index and/or summary based on the data element. The data structure may be stored in a file system maintained on a site, and sites may exchange information related to the notification data stored on each. In some embodiments, a query which is issued to a site may be processed using data transferred from other sites, and/or may be split into one or more additional queries which may be transmitted for processing to other sites.
44 Citations
52 Claims
-
1. A method, comprising:
-
collecting and storing a plurality of transmission events as network event data elements in a plurality of data structures, each transmission event being reported by one or more nodes of a network and stored in compressed form in at least one storage site; extracting said plurality of transmission events stored as network event data elements in said plurality of data structures; based on a set of predefined network event characteristics and the extracted plurality of transmission events, creating indices which identify data structures and locations of network event data elements within those data structures, wherein each index identifies a data structure and a respective location of a network event data element within that data structure; receiving a query that requests particular transmission event information; based on the query, accessing the indices to identify a location of at least one network event data element by apportioning said query into multiple partial queries that request the particular transmission event information and sending the multiple partial queries to different storage sites which store said plurality of data structures; and in response to the multiple partial queries, receiving query results from the different storage sites and combining the query results to form an analyzable aggregation of transmission event information; wherein the query results contain particular transmission events in the compressed form to maximize the amount of data conveyed in each disk cycle. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. An apparatus comprising:
-
a first controller processor, configured to collect and store a plurality of transmission events as network event data elements in a plurality of data structures, each transmission event being reported by one or more nodes of a network and stored in compressed form in at least one storage site; a second controller processor, configured to extract said plurality of transmission events stored as network event data elements in said plurality of data structures; and a third controller processor, configured to create indices which identify data structures and locations of network event data elements within those data structures based on a set of predefined network event characteristics and the extracted plurality of transmission events, wherein each index identifies a data structure and a respective location of a network event data element within that data structure; a forth control processor, configured to; receive a query that requests particular transmission event information; based on the query, access the indices to identify a location of at least one network event data element by apportioning said query into multiple partial queries that request the particular transmission event information and sending the multiple partial queries to different storage sites which store said plurality of data structures; and in response to the multiple partial queries, receive query results from the different storage sites and combine the query results to form an analyzable aggregation of transmission even information; wherein the query results contain particular transmission events in the compressed form to maximize the amount of data conveyed in each disk cycle. - View Dependent Claims (20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36)
-
-
37. A non-transitory computer-usable medium comprising computer readable instructions stored thereon for execution by a processor to perform a method comprising:
-
collecting and storing a plurality of transmission events as network event data elements in a plurality of data structures, each transmission event being reported by one or more nodes of a network and stored in compressed form in at least one storage site; extracting said plurality of transmission events stored as network event data elements in said plurality of data structures; based on a set of predefined network event characteristics and the extracted plurality of transmission events, creating indices which identify data structures and locations of network event data elements within those data structures, wherein each index identifies a data structure and a respective location of a network event data element within that data structure; receiving a query that requests particular transmission event information; based on the query, accessing the indices to identify a location of at least one network event data element by apportioning said query into multiple partial queries that request the particular transmission event information and sending the multiple partial queries to different storage sites which store said plurality of data structures; and in response to the multiple partial queries, receiving query results from the different storage sites and combining the query results to form an analyzable aggregation of transmission event information wherein the query results contain particular transmission events in the compressed form to maximize the amount of data conveyed in each disk cycle. - View Dependent Claims (38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52)
-
Specification