System and method for parsing, summarizing and reporting log data
First Claim
Patent Images
1. A data processing system comprising:
- one or more computers;
a log-producing device connected to the one or more computers through a network; and
,a non-transitory storage device storing computer instructions operable to cause the one or more computers to perform operations comprising;
receiving, by a log receiver, a log message from the log-producing device;
determining, by the log receiver, a property of the log message;
copying, by the log receiver, the log message into a first storage buffer;
parsing, by a log parser, content of the first storage buffer, including;
extracting data fields from the log message in the first storage buffer; and
converting the extracted data fields into one or more structured query language statements;
copying, by the log parser, the one or more structured query language statements into a second storage buffer;
reading, by a database inserter, the one or more structured query language statements in the second storage buffer, wherein reading the one or more structured query language statements comprises examining each of the one or more structured query language statements by the database inserter to determine a respective database table for storing a corresponding structured query language statement of the one of more structured query language statements;
inserting, by the database inserter, each of the one or more structured query language statements into a corresponding database table; and
summarizing, by a message collection engine, the one or more structured query language statements in the one or more database tables into summarized statements for storing on the non-transitory storage device.
14 Assignments
0 Petitions
Accused Products
Abstract
A system and method is disclosed which enables network administrators and the like to quickly analyze the data produced by log-producing devices such as network firewalls and routers. Unlike systems of the prior art, the system disclosed herein automatically parses and summarizes log data before inserting it into one or more databases. This greatly reduces the volume of data stored in the database and permits database queries to be run and reports generated while many types of attempted breaches of network security are still in progress. Database maintenance may also be accomplished automatically by the system to delete or archive old log data.
52 Citations
21 Claims
-
1. A data processing system comprising:
-
one or more computers; a log-producing device connected to the one or more computers through a network; and
,a non-transitory storage device storing computer instructions operable to cause the one or more computers to perform operations comprising; receiving, by a log receiver, a log message from the log-producing device; determining, by the log receiver, a property of the log message; copying, by the log receiver, the log message into a first storage buffer; parsing, by a log parser, content of the first storage buffer, including; extracting data fields from the log message in the first storage buffer; and converting the extracted data fields into one or more structured query language statements; copying, by the log parser, the one or more structured query language statements into a second storage buffer; reading, by a database inserter, the one or more structured query language statements in the second storage buffer, wherein reading the one or more structured query language statements comprises examining each of the one or more structured query language statements by the database inserter to determine a respective database table for storing a corresponding structured query language statement of the one of more structured query language statements; inserting, by the database inserter, each of the one or more structured query language statements into a corresponding database table; and summarizing, by a message collection engine, the one or more structured query language statements in the one or more database tables into summarized statements for storing on the non-transitory storage device. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A method comprising:
-
receiving, by a log receiver, a log message from the log-producing device; determining, by the log receiver, a property of the log message; copying, by the log receiver, the log message into a first storage buffer; parsing, by a log parser, content of the first storage buffer, including; extracting data fields from the log message in the first storage buffer; and converting the extracted data fields into one or more structured query language statements; copying, by the log parser, the one or more structured query language statements into a second storage buffer; reading, by a database inserter, the one or more structured query language statements in the second storage buffer, wherein reading the one or more structured query language statements comprises examining each of the one or more structured query language statements by the database inserter to determine a respective database table for storing a corresponding structured query language statement of the one or more structured query language statements; inserting, by the database inserter, each of the one or more structured query language statements into a corresponding database table; and summarizing, by a message collection engine, the one or more structured query language statements in the one or more database tables into summarized statements for storing on a non-transitory storage device, wherein the method is performed by one or more computers. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. A non-transitory storage device storing computer instructions operable to cause one or more computers to perform operations comprising:
-
receiving, by a log receiver, a log message from the log-producing device; determining, by the log receiver, a property of the log message; copying, by the log receiver, the log message into a first storage buffer; parsing, by a log parser, content of the first storage buffer, including; extracting data fields from the log message in the first storage buffer; and converting the extracted data fields into one or more structured query language statements; copying, by the log parser, the one or more structured query language statements into a second storage buffer; reading, by a database inserter, the one or more structured query language statements in the second storage buffer, wherein reading the one or more structured query language statements comprises examining each of the one or more structured query language statements by the database inserter to determine a respective database table for storing a corresponding structured query language statement of the one or more structured query language statements; inserting, by the database inserter, each of the one or more structured query language statements into a corresponding database table; and summarizing, by a message collection engine, the one or more structured query language statements in the one or more database tables into summarized statements for storing on the non-transitory storage device. - View Dependent Claims (16, 17, 18, 19, 20, 21)
-
Specification