Query pipeline
First Claim
Patent Images
1. A method of generating a query pipeline in a logging system comprising:
- parsing received log data into data chunks;
storing the data chunks in a storage;
generating metadata of the data chunks stored in the storage and storing the metadata in a database;
receiving a query request;
determining query operations based on the query request;
creating, by a processor, a query pipeline including the query operations based on the query request, wherein the query operations include a first query operator and a second query operator to perform a first query operation and a second query operation, wherein a result from the first query operation in the query pipeline is fed to the second query operation in the query pipeline;
identifying a username in the query request to determine a level of access of a user;
modifying the query pipeline, including adding a new operator between the first query operator and the second query operator to limit results from the query pipeline based on the determined level of access of the user;
optimizing, by the processor, the modified query pipeline including at least one of;
rearranging the query operators in the query pipeline, eliminating an operator from the query pipeline, or combining multiple operators in the query pipeline into a single operator; and
processing the query operations in the optimized query pipeline to obtain query results from the metadata in the database and the data chunks in the storage.
10 Assignments
0 Petitions
Accused Products
Abstract
A query pipeline is created (514) from a query request. The query pipeline includes multiple query operations including multiple query operators. A first query operator and a second query operator perform first and second query operations on a database (526) and on data outside the database (534). A result from the first query operation in the query pipeline is fed to the second query operation in the query pipeline.
9 Citations
14 Claims
-
1. A method of generating a query pipeline in a logging system comprising:
-
parsing received log data into data chunks; storing the data chunks in a storage; generating metadata of the data chunks stored in the storage and storing the metadata in a database; receiving a query request; determining query operations based on the query request; creating, by a processor, a query pipeline including the query operations based on the query request, wherein the query operations include a first query operator and a second query operator to perform a first query operation and a second query operation, wherein a result from the first query operation in the query pipeline is fed to the second query operation in the query pipeline; identifying a username in the query request to determine a level of access of a user; modifying the query pipeline, including adding a new operator between the first query operator and the second query operator to limit results from the query pipeline based on the determined level of access of the user; optimizing, by the processor, the modified query pipeline including at least one of;
rearranging the query operators in the query pipeline, eliminating an operator from the query pipeline, or combining multiple operators in the query pipeline into a single operator; andprocessing the query operations in the optimized query pipeline to obtain query results from the metadata in the database and the data chunks in the storage. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A logging system comprising:
-
a processor to parse received log data into data chunks, store the data chunks in a storage, generate metadata of the data chunks stored in the storage and store the metadata in a database, receive a query request, create a query pipeline including query operations based on the query request, wherein the query operations include a first query operator and a second query operator to perform a first query operation and a second query operation, identify a username in the query request to determine a level of access of a user;
modify the query pipeline by adding a new operator between the first query operator and the second query operator to limit results from the query pipeline based on the determined level of access of the user, optimize the modified query pipeline including at least one of;
rearranging the query operators in the query pipeline, eliminating an operator from the query pipeline, and combining multiple operators in the query pipeline into a single operator, process the query operations in the optimized query pipeline to obtain query results from the metadata in the database and the data chunks in the storage,wherein a result from the first query operation in the query pipeline is fed to the second query operation in the query pipeline. - View Dependent Claims (12)
-
-
13. A non-transitory computer readable medium storing machine readable instructions that when executed by a processor perform a method comprising:
-
parsing received log data into data chunks; storing the data chunks in a storage; generating metadata of the data chunks stored in the storage and storing the metadata in a relational database; receiving a query request; creating a query pipeline including query operations based on the query request, wherein the query operations include a first query operator and a second query operator to perform a first query operation and a second query operation, wherein a result from the first query operation in the query pipeline is fed to the second query operation in the query pipeline; identifying a username in the query request to determine a level of access of a user; modifying the query pipeline by adding a new operator between the first query operator and the second query operator to limit results from the query pipeline based on the determined level of access of the user; optimizing the modified query pipeline, including at least one of;
rearranging the query operators in the query pipeline, eliminating an operator from the query pipeline, and combining multiple operators in the query pipeline into a single operator; andprocessing the query operations in the optimized query pipeline to obtain query results from the metadata in the database and the data chunks in the storage. - View Dependent Claims (14)
-
Specification