Collection query driven generation of inverted index for raw machine data
First Claim
1. A method for searching data, the method comprising:
- providing a field searchable data store comprising a plurality of event records, each event record comprising a time-stamped portion of raw machine data;
receiving a query that comprises a plurality of parts including a collection query, wherein the collection query references a field name, wherein the field name is associated with a location in an event record containing a field value associated with the field name, wherein the collection query is user initiated, and wherein a first part in the plurality of parts is associated with the collection query and executable to generate an inverted index, and wherein one or more additional parts in the plurality of parts are executable for performing additional processing of the data in the inverted index;
responsive to the collection query, generating an inverted index by;
determining an extraction rule associated with the field name;
extracting a field value corresponding to the field name from one or more event records in the field searchable data store using the extraction rule; and
populating the inverted index responsive to each extracted field value, wherein each entry comprises the field name, the corresponding field value and a reference value that identifies a location in the field searchable data store where an associated event record is stored; and
performing the additional processing of the data in the inverted index in accordance with the one or more additional parts in the plurality of parts of the query.
1 Assignment
0 Petitions
Accused Products
Abstract
Embodiments of the present disclosure provide a method for generating an inverted index in accordance with a user generated collection query. The method comprises providing a field searchable data store that comprises a plurality of event records, each event record comprising a time-stamped portion of raw machine data. The method further comprises receiving a collection query that references a field name. Further, responsive to the collection query, an inverted index is generated by: a) determining an extraction rule associated with the field name; b) extracting a field value corresponding to the field name from one or more event records in the field searchable data store using the extraction rule; and c) populating the inverted index responsive to each extracted field value, wherein each entry comprises the field name, the corresponding field value and a reference value that identifies a location in the field searchable data store where an associated event record is stored.
-
Citations
31 Claims
-
1. A method for searching data, the method comprising:
-
providing a field searchable data store comprising a plurality of event records, each event record comprising a time-stamped portion of raw machine data; receiving a query that comprises a plurality of parts including a collection query, wherein the collection query references a field name, wherein the field name is associated with a location in an event record containing a field value associated with the field name, wherein the collection query is user initiated, and wherein a first part in the plurality of parts is associated with the collection query and executable to generate an inverted index, and wherein one or more additional parts in the plurality of parts are executable for performing additional processing of the data in the inverted index; responsive to the collection query, generating an inverted index by; determining an extraction rule associated with the field name; extracting a field value corresponding to the field name from one or more event records in the field searchable data store using the extraction rule; and populating the inverted index responsive to each extracted field value, wherein each entry comprises the field name, the corresponding field value and a reference value that identifies a location in the field searchable data store where an associated event record is stored; and performing the additional processing of the data in the inverted index in accordance with the one or more additional parts in the plurality of parts of the query. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A method for searching data, the method comprising:
-
providing a field searchable data store comprising a plurality of event records, each event record comprising a time-stamped portion of raw machine data; receiving a query that comprises a plurality of parts including a collection query, wherein the collection query references a field name, wherein the field name is associated with a location in an event record containing a field value associated with the field name, wherein the collection query is user initiated, and wherein a first part in the plurality of parts is associated with the collection query and executable to generate an inverted index, and wherein one or more additional parts in the plurality of parts are executable for performing additional processing of the data in the inverted index; responsive to the collection query, generating an inverted index by; determining at least one extraction rule, wherein each field name is associated with an extraction rule; extracting a field value corresponding to each of the at least one field names from one or more event records in the field searchable data store using one of the at least one extraction rules; and populating the inverted index responsive to each extracted field value, wherein each entry comprises the field name, the corresponding field value and a reference value that identifies a location in the field searchable data store where an associated event record is stored; and performing the additional processing of the data in the inverted index in accordance with the one or more additional parts in the plurality of parts of the query. - View Dependent Claims (15, 16, 17, 18, 19, 20, 21)
-
-
22. A network device that is operative for searching data, the network device comprising:
-
a transceiver that is operative to communicate over a network; a memory that is operative to store at least one instruction; and a processor device that is operative to execute instructions that enable actions, the actions comprising; providing a field searchable data store comprising a plurality of event records, each event record comprising a time-stamped portion of raw machine data; receiving a query that comprises a plurality of parts including a collection query, wherein the collection query references a field name, wherein the field name is associated with a location in an event record containing a field value associated with the field name, wherein the collection query is user initiated, and wherein a first part in the plurality of parts is associated with the collection query and executable to generate an inverted index, and wherein one or more additional parts in the plurality of parts are executable for performing additional processing of the data in the inverted index; responsive to the collection query, generating an inverted index by; retrieving an extraction rule associated with the field name from a configuration file; extracting a field value corresponding to the field name from one or more event records in the field searchable data store using the extraction rule; and populating the inverted index responsive to each extracted field value, wherein each entry comprises the field name, the corresponding field value and a reference value that identifies a location in the field searchable data store where an associated event record is stored; and performing the additional processing of the data in the inverted index in accordance with the one or more additional parts in the plurality of parts of the query. - View Dependent Claims (23, 24, 25, 26, 27, 28, 29, 30, 31)
-
Specification