Automatic recursive search on derived information

• US 10,127,280 B2
• Filed: 02/24/2016
• Issued: 11/13/2018
• Est. Priority Date: 02/25/2015
• Status: Active Grant

• Current Assignee: Sumo Logic, Inc.
• Original Assignee: Sumo Logic, Inc.
• Inventors: Tidwell, Kenny, Frampton, David, O'Connell, Brendan
• Primary Examiner(s): Kuddus, Daniel
• Application Number:
  US15/052,529
• Publication Number:
  US 20160253387A1
• Time in Patent Office: 993 Days
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/282 Hierarchical databases, e.g. IMS, LDAP data stores or Lotus Notes
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: H04L 63/14 for detecting or protecting against malicious traffic
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002

Abstract

A processing device receives a query comprising a first field value and a time period. The processing device performs a first search of a data store using the first field value to identify a first plurality of events having the time period and a field that comprises the first field value. The processing device determines, for one of the plurality of events, a second field value of a second field that is specified in a first context definition, the second field having an assigned field type. The processing device automatically performs a second search of the data store using the additional field value to identify a second plurality of events having the time period and the additional field value. Information from the first plurality of events and the second plurality of events is aggregated, and a response to the query is generated that comprises the aggregated information.

Claims

1. A method comprising:

receiving, by a processing device, a query comprising a first field value and a time period;

performing, by the processing device, a first search of a data store using the first field value to identify a first plurality of events having the time period and at least one field that comprises the first field value;

determining one or more additional searches to perform automatically based on the first plurality of events returned by the first search, wherein determining a second search to perform based on a first event of the plurality of events comprises;

determining a first context definition associated with the first event, wherein the first context definition identifies a specified plurality of fields to use as link keys, each of the specified plurality of fields having assigned field types; and

determining a second field value of a second field in the first event, wherein the second field is one of the plurality of fields in the first context definition that is specified to use as a link key;

automatically performing, by the processing device without receipt of a second query, the second search of the data store using the second field value that was not included in the query to identify a second plurality of events having the time period and the second field value;

aggregating information from the first plurality of events and the second plurality of events; and

generating a response to the query that comprises the information aggregated from the first plurality of events and the second plurality of events.

2. The method of claim 1, further comprising:

determining a first source type associated with the first event, wherein the first source type is based on a source of the first event; and

determining the first context definition based on the first source type.

3. The method of claim 1, further comprising:

determining that a second event of the plurality of events is associated with a second context definition;

determining, for the second event, a third field value of a third field that is specified in the second context definition;

automatically performing a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregating information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

4. The method of claim 3, wherein the response comprises a first section comprising a first set of field values of a first set of fields specified in the first context definition and a second section comprising a second set of field values of a second set of fields specified in the second context definition.

5. The method of claim 1, further comprising:

determining that a second event of the first plurality of events is associated with the first context definition;

determining, for the second event, a third field value of the second field;

automatically performing a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregating information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

6. The method of claim 1, wherein the data store comprises a database, the method further comprising:

adding an additional field to the first context definition without modifying a schema of the database;

determining, for the first event, a third field value of the additional field;

performing a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregating information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

7. The method of claim 1, wherein the first context definition is an entry in configuration data that is stored at a location other than the data store.

8. A non-transitory computer readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:

receiving, by the processing device, a query comprising a first field value and a time period;

determining one or more additional searches to perform automatically based on a first plurality of events returned by a first search, wherein determining a second search to perform based on a first event of the plurality of events comprises;

determining a first context definition associated with the first event, wherein the first context definition identifies a specified plurality of fields to use as link keys, each of the specified plurality of fields having assigned field types; and

determining a second field value of a second field in the first event, wherein the second field is one of the plurality of fields in the first context definition that is specified to use as a link key;

automatically performing, by the processing device without receipt of a second query, the second search of a data store using the second field value that was not included in the query to identify a second plurality of events having the time period and the second field value;

aggregating information from the first plurality of events and the second plurality of events; and

generating a response to the query that comprises the information aggregated from the first plurality of events and the second plurality of events.

9. The non-transitory computer readable storage medium of claim 8, the operations further comprising:

determining a first source type associated with the first event, wherein the first source type is based on a source of the first event; and

determining the first context definition based on the first source type.

10. The non-transitory computer readable storage medium of claim 8, the operations further comprising:

determining that a second event of the plurality of events is associated with a second context definition;

determining, for the second event, a third field value of a third field that is specified in the second context definition;

automatically performing a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregating information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

11. The non-transitory computer readable storage medium of claim 10, wherein the response comprises a first section comprising a first set of field values of a first set of fields specified in the first context definition and a second section comprising a second set of field values of a second set of fields specified in the second context definition.

12. The non-transitory computer readable storage medium of claim 8, the operations further comprising:

determining that a second event of the first plurality of events is associated with the first context definition;

determining, for the second event, a third field value of the second field;

performing a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregating information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

13. The non-transitory computer readable storage medium of claim 8, wherein the data store comprises a database, the operations further comprising:

adding an additional field to the first context definition without modifying a schema of the database;

determining, for the first event, a third field value of the additional field;

performing a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregating information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

14. The non-transitory computer readable storage medium of claim 8, wherein the first context definition is an entry in configuration data that is stored at a location other than the data store.

15. A computing device comprising:

a memory; and

a processing device connected to the memory, wherein the processing device is configured to;

receive a query comprising a first field value and a time period;

perform a first search of a data store using the first field value to identify a first plurality of events having the time period and at least one field that comprises the first field value;

determine one or more additional searches to perform automatically based on the first plurality of events returned by the first search, wherein determining a second search to perform based on a first event of the plurality of events comprises;

determining a first context definition associated with the first event, wherein the first context definition identifies a specified plurality of fields to use as link keys, each of the specified plurality of fields having assigned field types; and

determining a second field value of a second field in the first event, wherein the second filed is one of the plurality of fields in the first context definition is specified to use as a link key;

automatically perform, without receipt of a second query, the second search of the data store using the second field value that was not included in the query to identify a second plurality of events having the time period and the second field value;

aggregate information from the first plurality of events and the second plurality of events; and

generate a response to the query that comprises the information aggregated from the first plurality of events and the second plurality of events.

16. The computing device claim 15, wherein the processing device is further to:

determine a first source type associated with the first event, wherein the first source type is based on a source of the first event; and

determine the first context definition based on the first source type.

17. The computing device of claim 15, wherein the processing device is further to:

determine that a second event of the plurality of events is associated with a second context definition;

determine, for the second event, a third field value of a third field that is specified in the second context definition;

automatically perform a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregate information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

18. The computing device of claim 17, wherein the response comprises a first section comprising a first set of field values of a first set of fields specified in the first context definition and a second section comprising a second set of field values of a second set of fields specified in the second context definition.

19. The computing device of claim 15, wherein the processing device is further to:

determine that a second event of the first plurality of events is associated with the first context definition;

determine, for the second event, a third field value of the second field;

automatically perform a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregate information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

20. The computing device of claim 15, wherein the data store comprises a database, and wherein the processing device is further to:

add an additional field to the first context definition without modifying a schema of the database;

determine, for the first event, a third field value of the additional field;

perform a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregate information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.