AUTOMATIC RECURSIVE SEARCH ON DERIVED INFORMATION

• US 20160253387A1
• Filed: 02/24/2016
• Published: 09/01/2016
• Est. Priority Date: 02/25/2015
• Status: Active Grant

• Current Assignee: Sumo Logic, Inc.
• Original Assignee: FactorChain, Inc. (Sumo Logic, Inc.)
• Inventors: Tidwell, Kenny, Frampton, David, O'Connell, Brendan
• Granted Patent: US 10,127,280 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/282 Hierarchical databases, e.g. IMS, LDAP data stores or Lotus Notes
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: H04L 63/14 for detecting or protecting against malicious traffic
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H05K 999/99 dummy group

Abstract

A processing device receives a query comprising a first field value and a time period. The processing device performs a first search of a data store using the first field value to identify a first plurality of events having the time period and a field that comprises the first field value. The processing device determines, for one of the plurality of events, a second field value of a second field that is specified in a first context definition, the second field having an assigned field type. The processing device automatically performs a second search of the data store using the additional field value to identify a second plurality of events having the time period and the additional field value. Information from the first plurality of events and the second plurality of events is aggregated, and a response to the query is generated that comprises the aggregated information.

Claims

1. A method comprising:

receiving, by a processing device, a query comprising a first field value and a time period;

performing, by the processing device, a first search of a data store using the first field value to identify a first plurality of events having the time period and at least one field that comprises the first field value;

determining, for a first event of the plurality of events, a second field value of a second field that is specified in a first context definition, the second field having an assigned field type;

performing, by the processing device without receipt of a second query, a second search of the data store using the additional field value to identify a second plurality of events having the time period and the additional field value;

aggregating information from the first plurality of events and the second plurality of events; and

generating a response to the query that comprises the information aggregated from the first plurality of events and the second plurality of events.

2. The method of claim 1, further comprising:

determining a first source type associated with the first event, wherein the first source type is based on a source of the first event; and

determining that the first source type is associated with the first context definition, wherein the first context definition identifies a specified plurality of fields to use as link keys, each of the specified plurality of fields having assigned field types, and wherein the second field is included in the specified plurality of fields.

3. The method of claim 1, further comprising:

determining that a second event of the plurality of events is associated with a second context definition;

determining, for the second event, a third field value of a third field that is specified in the second context definition;

performing a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregating information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

4. The method of claim 3, wherein the response comprises a first section comprising a first set of field values of a first set of fields specified in the first context definition and a second section comprising a second set of field values of a second set of fields specified in the second context definition.

5. The method of claim 1, further comprising:

determining that a second event of the first plurality of events is associated with the first context definition;

determining, for the second event, a third field value of the second field;

performing a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregating information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

6. The method of claim 1, wherein the data store comprises a database, the method further comprising:

adding an additional field to the first context definition without modifying a schema of the database;

determining, for the first event, a third field value of the additional field;

performing a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregating information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

7. The method of claim 1, wherein the first context definition is an entry in configuration data that is external to the data store.

8. A computer readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:

receiving, by the processing device, a query comprising a first field value and a time period;

performing, by the processing device, a first search of a data store using the first field value to identify a first plurality of events having the time period and at least one field that comprises the first field value;

determining, for a first event of the plurality of events, a second field value of a second field that is specified in a first context definition, the second field having an assigned field type;

performing, by the processing device without receipt of a second query, a second search of the data store using the additional field value to identify a second plurality of events having the time period and the additional field value;

aggregating information from the first plurality of events and the second plurality of events; and

generating a response to the query that comprises the information aggregated from the first plurality of events and the second plurality of events.

9. The computer readable storage medium of claim 8, the operations further comprising:

determining a first source type associated with the first event, wherein the first source type is based on a source of the first event; and

determining that the first source type is associated with the first context definition, wherein the first context definition identifies a specified plurality of fields to use as link keys, each of the specified plurality of fields having assigned field types, and wherein the second field is included in the specified plurality of fields.

10. The computer readable storage medium of claim 8, the operations further comprising:

determining that a second event of the plurality of events is associated with a second context definition;

determining, for the second event, a third field value of a third field that is specified in the second context definition;

performing a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregating information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

11. The computer readable storage medium of claim 10, wherein the response comprises a first section comprising a first set of field values of a first set of fields specified in the first context definition and a second section comprising a second set of field values of a second set of fields specified in the second context definition.

12. The computer readable storage medium of claim 8, the operations further comprising:

determining that a second event of the first plurality of events is associated with the first context definition;

determining, for the second event, a third field value of the second field;

performing a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregating information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

13. The computer readable storage medium of claim 8, wherein the data store comprises a database, the operations further comprising:

adding an additional field to the first context definition without modifying a schema of the database;

determining, for the first event, a third field value of the additional field;

performing a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregating information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

14. The computer readable storage medium of claim 8, wherein the first context definition is an entry in configuration data that is external to the data store.

15. A computing device comprising:

a memory; and

a processing device operatively coupled to the memory, the processing device to;

receive a query comprising a first field value and a time period;

perform a first search of a data store using the first field value to identify a first plurality of events having the time period and at least one field that comprises the first field value;

determine, for a first event of the plurality of events, a second field value of a second field that is specified in a first context definition, the second field having an assigned field type;

perform, without receipt of a second query, a second search of the data store using the additional field value to identify a second plurality of events having the time period and the additional field value;

aggregate information from the first plurality of events and the second plurality of events; and

generate a response to the query that comprises the information aggregated from the first plurality of events and the second plurality of events.

16. The computing device claim 15, wherein the processing device is further to:

determine a first source type associated with the first event, wherein the first source type is based on a source of the first event; and

determine that the first source type is associated with the first context definition, wherein the first context definition identifies a specified plurality of fields to use as link keys, each of the specified plurality of fields having assigned field types, and wherein the second field is included in the specified plurality of fields.

17. The computing device of claim 15, wherein the processing device is further to:

determine that a second event of the plurality of events is associated with a second context definition;

determine, for the second event, a third field value of a third field that is specified in the second context definition;

perform a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregate information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

18. The computing device of claim 17, wherein the response comprises a first section comprising a first set of field values of a first set of fields specified in the first context definition and a second section comprising a second set of field values of a second set of fields specified in the second context definition.

19. The computing device of claim 15, wherein the processing device is further to:

determine that a second event of the first plurality of events is associated with the first context definition;

determine, for the second event, a third field value of the second field;

perform a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregate information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.

20. The computing device of claim 15, wherein the data store comprises a database, and wherein the processing device is further to:

add an additional field to the first context definition without modifying a schema of the database;

determine, for the first event, a third field value of the additional field;

perform a third search of the data store using the third field value to identify a third plurality of events having the time period and the third field value; and

aggregate information from the third plurality of events to the information from the first plurality of events and the second plurality of events, wherein the response further comprises the information from the third plurality of events.