IDENTIFYING NOTABLE EVENTS BASED ON EXECUTION OF CORRELATION SEARCHES

• US 20170142143A1
• Filed: 01/31/2017
• Published: 05/18/2017
• Est. Priority Date: 12/19/2013
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Seward, Mark, Coates, John Robert
• Granted Patent: US 11,196,756 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/212 with details for data modelling support
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

Systems and methods are provided for identifying network addresses and/or IDs of a deduplicated list among network data, machine data, and/or events derived from network data and/or machine data, and for identifying notable events by searching for the presence of network addresses and/or network IDs that are deduplicated across lists received from multiple external sources. One method includes receiving a plurality of lists of network locations, wherein each list is received from over a network, wherein each of the network locations includes a domain name or an IP address, and wherein at least two of the plurality of lists each include a same network location; aggregating the plurality of lists of network locations into a deduplicated list of unique network locations; and searching network data or machine data for a network location included in the deduplicated list of unique network locations.

Claims

1. A computer-implemented method, comprising:

obtaining network data generated by at least one component in an IT environment;

generating event data based on the network data;

receiving input indicating search criteria which, when executed as part of a search query, identify instances of notable events, wherein a notable event comprises one or more events of the event data satisfying the search criteria; and

executing a search query including the search criteria to identify at least one notable event.

2. The computer-implemented method of claim 1, further comprising:

receiving machine data generated by at least one component in the IT environment;

wherein generating the event data further includes generating event data based on the machine data; and

wherein executing the search query including the search criteria includes identifying at least one event generated based on the network data and at least one event generated based on the machine data.

3. The computer-implemented method of claim 1, wherein generating the event data based on the network data comprises:

determining a time stamp for each event of the event data;

associating the time stamp with each event of the event data; and

storing each event of the event data and the associated time stamp in a field-searchable data store.

4. The computer-implemented method of claim 1, wherein executing the query further comprises:

using a late-binding schema to extract values from events of the event data, wherein the late-binding schema is a schema applied to events during execution of the search query; and

using the extracted values to identify the at least one notable event.

5. The computer-implemented method of claim 1, further causing display, on a graphical interface, of an indication of the at least one notable event.

6. The computer-implemented method of claim 1, wherein the at least one of the component in the IT environment includes one or more of:

a web server, an application server, a host, a database, a firewall, a router, an operating system, a software application, a mobile device, and a sensor.

7. The computer-implemented method of claim 1, wherein the search criteria include a time range, and wherein the at least one notable event is associated one or more events having a time stamp in the time range.

8. The computer-implemented method of claim 1, further comprising:

receiving machine data generated by at least one component in the IT environment;

wherein generating the event data further includes generating event data based on the machine data; and

wherein the machine data includes one or more of;

an operating system log, an application server log, a web server log, a firewall log, a software application log, and an activity log.

9. The computer-implemented method of claim 1, wherein executing the search query includes a search head sending the search query to an indexer which searches an associated field-searchable data store for events responsive to the query.

10. The computer-implemented method of claim 1, further comprising, in response to executing the search query including the search criteria to identify the at least one notable event, generating an alert.

11. A non-transitory computer-readable storage medium storing instructions which, when executed by one or more processors, cause performance of operations, comprising:

obtaining network data generated by at least one component in an IT environment;

generating event data based on the network data;

receiving input indicating search criteria which, when executed as part of a search query, identify instances of notable events, wherein a notable event comprises one or more events of the event data satisfying the search criteria; and

executing a search query including the search criteria to identify at least one notable event.

12. The non-transitory computer-readable storage medium of claim 11, further comprising:

receiving machine data generated by at least one component in the IT environment;

wherein generating the event data further includes generating event data based on the machine data; and

wherein executing the search query including the search criteria includes identifying at least one event generated based on the network data and at least one event generated based on the machine data.

13. The non-transitory computer-readable storage medium of claim 11, wherein generating the event data based on the network data comprises:

determining a time stamp for each event of the event data;

associating the time stamp with each event of the event data; and

storing each event of the event data and the associated time stamp in a field-searchable data store.

14. The non-transitory computer-readable storage medium of claim 11, wherein executing the query further comprises:

using a late-binding schema to extract values from events of the event data, wherein the late-binding schema is a schema applied to events during execution of the search query; and

using the extracted values to identify the at least one notable event.

15. The non-transitory computer-readable storage medium of claim 11, further causing display, on a graphical interface, of an indication of the at least one notable event.

16. The non-transitory computer-readable storage medium of claim 11, wherein the at least one of the component in the IT environment includes one or more of:

a web server, an application server, a host, a database, a firewall, a router, an operating system, a software application, a mobile device, and a sensor.

17. The non-transitory computer-readable storage medium of claim 11, wherein the search criteria include a time range, and wherein the at least one notable event is associated one or more events having a time stamp in the time range.

18. The non-transitory computer-readable storage medium of claim 11, further comprising:

receiving machine data generated by at least one component in the IT environment;

wherein generating the event data further includes generating event data based on the machine data; and

wherein the machine data includes one or more of;

an operating system log, an application server log, a web server log, a firewall log, a software application log, and an activity log.

19. The non-transitory computer-readable storage medium of claim 11, wherein executing the search query includes a search head sending the search query to an indexer which searches an associated field-searchable data store for events responsive to the query.

20. The non-transitory computer-readable storage medium of claim 11, further comprising, in response to executing the search query including the search criteria to identify the at least one notable event, generating an alert.

21. An apparatus, comprising:

one or more processors;

a non-transitory computer-readable storage medium coupled to the one or more processors, the computer-readable storage medium storing instructions which, when executed by the one or more processors, causes the apparatus to;

obtain network data generated by at least one component in an IT environment;

generate event data based on the network data;

receive input indicating search criteria which, when executed as part of a search query, identify instances of notable events, wherein a notable event comprises one or more events of the event data satisfying the search criteria; and

execute a search query including the search criteria to identify at least one notable event.

22. The apparatus of claim 21, further comprising:

receiving machine data generated by at least one component in the IT environment;

wherein generating the event data further includes generating event data based on the machine data; and

wherein executing the search query including the search criteria includes identifying at least one event generated based on the network data and at least one event generated based on the machine data.

23. The apparatus of claim 21, wherein generating the event data based on the network data comprises:

determining a time stamp for each event of the event data;

associating the time stamp with each event of the event data; and

storing each event of the event data and the associated time stamp in a field-searchable data store.

24. The apparatus of claim 21, wherein executing the query further comprises:

using a late-binding schema to extract values from events of the event data, wherein the late-binding schema is a schema applied to events during execution of the search query; and

using the extracted values to identify the at least one notable event.

25. The apparatus of claim 21, further causing display, on a graphical interface, of an indication of the at least one notable event.

26. The apparatus of claim 21, wherein the at least one of the component in the IT environment includes one or more of:

a web server, an application server, a host, a database, a firewall, a router, an operating system, a software application, a mobile device, and a sensor.

27. The apparatus of claim 21, wherein the search criteria include a time range, and wherein the at least one notable event is associated one or more events having a time stamp in the time range.

28. The apparatus of claim 21, further comprising:

receiving machine data generated by at least one component in the IT environment;

wherein generating the event data further includes generating event data based on the machine data; and

wherein the machine data includes one or more of;

an operating system log, an application server log, a web server log, a firewall log, a software application log, and an activity log.

29. The apparatus of claim 21, wherein executing the search query includes a search head sending the search query to an indexer which searches an associated field-searchable data store for events responsive to the query.

30. The apparatus of claim 21, further comprising, in response to executing the search query including the search criteria to identify the at least one notable event, generating an alert.