Configuring rules for filtering events to be included in event streams

• US 10,701,191 B2
• Filed: 04/15/2019
• Issued: 06/30/2020
• Est. Priority Date: 10/30/2014
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Shcherbakov, Vladimir A., Dickey, Michael
• Primary Examiner(s): Zhao, Wei
• Application Number:
  US16/384,688
• Publication Number:
  US 20190245950A1
• Time in Patent Office: 442 Days
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 69/22 Parsing or analysis of headers

Abstract

The disclosed embodiments provide a system that processes network data. During operation, the system obtains, at a remote capture agent, a first protocol classification for a first packet flow captured by the remote capture agent. Next, the system uses configuration information associated with the first protocol classification to build a first event stream from the first packet flow at the remote capture agent, wherein the first event stream comprises time-series event data generated from network packets in the first packet flow based on the first protocol classification. The system then transmits the first event stream over a network for subsequent storage and processing of the first event stream by one or more components on the network.

Claims

1. A computer-implemented method performed by a configuration server coupled to a network, the method comprising:

receiving input defining a filter to be applied to timestamped events generated by one or more remote capture agents from network data monitored by the one or more remote capture agents, the filter used to identify a subset of the timestamped events to be included in one or more event streams generated by the one or more remote capture agents;

generating configuration data based at least in part on the input defining the filter; and

sending, over the network, the configuration data to the one or more remote capture agents, the configuration data causing the one or more remote capture agents to apply the filter to events generated by the one or more remote capture agents for inclusion in the one or more event streams.

2. The computer-implemented method of claim 1, further comprising:

obtaining, by the one or more remote capture agents, the configuration data from the configuration server; and

using the configuration data to configure generation of the one or more event streams.

3. The computer-implemented method of claim 1, wherein sending the configuration data to the one or more remote capture agents causes the one or more remote capture agent to configure generation of the timestamped events from the network data during runtime of the one or more remote capture agents.

4. The computer-implemented method of claim 1, wherein the input is first input, and wherein the method further comprises:

receiving second input indicating a modification to the filter;

generating updated configuration data based at least in part on the second input; and

sending the updated configuration data to the one or more remote capture agents.

5. The computer-implemented method of claim 1, wherein the configuration data instructs the one or more remote capture agents to send the one or more event streams to another component on the network for subsequent processing.

6. The computer-implemented method of claim 1, wherein the configuration data instructs the one or more remote capture agents to, in response to detecting encryption of the network data, decrypt the network data prior to generating the one or more event streams.

7. The computer-implemented method of claim 1, wherein the one or more remote capture agents generate the one or more event streams in part by:

identifying one or more event attributes defined in the configuration data;

extracting the one or more event attributes from network packets comprising the network data; and

including the one or more event attributes in the one or more event streams.

8. The computer-implemented method of claim 1, wherein the one or more remote capture agents generate the one or more event streams in part by:

identifying one or more event attributes defined in the configuration data;

extracting the one or more event attributes from network packets comprising the network data;

transforming, based on the configuration data, the one or more event attributes extracted from the network packets to obtain one or more transformed event attributes; and

including the one or more transformed event attributes in the one or more event streams.

9. The computer-implemented method of claim 1, wherein the input is received via a graphical user interface (GUI) including interface elements that enable input defining the filter.

10. The computer-implemented method of claim 1, wherein at least one of the one or more remote capture agents is installed in a cloud computing environment.

11. A configuration server, comprising:

a processor;

a non-transitory computer readable storage medium storing instructions which, when executed by the processor, cause the configuration server to;

receive input defining a filter to be applied to timestamped events generated by one or more remote capture agents from network data monitored by the one or more remote capture agents, the filter used to identify a subset of the timestamped events to be included in one or more event streams generated by the one or more remote capture agents;

generate configuration data based at least in part on the input defining the filter; and

send, over a network, the configuration data to the one or more remote capture agents, the configuration data causing the one or more remote capture agents to apply the filter to events generated by the one or more remote capture agents for inclusion in the one or more event streams.

12. The configuration server of claim 11, wherein the instructions, when executed by the processor, further cause the configuration server to:

cause the one or more remote capture agents to obtain the configuration data from the configuration server; and

cause the one or more remote capture agents to use the configuration data to configure generation of the one or more event streams.

13. The configuration server of claim 11, wherein sending the configuration data to the one or more remote capture agents causes the one or more remote capture agent to configure generation of the timestamped events from the network data during runtime of the one or more remote capture agents.

14. The configuration server of claim 11, wherein the input is first input, and wherein the instructions, when executed by the processor, further cause the configuration to:

receive second input indicating a modification to the filter;

generate updated configuration data based at least in part on the second input; and

send the updated configuration data to the one or more remote capture agents.

15. The configuration server of claim 11, wherein the configuration data instructs the one or more remote capture agents to send the one or more event streams to another component on the network for subsequent processing.

16. The configuration server of claim 11, wherein the configuration data instructs the one or more remote capture agents to, in response to detecting encryption of the network data, decrypt the network data prior to generating the one or more event streams.

17. The configuration server of claim 11, wherein the one or more remote capture agents generate the one or more event streams in part by:

identifying one or more event attributes defined in the configuration data;

extracting the one or more event attributes from network packets comprising the network data; and

including the one or more event attributes in the one or more event streams.

18. The configuration server of claim 11, wherein the one or more remote capture agents generate the one or more event streams in part by:

identify one or more event attributes defined in the configuration data;

extract the one or more event attributes from network packets comprising the network data;

transform, based on the configuration data, the one or more event attributes extracted from the network packets to obtain one or more transformed event attributes; and

include the one or more transformed event attributes in the one or more event streams.

19. The configuration server of claim 11, wherein the input is received via a graphical user interface (GUI) including interface elements that enable input defining the filter.

20. The configuration server of claim 11, wherein at least one of the one or more remote capture agents is installed in a cloud computing environment.

21. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform operations comprising:

receiving, by a configuration server, input defining a filter to be applied to timestamped events generated by one or more remote capture agents from network data monitored by the one or more remote capture agents, the filter used to identify a subset of the timestamped events to be included in one or more event streams generated by the one or more remote capture agents;

generating configuration data based at least in part on the input defining the filter; and

sending, over a network, the configuration data to the one or more remote capture agents, the configuration data causing the one or more remote capture agents to apply the filter to events generated by the one or more remote capture agents for inclusion in the one or more event streams.

22. The non-transitory computer-readable storage medium of claim 21, wherein the instructions, when executed, further cause the configuration server to:

cause the one or more remote capture agents to obtain the configuration data from the configuration server; and

cause the one or more remote capture agents to use the configuration data to configure generation of the one or more event streams.

23. The non-transitory computer-readable storage medium of claim 21, wherein sending the configuration data to the one or more remote capture agents causes the one or more remote capture agent to configure generation of the timestamped events from the network data during runtime of the one or more remote capture agents.

24. The non-transitory computer-readable storage medium of claim 21, wherein the input is first input, and wherein the instructions, when executed, further cause the configuration server to:

receive second input indicating a modification to the filter;

generate updated configuration data based at least in part on the second input; and

send the updated configuration data to the one or more remote capture agents.

25. The non-transitory computer-readable storage medium of claim 21, wherein the configuration data instructs the one or more remote capture agents to send the one or more event streams to another component on the network for subsequent processing.

26. The non-transitory computer-readable storage medium of claim 21, wherein the configuration data instructs the one or more remote capture agents to, in response to detecting encryption of the network data, decrypt the network data prior to generating the one or more event streams.

27. The non-transitory computer-readable storage medium of claim 21, wherein the one or more remote capture agents generate the one or more event streams in part by:

identifying one or more event attributes defined in the configuration data;

extracting the one or more event attributes from network packets comprising the network data; and

including the one or more event attributes in the one or more event streams.

28. The non-transitory computer-readable storage medium of claim 21, wherein the one or more remote capture agents generate the one or more event streams in part by:

identifying one or more event attributes defined in the configuration data;

extracting the one or more event attributes from network packets comprising the network data;

transforming, based on the configuration data, the one or more event attributes extracted from the network packets to obtain one or more transformed event attributes; and

including the one or more transformed event attributes in the one or more event streams.

29. The non-transitory computer-readable storage medium of claim 21, wherein the input is received via a graphical user interface (GUI) including interface elements that enable input defining the filter.

30. The non-transitory computer-readable storage medium of claim 21, wherein at least one of the one or more remote capture agents is installed in a cloud computing environment.