Adaptive control of data collection requests sent to external data sources

• US 10,007,710 B2
• Filed: 01/30/2016
• Issued: 06/26/2018
• Est. Priority Date: 09/21/2015
• Status: Active Grant

• Current Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Original Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Inventors: Chen, Ken, Tao, Gang, Ding, Lai Qiang, Hao, Junqing, Wang, Ting, Haddad, Elias, Bitincka, Dritan
• Primary Examiner(s): Ibrahim, Mohamed
• Application Number:
  US15/011,525
• Publication Number:
  US 20170085447A1
• Time in Patent Office: 878 Days
• Field of Search: None
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: H04L 43/024 by adaptive sampling
• CPC Class Codes: H04L 43/0817 by checking functioning

Abstract

Techniques and mechanisms are disclosed that enable a data collection system to adaptively control collection of data from one or more external data sources. At a high level, adaptively controlling collection of data from external data sources may include collecting performance information related to one or more data collection nodes and, in response to analyzing the collected performance information, adapting rates at which the data collection nodes send data collection requests to external data sources. Data collection performance information generally may include, but is not limited to, network traffic data, error messages generated by external data sources and/or data collection nodes, computing device performance information, and any other types of information related to a data collection node'"'"'s ability to collect data from external data sources.

Claims

1. A method, comprising:

receiving, by a controller node of a data intake and query system, data collection performance information from a plurality of data collection nodes, the data collection performance information related to data collection requests sent by the plurality of data collection nodes to at least one component of a plurality of components of a cloud computing service, the plurality of components of the cloud computing service including distributed computing resources used to generate virtual machine instances, the data collection requests including a request for machine data generated by one or more components of the cloud computing service;

generating, based on the data collection performance information, instructions for a data collection node of the plurality of data collection nodes to modify a rate at which the data collection node sends data collection requests to the at least one component of the cloud computing service; and

sending the instructions to the data collection node.

2. The method of claim 1, wherein the data collection requests are sent to an application programming interface (API) associated with the cloud computing service.

3. The method of claim 1, wherein the instructions instruct the data collection node to increase the rate at which the data collection node sends the data collection requests to the at least one component of the cloud computing service.

4. The method of claim 1, wherein the instructions instruct the data collection node to decrease the rate at which the data collection node sends the data collection requests to the at least one component of the cloud computing service.

5. The method of claim 1, wherein the data collection performance information includes information indicating one or more error messages received by the data collection node from the at least one component of the cloud computing service.

6. The method of claim 1, wherein the data collection performance information includes network traffic data.

7. The method of claim 1, wherein the data collection performance information includes CPU utilization information for the data collection node.

8. The method of claim 1, wherein the data collection performance information includes information indicating one or more attributes of the cloud computing service.

9. The method of claim 1, wherein the data collection performance information includes information indicating one or more data collection policies of the cloud computing service.

10. The method of claim 1, wherein the controller node receives particular data collection performance information from each data collection node of the plurality of data collection nodes.

11. The method of claim 1, wherein the controller node receives particular data collection performance information from each data collection node of the plurality of data collection nodes, and wherein the plurality of data collection nodes send data collection requests to a plurality of cloud computing services.

12. The method of claim 1, wherein the data collection node sends data collection requests to a plurality of cloud computing services.

13. The method of claim 1, further comprising:

subsequent to generating the instructions, receiving additional data collection performance information from the data collection node;

generating, based at least in part on the additional data collection performance information, updated instructions for the data collection node to modify the rate at which the data collection node sends data collection requests to the at least one component of the cloud computing service; and

sending the updated instructions to the data collection node.

14. The method of claim 1, wherein the data collection node sends data collection requests to at least one component of a plurality of cloud computing services, and wherein the controller node generates and sends to the data collection node separate instructions for each different cloud computing service of the plurality of cloud computing services.

15. The method of claim 1, further comprising:

generating, based on the data collection performance information, second instructions for a different data collection node of the plurality of data collection nodes to send data collection requests to the at least one component of the cloud computing service; and

sending the second instructions to the different data collection node.

16. The method of claim 1, wherein the data collection requests sent to the at least one component of the cloud computing service request data related to performance and/or security information for one or more of the distributed computing resources of the cloud computing service.

17. The method of claim 1, wherein data collected from the cloud computing service is sent to one or more indexers to be parsed and stored as event data reflecting events of one or more resources of an information technology environment.

18. The method of claim 1, wherein data collected from the cloud computing service comprises raw data sent to one or more indexers, and wherein the one or more indexers parse a plurality of timestamped events from the raw data, and wherein each timestamped event of the plurality of timestamped events includes a portion of the raw data.

19. The method of claim 1, further comprising:

receiving, by the data collection node, the instructions; and

sending, by the data collection node, data collection requests to the at least one component of the cloud computing service at a rate based on the instructions.

20. The method of claim 1, further comprising:

receiving, by the controller node, an indication that new data collection node has joined the plurality of data collection nodes; and

modifying instructions for one or more data collection nodes of the plurality of data collection nodes based on the indication that the new data collection node has joined the plurality of data collection nodes.

21. The method of claim 1, further comprising:

receiving, by the controller node, an indication that data collection node has left the plurality of data collection nodes; and

modifying instructions for one or more data collection nodes of the plurality of data collection nodes based on the indication that a data collection node has left the plurality of data collection nodes.

22. The method of claim 1, further comprising:

determining that the data collection performance information indicates that the cloud computing service has upgraded its data collection capabilities; and

in response to determining that the cloud computing service has upgraded its data collection capabilities, instructing the data collection node to increase the rate at which the data collection node sends the data collection requests to the at least one component of the cloud computing service.

23. The method of claim 1, wherein the data collection performance information indicates that the cloud computing service has upgraded its data collection capabilities based on one or more of:

one or more status messages, one or more version identifiers, an operational pattern of the distributed computing resources of the cloud computing service, and data retrieved from a website separate from the cloud computing service,

the method further comprising instructing the data collection node to increase the rate at which the data collection node sends the data collection requests to the at least one component of the cloud computing service in response to determining that the data collection performance information includes information indicating an upgrade to the cloud computing service.

24. Non-transitory storage media storing computer-executable instructions which, when executed by a controller node of a data intake and query system, cause the controller node to:

receive data collection performance information from a plurality of data collection nodes, the data collection performance information related to data collection requests sent by the plurality of data collection nodes to at least one component of a plurality of components of a cloud computing service, the plurality of components of the cloud computing service including distributed computing resources used to generate virtual machine instances, the data collection requests including a request for machine data generated by one or more components of the cloud computing service;

generate, based on the data collection performance information, instructions for a data collection node of the plurality of data collection nodes to modify a rate at which the data collection node sends data collection requests to the at least one component of the cloud computing service; and

send the instructions to the data collection node.

25. The non-transitory storage media of claim 24, wherein the instructions instruct the data collection node to increase the rate at which the data collection node sends the data collection requests to the at least one component of the cloud computing service or to decrease the rate at which the data collection node sends the data collection requests to the at least one component of the cloud computing service.

26. The non-transitory storage media of claim 24, wherein the data collection performance information includes information indicating one or more error messages received by the data collection node from the at least one component of the cloud computing service.

27. The non-transitory storage media of claim 24, wherein the data collection performance information includes CPU utilization information for the data collection node.

28. The non-transitory storage media of claim 24, wherein the data collection performance information includes information indicating one or more attributes of the cloud computing service.

29. The non-transitory storage media of claim 24, wherein the data collection performance information includes information indicating one or more data collection policies of the cloud computing service.

30. The non-transitory storage media of claim 24, wherein the controller node receives particular data collection performance information from each data collection node of the plurality of data collection nodes, and wherein the plurality of data collection nodes send data collection requests to a plurality of cloud computing services.

31. The non-transitory storage media of claim 24, wherein the computer-executable instructions, when executed by the controller node, further cause the controller node to:

receive additional data collection performance information from the data collection node subsequent to generation of the instructions;

generate, based at least in part on the additional data collection performance information, updated instructions for the data collection node to modify the rate at which the data collection node sends data collection requests to the at least one component of the cloud computing service; and

communicate the updated instructions to the data collection node.

32. The non-transitory storage media of claim 24, wherein the data collection node sends data collection requests to at least one component of a plurality of cloud computing services, and wherein the computer-executable instructions, when executed by the controller node, cause the controller node to generate and send to the data collection node separate instructions for each different cloud computing service of the plurality of cloud computing services.

33. The non-transitory storage media of claim 24, wherein the computer-executable instructions, when executed by the controller node, further cause the controller node to:

generate, based on the data collection performance information, second instructions for a different data collection node of the plurality of data collection nodes to send data collection requests to the at least one component of the cloud computing service; and

send the second instructions to the different data collection node.

34. The non-transitory storage media of claim 24, wherein the data collection requests sent to the at least one component of the cloud computing service request data related to performance and/or security information for one or more of the distributed computing resources of the cloud computing service.

35. The non-transitory storage media of claim 24, wherein data collected from the cloud computing service is sent to one or more indexers to be parsed and stored as event data reflecting events of one or more resources of an information technology environment.

36. The non-transitory storage media of claim 24, wherein data collected from the cloud computing service comprises raw data sent to one or more indexers, and wherein the one or more indexers parse a plurality of timestamped events from the raw data, and wherein each timestamped event of the plurality of timestamped events includes a portion of the raw data.

37. The non-transitory storage media of claim 24, wherein the data collection node is configured to:

receive the instructions, and send data collection requests to the at least one component of the cloud computing service at a rate based on the instructions.

38. The non-transitory storage media of claim 24, wherein the computer-executable instructions, when executed by the controller node, further cause the controller node to:

receive an indication that new data collection node has joined the plurality of data collection nodes; and

modify instructions for one or more data collection nodes of the plurality of data collection nodes based on the indication that the new data collection node has joined the plurality of data collection nodes.

39. The non-transitory storage media of claim 24, wherein the computer-executable instructions, when executed by the controller node, further cause the controller node to:

receive an indication that data collection node has left the plurality of data collection nodes; and

modify instructions for one or more data collection nodes of the plurality of data collection nodes based on the indication that a data collection node has left the plurality of data collection nodes.

40. The non-transitory storage media of claim 24, wherein the computer-executable instructions, when executed by the controller node, further cause the controller node to:

determine that the data collection performance information indicates that the cloud computing service has upgraded its data collection capabilities; and

in response to a determination that the cloud computing service has upgraded its data collection capabilities, instruct the data collection node to increase the rate at which the data collection node sends the data collection requests to the at least one component of the cloud computing service.

41. The non-transitory storage media of claim 24, wherein the data collection performance information indicates that the cloud computing service has upgraded its data collection capabilities based on one or more of:

one or more status messages, one or more version identifiers, an operational pattern of the distributed computing resources of the cloud computing service, and data retrieved from a website separate from the cloud computing service, wherein the computer-executable instructions, when executed by the controller node, further cause the controller node to instruct the data collection node to increase the rate at which the data collection node sends the data collection requests to the at least one component of the cloud computing service in response to the data collection performance information indicating an upgrade to the cloud computing service.

42. A controller node of a data intake and query system, the controller node comprising:

a memory; and

one or more processing devices coupled to the memory and configured to;

receive data collection performance information from a plurality of data collection nodes, the data collection performance information related to data collection requests sent by the plurality of data collection nodes to at least one component of a plurality of components of a cloud computing service, the plurality of components of the cloud computing service including distributed computing resources used to generate virtual machine instances, the data collection requests including a request for machine data generated by one or more components of the cloud computing service;

generate based on the data collection performance information, instructions for a data collection node of the plurality of data collection nodes to modify a rate at which the data collection node sends data collection requests to the at least one component of the cloud computing service; and

send the instructions to the data collection node.

43. The controller node of claim 42, wherein the instructions instruct the data collection node to increase the rate at which the data collection node sends the data collection requests to the at least one component of the cloud computing service or to decrease the rate at which the data collection node sends the data collection requests to the at least one component of the cloud computing service.

44. The controller node of claim 42, wherein the data collection performance information includes information indicating one or more error messages received by the data collection node from the at least one component of the cloud computing service.

45. The controller node of claim 42, wherein the data collection performance information includes CPU utilization information for the data collection node.

46. The controller node of claim 42, wherein the data collection performance information includes information indicating one or more attributes of the cloud computing service.

47. The controller node of claim 42, wherein the data collection performance information includes information indicating one or more data collection policies of the cloud computing service.

48. The controller node of claim 42, wherein the controller node receives particular data collection performance information from each data collection node of the plurality of data collection nodes, and wherein the plurality of data collection nodes send data collection requests to a plurality of cloud computing services.

49. The controller node of claim 42, wherein the one or more processing devices are further configured to:

receive additional data collection performance information from the data collection node subsequent to generation of the instructions;

generate, based at least in part on the additional data collection performance information, updated instructions for the data collection node to modify the rate at which the data collection node sends data collection requests to the at least one component of the cloud computing service; and

communicate the updated instructions to the data collection node.

50. The controller node of claim 42, wherein the data collection node sends data collection requests to at least one component of a plurality of cloud computing services, and wherein the controller node generates and sends to the data collection node separate instructions for each different cloud computing service of the plurality of cloud computing services.

51. The controller node of claim 42, wherein the one or more processing devices are further configured to:

generate, based on the data collection performance information, second instructions for a different data collection node of the plurality of data collection nodes to send data collection requests to the at least one component of the cloud computing service; and

send the second instructions to the different data collection node.

52. The controller node of claim 42, wherein the data collection requests sent to the at least one component of the cloud computing service request data related to performance and/or security information for one or more of the distributed computing resources of the cloud computing service.

53. The controller node of claim 42, wherein data collected from the cloud computing service is sent to one or more indexers to be parsed and stored as event data reflecting events of one or more resources of an information technology environment.

54. The controller node of claim 42, wherein data collected from the cloud computing service comprises raw data sent to one or more indexers, and wherein the one or more indexers parse a plurality of timestamped events from the raw data, and wherein each timestamped event of the plurality of timestamped events includes a portion of the raw data.

55. The controller node of claim 42, wherein the data collection node is configured to:

receive the instructions, and send data collection requests to the at least one component of the cloud computing service at a rate based on the instructions.

56. The controller node of claim 42, wherein the one or more processing devices are further configured to:

receive an indication that new data collection node has joined the plurality of data collection nodes; and

modify instructions for one or more data collection nodes of the plurality of data collection nodes based on the indication that the new data collection node has joined the plurality of data collection nodes.

57. The controller node of claim 42, wherein the one or more processing devices are further configured to:

receive an indication that data collection node has left the plurality of data collection nodes; and

modify instructions for one or more data collection nodes of the plurality of data collection nodes based on the indication that a data collection node has left the plurality of data collection nodes.

58. The controller node of claim 42, wherein the one or more processing devices are further configured to:

determine that the data collection performance information indicates that the cloud computing service has upgraded its data collection capabilities; and

in response to a determination that the cloud computing service has upgraded its data collection capabilities, instruct the data collection node to increase the rate at which the data collection node sends the data collection requests to the at least one component of the cloud computing service.

59. The controller node of claim 42, wherein the data collection performance information indicates that the cloud computing service has upgraded its data collection capabilities based on one or more of:

one or more status messages, one or more version identifiers, an operational pattern of the distributed computing resources of the cloud computing service, and data retrieved from a website separate from the cloud computing service, wherein the one or more processing devices are further configured to instruct the data collection node to increase the rate at which the data collection node sends the data collection requests to the at least one component of the cloud computing service in response to the data collection performance information indicating an upgrade to the cloud computing service.