Managing datasets produced by alert-triggering search queries

• US 10,585,851 B2
• Filed: 03/16/2017
• Issued: 03/10/2020
• Est. Priority Date: 07/09/2014
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Zhong, Qianjie, Wang, Ting, Lee, Margaret, Li, Dawei, Filippi, Nick, Ni, Yue, Yuan, Shiming
• Primary Examiner(s): Gofman, Alex
• Assistant Examiner(s): Mian, Umar
• Application Number:
  US15/461,076
• Publication Number:
  US 20170185607A1
• Time in Patent Office: 1,090 Days
• Field of Search: None
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0775 Content or structure details of the error report, e.g. specific table structure, specific error fields
• CPC Class Codes: G06F 16/125 characterised by the use of retention policies retention policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24565 Triggers; Constraints
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G08B 21/18 Status alarms G08B21/02 takes precedence

Abstract

An example method for managing datasets produced by alert-triggering search queries may include producing a dataset by executing a search query on a portion of data associated with a time window defined relative to a current time. The method may further include responsive to determining that a portion of the dataset satisfies a condition defining an alert, generating an instance of the alert. The method may further include associating, by a memory data structure, the instance of the alert with an identifier of the query and a parameter specifying a time of execution of the query that has triggered the instance. The method may further include receiving a request for the dataset portion. The method may further include substituting, in a definition of the time window, the current time with the time parameter. The method may further include reproducing the dataset portion by re-executing the query using the time window.

Claims

1. A method, comprising:

executing, by one or more processing devices, a search query on a portion of searchable data associated with a time window to produce a dataset comprising one or more results, wherein the time window is defined relative to a current time;

responsive to determining that a throttling condition is satisfied and at least a portion of the dataset satisfies a triggering condition defining an alert associated with the search query, generating an instance of the alert, wherein the triggering condition indicates whether a secondary conditional search performed on the dataset has produced at least one result, and wherein the throttling condition suppresses triggering alert instances for a certain period of time for one or more data items identified by respective name-value pairs in the dataset;

associating, using a memory data structure, the instance of the alert with an identifier of the search query and a time parameter specifying a time of execution of the search query that has triggered the instance of the alert;

receiving, from a client computing device, a request for the portion of the dataset;

determining that the portion of the dataset is not stored in a memory in a manner associating the portion of the dataset with the instance of the alert;

substituting, in a definition of the time window utilized by the search query, the current time with the time parameter; and

reproducing the portion of the dataset by re-executing the search query in view of the time window.

2. The method of claim 1, further comprising:

storing, in a memory associated with the computer system, the portion of the dataset and an association of the stored portion of the dataset with the instance of the alert.

3. The method of claim 1, further comprising:

implementing a file retention policy with respect to datasets stored in the memory, wherein the file retention policy requires deleting certain datasets responsive to evaluating corresponding file retention conditions.

4. The method of claim 1, further comprising:

transmitting a copy of the portion of the dataset to the client computing device.

5. The method of claim 1, further comprising associating the instance of the alert with an identifier of the triggering condition.

6. The method of claim 1, wherein the searchable data includes time-stamped events having portions of raw machine data.

7. The method of claim 1, further comprising:

transmitting, to the client computing device, a notification of the instance of the alert.

8. The method of claim 1, wherein the client computing device includes at least one of:

a desktop computing device or a mobile computing device.

9. The method of claim 1, wherein executing the search query on the portion of searchable data includes applying a late binding schema to the data, the late binding schema associated with one or more extraction rules defining one or more fields.

10. The method of claim 1, wherein the portion of searchable data includes machine data generated by at least one of a server, a database, an application, or a network.

11. The method of claim 1, wherein the search query is executed on a schedule that is associated with the alert.

12. The method of claim 1, wherein the triggering condition requires that the portion of the dataset includes at least a predetermined number of results.

13. The method of claim 1, further comprising:

preforming at least one action associated with the alert, wherein the action includes;

sending an electronic mail message, creating a Really Simple Syndication (RSS) feed, executing a script, or causing visual display of the instance of the alert.

14. A computer system comprising:

a memory; and

one or more processing devices, coupled to the memory, to;

execute a search query on a portion of searchable data associated with a time window to produce a dataset comprising one or more results, wherein the time window is defined relative to a current time;

responsive to determining that a throttling condition is satisfied and at least a portion of the dataset satisfies a triggering condition defining an alert associated with the search query, generate an instance of the alert, wherein the triggering condition indicates whether a secondary conditional search performed on the dataset has produced at least one result, and wherein the throttling condition suppresses triggering alert instances for a certain period of time for one or more data items identified by respective name-value pairs in the dataset;

associate, using a memory data structure, the instance of the alert with an identifier of the search query and a time parameter specifying a time of execution of the search query that has triggered the instance of the alert;

receive, from a client computing device, a request for the portion of the dataset;

determine that the portion of the dataset is not stored in the memory in a manner associating the portion of the dataset with the instance of the alert;

substitute, in a definition of the time window utilized by the search query, the current time with the time parameter; and

reproduce the portion of the dataset by re-executing the search query in view of the time window.

15. The computer system of claim 14, wherein the processing devices are further to:

store, in a memory associated with the computer system, the portion of the dataset and an association of the stored portion of the dataset with the instance of the alert.

16. The computer system of claim 14, wherein the processing devices are further to:

implement a file retention policy with respect to datasets stored in the memory, wherein the file retention policy requires deleting certain datasets responsive to evaluating corresponding file retention conditions.

17. The computer system of claim 14, wherein the processing devices are further to:

transmit a copy of the portion of the dataset to the client computing device.

18. The computer system of claim 14, wherein the processing devices are further to:

associate the instance of the alert with an identifier of the triggering condition.

19. The computer system of claim 14, wherein the searchable data includes time-stamped events having portions of raw machine data.

20. The computer system of claim 14, wherein the processing devices are further to:

transmit, to the client computing device, a notification of the instance of the alert.

21. The computer system of claim 14, wherein executing the search query on the portion of searchable data includes applying a late binding schema to the data, the late binding schema associated with one or more extraction rules defining one or more fields.

22. A computer-readable non-transitory storage medium comprising executable instructions that, when executed by a computer system, cause the computer system to perform operations comprising:

executing a search query on a portion of searchable data associated with a time window to produce a dataset comprising one or more results, wherein the time window is defined relative to a current time;

responsive to determining that a throttling condition is satisfied and at least a portion of the dataset satisfies a triggering condition defining an alert associated with the search query, generating an instance of the alert, wherein the triggering condition indicates whether a secondary conditional search performed on the dataset has produced at least one result, and wherein the throttling condition suppresses triggering alert instances for a certain period of time for one or more data items identified by respective name-value pairs in the dataset;

associating, using a memory data structure, the instance of the alert with an identifier of the search query and a time parameter specifying a time of execution of the search query that has triggered the instance of the alert;

receiving, from a client computing device, a request for the portion of the dataset;

determining that the portion of the dataset is not stored in the memory in a manner associating the portion of the dataset with the instance of the alert;

substituting, in a definition of the time window utilized by the search query, the current time with the time parameter; and

reproducing the portion of the dataset by re-executing the search query in view of the time window.

23. The computer-readable non-transitory storage medium of claim 22, further comprising executable instructions causing the computer system to:

store, in a memory associated with the computer system, the portion of the dataset and an association of the stored portion of the dataset with the instance of the alert.

24. The computer-readable non-transitory storage medium of claim 22, further comprising executable instructions causing the computer system to:

implement a file retention policy with respect to datasets stored in the memory, wherein the file retention policy requires deleting certain datasets responsive to evaluating corresponding file retention conditions.

25. The computer-readable non-transitory storage medium of claim 22, further comprising executable instructions causing the computer system to:

transmit the copy of the portion of the dataset to the client computing device.

26. The computer-readable non-transitory storage medium of claim 22, further comprising executable instructions causing the computer system to:

associate the instance of the alert with an identifier of the triggering condition.

27. The computer-readable non-transitory storage medium of claim 22, further comprising executable instructions causing the computer system to:

transmit, to the client computing device, a notification of the instance of the alert.

28. The computer-readable non-transitory storage medium of claim 22, wherein the searchable data includes time-stamped events having portions of raw machine data.