Searching for associated events in log data

• US 9,171,037 B2
• Filed: 11/05/2012
• Issued: 10/27/2015
• Est. Priority Date: 10/02/2007
• Status: Active Grant

• Current Assignee: Cloud Software Group
• Original Assignee: TIBCO Software Incorporated (Cloud Software Group)
• Inventors: Galitsky, Boris, Botros, Sherif
• Primary Examiner(s): PYO, MONICA M
• Application Number:
  US13/668,847
• Publication Number:
  US 20130185286A1
• Time in Patent Office: 1,086 Days
• Field of Search: 707/713, 707/718, 707/720, 707/771, 707/785
• US Class Current: 1/1
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/332 Query formulation
• CPC Class Codes: G06F 16/3335 Syntactic pre-processing, e.g. stopword elimination, stemming

Abstract

To retrieve a sequence of associated events in log data, a request expression is parsed to retrieve types of dependencies between events which are searched, and the constraints (e.g., keywords) which characterize each event. Based on the parsing results, query components can be formed, expressing the constraints for individual events and interrelations (e.g., time spans) between events. A resultant span query comprising the query components can then be run against an index of events, which encodes a mutual location of associated events in storage.

Claims

1. A computer-implemented method, comprising:

receiving, by a processor, a query searching for associated events in log data, the associated events being multiple events that are related to one another by a common component;

parsing the received query, including;

locating a reserved term from the received query;

identifying an intermediate component of the received query, the intermediate component including a portion of the received query that is located after the reserved term; and

identifying a final component of the received query, the final component including a portion of the received query preceding the reserved term, wherein the reserved term comprises a term indicating that the intermediate component constrains a variable in the final component;

forming an intermediate query for the intermediate component, including constructing a first search term for the intermediate component, the first search term explicitly indicating one or more first keywords that appeared in the intermediate component of the received query;

performing the intermediate query, including determining, using the intermediate query, one or more second keywords, each second keyword satisfying the search term in the intermediate query;

forming a final query for the final component, including constructing a second search term for the final component, the second search term explicitly indicates the one or more second keywords resulted from performing the intermediate query;

merging a result of performing the intermediate query and a result of performing the final query; and

designating the merged results as the associated events in response to the received query,

wherein the method is performed by one or more computers.

2. The computer-implemented method of claim 1, wherein:

the common component includes at least one of a device or a user, and

the reserved term includes a reserved word for identifying the device or a reserved word for identifying the user.

3. The computer-implemented method of claim 2, wherein the reserved word for identifying the device is “

which” and

the reserved word for identifying the user is “

who”

.

4. The computer-implemented method of claim 1, wherein the result of the intermediate query is designated as a selection constraint in the final query.

5. The computer-implemented method of claim 1, wherein designating the merged results as the associated events comprises:

providing for display a user interface view that presents an individual event from the intermediate query along with a set of events from the final query associated with the individual event.

6. The computer-implemented method of claim 1, wherein each of the received query, the intermediate query, and the final query is written in a structured query language.

7. A system comprising:

a storage device operable for storing one or more events as log messages; and

a processor coupled to the storage device and configured to perform operations comprising;

receiving a query searching for associated events in log data, the associated events being multiple events that are related to one another by a common component;

parsing the received query, including;

locating a reserved term from the received query;

identifying an intermediate component of the received query, the intermediate component including a portion of the received query that is located after the reserved term; and

identifying a final component of the received query, the final component including a portion of the received query preceding the reserved term, wherein the reserved term comprises a term indicating that the intermediate component constrains a variable in the final component;

forming an intermediate query for the intermediate component, including constructing a first search term for the intermediate component, the first search term explicitly indicating one or more first keywords that appeared in the intermediate component of the received query;

performing the intermediate query, including determining, using the intermediate query, one or more second keywords, each second keyword satisfying the first search term in the intermediate query;

forming a final query for the final component, including constructing a second search term for the final component, the second search term explicitly indicates the one or more second keywords resulted from performing the intermediate query;

merging a result of performing the intermediate query and a result of performing the final query; and

designating the merged results as the associated events in response to the received query.

8. The system of claim 7, wherein:

the common component includes at least one of a device or a user, and

the reserved term includes a reserved word for identifying the device or a reserved word for identifying the user.

9. The system of claim 8, wherein the reserved word for identifying the device is “

which” and

the reserved word for identifying the user is “

who”

.

10. The system of claim 7, wherein the result of the intermediate query is designated as a selection constraint in the final query.

11. The system of claim 7, wherein designating the merged results as the associated events comprises:

providing for display a user interface view that presents an individual event from the intermediate query along with a set of events from the final query associated with the individual event.

12. The system of claim 7, wherein each of the received query, the intermediate query, and the final query is written in a structured query language.

13. A non-transitory storage device storing instructions operable to cause one or more computers to perform operations comprising:

receiving a query searching for associated events in log data, the associated events being multiple events that are related to one another by a common component;

parsing the received query, including;

locating a reserved term from the received query;

identifying a portion of the received query that is located after the reserved term;

identifying a final component of the received query, the final component including a portion of the received query preceding the reserved term, wherein the reserved term comprises a term indicating that the intermediate component constrains a variable in the final component;

forming an intermediate query for the intermediate component, including constructing a first search term for the intermediate component, the first search term explicitly indicating one or more first keywords that appeared in the intermediate component of the received query;

performing the intermediate query, including determining, using the intermediate query, one or more second keywords, each second keyword satisfying the first search term in the intermediate query;

forming a final query for the final component, including constructing a second search term for the final component, the second search term explicitly indicates the one or more second keywords resulted from performing the intermediate query;

merging a result of performing the intermediate query and a result of performing the final query; and

designating the merged results as the associated events in response to the received query.

14. The non-transitory storage device of claim 13, wherein:

the common component includes at least one of a device or a user, and

the reserved term includes a reserved word for identifying the device or a reserved word for identifying the user.

15. The non-transitory storage device of claim 14, wherein the reserved word for identifying the device is “

which” and

the reserved word for identifying the user is “

who”

.

16. The non-transitory storage device of claim 13, wherein the result of the intermediate query is designated as a selection constraint in the final query.

17. The non-transitory storage device of claim 13, wherein designating the merged results as the associated events comprises:

providing for display a user interface view that presents an individual event from the intermediate query along with a set of events from the final query associated with the individual event.

18. The non-transitory storage device of claim 13, wherein each of the received query, the intermediate query, and the final query is written in a structured query language.