Technique for context updating and query processing for evaluating qualitative search terms

• US 10,095,741 B2
• Filed: 06/03/2015
• Issued: 10/09/2018
• Est. Priority Date: 07/31/2014
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Cormier, Michael E., Thackrey, William E., Cox, Earl D.
• Primary Examiner(s): Chbouki, Tarek
• Application Number:
  US14/729,964
• Publication Number:
  US 20160034534A1
• Time in Patent Office: 1,224 Days
• CPC Class Codes: G06F 16/24537 of operators
• CPC Class Codes: G06F 16/24545 Selectivity estimation or determination
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/2468 Fuzzy queries
• CPC Class Codes: G06F 16/36 Creation of semantic tools, e.g. ontology or thesauri
• CPC Class Codes: G06F 16/90335 Query processing

Abstract

The disclosed embodiments relate to a system that updates a context that facilitates evaluating qualitative search terms for an attribute during query processing. During operation, the system extracts a value for the attribute from each data item in a set of data items. Next, the system updates the context based on the extracted attribute values, wherein the context includes a concept-mapping for one or more qualitative search terms applied to the attribute, and wherein each concept-mapping associates a given attribute value with a numerical compatibility index that indicates a compatibility between the given attribute value and a corresponding qualitative search term.

Claims

1. A computer-implemented method for managing a concept-mapping function that facilitates evaluating a qualitative search term applied to an attribute during query processing, the method comprising:

extracting a numerical value for the attribute from each data item in a set of data items;

updating the concept-mapping function based on the extracted numerical attribute values,

wherein the concept-mapping function associates a given numerical value for the attribute with a numerical compatibility index that indicates a compatibility between the given numerical value and the qualitative search term as applied to the attribute,

wherein the qualitative search term comprises one or more words that is a qualitative measurement of the numerical attribute value;

wherein updating the concept-mapping function comprises,

determining boundaries for the concept-mapping function based on the extracted numerical attribute values, the boundaries corresponding to a start point and an end point of a range of the extracted numerical attribute values associated with the concept mapping function, and

within the determined boundaries, using a pre-specified distribution to update the concept-mapping function;

receiving a query on a set of stored events, the query comprising the qualitative search term;

for at least a subset of events in the set of stored events;

applying the concept-mapping function to a numerical attribute value in the event to obtain the numerical compatibility index for the event and qualitative search term, and

determining whether to include the event in a set of query results based on the numerical compatibility index; and

responding to the query using the set of query results.

2. The computer-implemented method of claim 1, wherein updating the concept-mapping function includes using a pre-specified distribution obtained from a domain expert or a user.

3. The computer-implemented method of claim 1, wherein updating the concept-mapping function additionally includes using information obtained from a current version of the concept-mapping function.

4. The computer-implemented method of claim 1,

wherein the concept-mapping function belongs to a set of concept-mapping functions for a set of qualitative search terms applied to the attribute;

wherein the set of concept-mapping functions define a context for the attribute; and

wherein updating the concept-mapping function additionally includes updating the set of concept-mapping functions in the context.

5. The computer-implemented method of claim 1,

wherein the concept-mapping function belongs to a set of concept-mapping functions for a set of qualitative search terms applied to the attribute;

wherein the set of concept-mapping functions define a context for the attribute;

wherein a context for an attribute “

arrival time”

includes concept-mapping functions for a set of qualitative search terms, including “

early,”

“

on time,” and

“

late”

;

wherein a context for an attribute “

height”

includes concept-mapping functions for a set of qualitative search terms, including “

short” and

“

tall”

;

wherein a context for an attribute “

response time”

includes concept-mapping functions for a set of qualitative search terms, including “

slow” and

“

fast”

; and

wherein a context for an attribute “

speed”

includes concept-mapping functions for a set of qualitative search terms, including “

fast” and

“

slow”

.

6. The computer-implemented method of claim 1, further comprising storing the concept-mapping function as an array of numerical values that maps numerical attribute values to corresponding numerical compatibility indices.

7. The computer-implemented method of claim 1, further comprising storing the concept-mapping function as a representation of mathematical function that maps numerical attribute values to corresponding numerical compatibility indices.

8. The computer-implemented method of claim 1, wherein determining the boundaries for the concept-mapping function includes determining a minimum numerical attribute value and a maximum numerical attribute value in a range defined by the extracted numerical attribute values.

9. The computer-implemented method of claim 1, wherein determining the boundaries for the concept-mapping function includes:

computing an average numerical attribute value and a standard deviation based for the extracted numerical attribute values; and

using the average numerical attribute value and the standard deviation to determine the boundaries for the concept-mapping.

10. The computer-implemented method of claim 1, wherein determining the boundaries for the concept-mapping function includes:

computing a mean numerical attribute value and a standard deviation based for the extracted numerical attribute values; and

using the mean numerical attribute value and the standard deviation to determine the boundaries for the concept-mapping.

11. The computer-implemented method of claim 1, wherein determining the boundaries for the concept-mapping function includes:

obtaining a user-specified center value for a context associated with the concept-mapping function;

computing a standard deviation based on the extracted numerical attribute values; and

determining boundaries for the concept-mapping function based on the user-specified center value and the standard deviation.

12. The computer-implemented method of claim 1, wherein instructions for updating the concept-mapping function are stored in a context associated with the concept-mapping function.

31. The computer-implemented method of claim 1, wherein the concept-mapping function belongs to a set of concept-mapping functions for a context of the attribute, and wherein the set of concept-mapping functions comprise overlapping ranges.

13. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for managing a concept-mapping function that facilitates evaluating a qualitative search term applied to an attribute during query processing, the method comprising:

extracting a numerical value for the attribute from each data item in a set of data items;

updating the concept-mapping function based on the extracted numerical attribute values,

wherein the concept-mapping function associates a given numerical value for the attribute with a numerical compatibility index that indicates a compatibility between the given numerical value and the qualitative search term as applied to the attribute,

wherein the qualitative search term comprises one or more words that is a qualitative measurement of the numerical attribute value;

wherein updating the concept-mapping function comprises,

determining boundaries for the concept-mapping function based the extracted numerical attribute values, the boundaries corresponding to a start point and an end point of a range of the extracted numerical attribute values associated with the concept mapping function, and

within the determined boundaries, using a pre-specified distribution to update the concept-mapping function;

receiving a query on a set of stored events, the query comprising the qualitative search term;

for at least a subset of events in the set of stored events;

applying the concept-mapping function to a numerical attribute value in the event to obtain the numerical compatibility index for the event and qualitative search term, and

determining whether to include the event in a set of query results based on the numerical compatibility index; and

responding to the query using the set of query results.

14. The non-transitory computer-readable storage medium of claim 13, wherein updating the concept-mapping function includes using a pre-specified distribution obtained from a domain expert or a user.

15. The non-transitory computer-readable storage medium of claim 13, wherein updating the concept-mapping function additionally includes using information obtained from a current version of the concept-mapping function.

16. The non-transitory computer-readable storage medium of claim 13,

wherein the concept-mapping function belongs to a set of concept-mapping functions for a set of qualitative search terms applied to the attribute;

wherein the set of concept-mapping functions define a context for the attribute; and

wherein updating the concept-mapping function additionally includes updating the set of concept-mapping functions in the context.

17. The non-transitory computer-readable storage medium of claim 13,

wherein the concept-mapping function belongs to a set of concept-mapping functions for a set of qualitative search terms applied to the attribute;

wherein the set of concept-mapping functions define a context for the attribute;

wherein a context for an attribute “

arrival time”

includes concept-mapping functions for a set of qualitative search terms, including “

early,”

“

on time,” and

“

late”

;

wherein a context for an attribute “

height”

includes concept-mapping functions for a set of qualitative search terms, including “

short” and

“

tall”

;

wherein a context for an attribute “

response time”

includes concept-mapping functions for a set of qualitative search terms, including “

slow” and

“

fast”

; and

wherein a context for an attribute “

speed”

includes concept-mapping functions for a set of qualitative search terms, including “

fast” and

“

slow”

.

18. The non-transitory computer-readable storage medium of claim 13, wherein the method further comprises storing the concept-mapping function as an array of numerical values that maps numerical attribute values to corresponding numerical compatibility indices.

19. The non-transitory computer-readable storage medium of claim 13, wherein the method further comprises storing the concept-mapping function as a representation of mathematical function that maps numerical attribute values to corresponding numerical compatibility indices.

20. The non-transitory computer-readable storage medium of claim 13, wherein determining the boundaries for the concept-mapping function includes determining a minimum numerical attribute value and a maximum numerical attribute value in a range defined by the extracted numerical attribute values.

21. The non-transitory computer-readable storage medium of claim 13, wherein determining the boundaries for the concept-mapping function includes:

computing an average numerical attribute value and a standard deviation based for the extracted numerical attribute values; and

using the average numerical attribute value and the standard deviation to determine the boundaries for the concept-mapping.

22. The non-transitory computer-readable storage medium of claim 13, wherein determining the boundaries for the concept-mapping function includes:

computing a mean numerical attribute value and a standard deviation based for the extracted numerical attribute values; and

using the mean numerical attribute value and the standard deviation to determine the boundaries for the concept-mapping.

23. The non-transitory computer-readable storage medium of claim 13, wherein determining the boundaries for the concept-mapping function includes:

obtaining a user-specified center value for a context associated with the concept-mapping function;

computing a standard deviation based on the extracted numerical attribute values; and

determining boundaries for the concept-mapping function based on the user-specified center value and the standard deviation.

24. The non-transitory computer-readable storage medium of claim 13, wherein instructions for updating the concept-mapping function are stored in a context associated with the concept-mapping function.

25. A system for managing a concept-mapping function that facilitates evaluating a qualitative search term applied to an attribute during query processing, comprising:

at least one processor and at least one associated memory; and

an updating mechanism that executes on the at least one processor and during operation;

extracts a numerical value for the attribute from each data item in a set of data items;

updates the concept-mapping function based on the extracted numerical attribute values,

wherein the concept-mapping function associates a given numerical value for the attribute with a numerical compatibility index that indicates a compatibility between the given numerical value and the qualitative search term as applied to the attribute,

wherein the qualitative search term comprises one or more words that is a qualitative measurement of the numerical attribute value;

wherein while updating the concept-mapping function, the updating mechanism,

determines boundaries for the concept-mapping function based the extracted numerical attribute values, the boundaries corresponding to a start point and an end point of a range of the extracted numerical attribute values associated with the concept mapping function, and

within the determined boundaries, uses a pre-specified distribution to update the concept-mapping function;

receive a query on a set of stored events, the query comprising the qualitative search term;

for at least a subset of events in the set of stored events;

apply the concept-mapping function to a numerical attribute value in the event to obtain the numerical compatibility index for the event and qualitative search term, and

determine whether to include the event in a set of query results based on the numerical compatibility index; and

respond to the query using the set of query results.

26. The system of claim 25, wherein while updating the concept-mapping function, the updating mechanism uses a pre-specified distribution obtained from a domain expert or a user.

27. The system of claim 25, wherein while updating the concept-mapping function, the updating mechanism additionally uses information obtained from a current version of the concept-mapping function.

28. The system of claim 25,

wherein the concept-mapping function belongs to a set of concept-mapping functions for a set of qualitative search terms applied to the attribute;

wherein the set of concept-mapping functions define a context for the attribute; and

wherein updating the concept-mapping function additionally includes updating the set of concept-mapping functions in the context.

29. The system of claim 25,

wherein the concept-mapping function belongs to a set of concept-mapping functions for a set of qualitative search terms applied to the attribute;

wherein the set of concept-mapping functions define a context for the attribute;

wherein a context for an attribute “

arrival time”

includes concept-mapping functions for a set of qualitative search terms, including “

early,”

“

on time,” and

“

late”

;

wherein a context for an attribute “

height”

includes concept-mapping functions for a set of qualitative search terms, including “

short” and

“

tall”

;

wherein a context for an attribute “

response time”

includes concept-mapping functions for a set of qualitative search terms, including “

slow” and

“

fast”

; and

wherein a context for an attribute “

speed”

includes concept-mapping functions for a set of qualitative search terms, including “

fast” and

“

slow”

.

30. The system of claim 25, wherein the updating mechanism additionally stores the concept-mapping function as an array of numerical values that maps numerical attribute values to corresponding numerical compatibility indices.