Extensible indexing framework using data cartridges

• US 9,305,057 B2
• Filed: 10/27/2010
• Issued: 04/05/2016
• Est. Priority Date: 12/28/2009
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: de Castro Alves, Alexandre, Deshmukh, Unmesh, Park, Hoyong, Mishra, Shailendra, Srinivasan, Anand
• Primary Examiner(s): Hasan, Syed
• Assistant Examiner(s): NGUYEN, THU N
• Application Number:
  US12/913,636
• Publication Number:
  US 20110161352A1
• Time in Patent Office: 1,987 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems

Abstract

A framework or infrastructure (extensibility framework/infrastructure) for extending the indexing capabilities of an event processing system. The capabilities of an event processing system may be extended to support indexing schemes, including related data types and operations, which are not natively supported by the event processing system. The extensibility is enabled by one or more plug-in extension components called data cartridges.

Claims

1. A system comprising:

a memory; and

a processor coupled to the memory, the processor configured to;

determine that a query includes a first function for whose evaluation an index is not natively provided by an event processing system that is to compile the query;

provide a data cartridge to the event processing system, the data cartridge providing an indexing scheme for evaluating the first function of the query; and

use, by the event processing system, the indexing scheme provided by the data cartridge to evaluate the first function of the query, wherein the processor is further configured to;

determine an occurrence of a link definition in the query, the link definition identifying the first function and the data cartridge;

determine a list of input arguments to the first function, wherein the list of input arguments comprises a collection;

determine whether the data cartridge provides an index for evaluation of the first function with the set of input arguments;

compile the query using information provided by the data cartridge to generate executable code, the executable code comprising an index instance generated using the information provided by the data cartridge, the index instance being generated for evaluation of the first function; and

execute the executable code, wherein the execution causes execution of the first function provided by the data cartridge related to the index instance.

2. The system of claim 1 wherein the processor is configured to:

communicate, to the data cartridge, information identifying the first function;

communicate, to the data cartridge, a position of the collection in the list of input arguments;

receive metadata information from the data cartridge, the metadata information identifying an index type to be used for evaluating the first function for the collection; and

generate the index instance based upon the index type.

3. The system of claim 2 wherein:

the metadata information comprises context information, the context information comprising information identifying the first function provided by the data cartridge; and

the processor is configured to communicate the information identifying the function to the data cartridge during execution of the executable code.

4. The system of claim 1 wherein the processor is configured to:

categorize, based on mappings that map various functions to various categories, the first function into one of a first category, a second category, or a third category, wherein categorization of the first function into the first category indicates that the system provides an index for evaluation of the first function, wherein categorization of the first function into the second category indicates that the data cartridge provides an index for evaluation of the first function, and wherein categorization of the first function into the third category indicates that no index is provided for evaluation of the first function;

wherein the data cartridge manages all metadata for the index instance;

wherein metadata for the index instance is not stored in memory of the event processing system.

5. The system of claim 1 wherein the processor is configured to:

use the index instance to evaluate a second function, wherein the collection is provided as an input argument for both the first function and the second function and the index instance is generated for the collection.

6. The system of claim 1 wherein:

the collection comprising a plurality of elements is provided as an input argument to the first function;

the first function provided by the data cartridge is an index scan function using the index instance; and

the processor is configured to receive a result set from execution of the first function provided by the data cartridge, the result set resulting from evaluation of the first function and comprising a set of items from the collection.

7. The system of claim 1 wherein:

the collection comprising a plurality of elements is provided as an input argument to the first function;

the first function provided by the data cartridge is an index scan function using the index instance;

the processor is configured to;

receive a result set from execution of the first function provided by the data cartridge, the result set comprising a set of items from the collection resulting from evaluation of the first function and also additional one or more items from the collection; and

determine the set of items resulting from evaluation of the first function from the result set.

8. The system of claim 1 wherein the processor is configured to:

execute the executable code upon receiving an event via an event stream, the event comprising data and a time stamp;

wherein the processor is configured to compile the query using the information provided by the data cartridge without using a user-defined function designed to interact specifically with a particular application that consumes results of execution of the executable code.

9. A non-transitory computer-readable storage medium storing a plurality of instructions for controlling a computer system, the plurality of instructions comprising:

instructions that cause the computer system to determine that a query includes a first function for whose evaluation an index is not natively provided by an event processing system that is to compile the query;

instructions that cause the computer system to provide a data cartridge to the event processing system, the data cartridge providing an indexing scheme for evaluating the first function of the query; and

instructions that cause the computer system to use the indexing scheme provided by the data cartridge to evaluate the first function of the query, wherein the instructions further comprise;

instructions that cause the computer system to determine an occurrence of a link definition in the query, the link definition identifying the first function and the data cartridge;

instructions that cause the computer system to determine a list of input arguments to the first function, wherein the list of input arguments comprises a collection;

instructions that cause the computer system to determine whether the data cartridge provides an index for evaluation of the first function with the set of input arguments;

instructions that cause the computer system to compile the query using information provided by the data cartridge to generate executable code, the executable code comprising an index instance generated using the information provided by the data cartridge, the index instance being generated for evaluation of the first function; and

instructions that cause the computer system to execute the executable code, wherein the execution causes execution of the first function provided by the data cartridge related to the index instance.

10. The non-transitory computer-readable storage medium of claim 9 wherein the instructions that cause the computer system to compile the query comprise:

instructions that cause the computer system to communicate information to the data cartridge identifying the first function;

instructions that cause the computer system to communicate to the data cartridge a position of the collection in the list of input arguments;

instructions that cause the computer system to receive metadata information from the data cartridge, the metadata information identifying an index type to be used for evaluating the first function for the collection; and

instructions that cause the computer system to generate the index instance based upon the index type.

11. The non-transitory computer-readable storage medium of claim 10 wherein:

the metadata information comprises context information, the context information comprising information identifying the first function provided by the data cartridge; and

instructions that cause the computer system to execute the execute code comprise instructions that cause the computer system to communicate the information identifying the first function provided by the data cartridge to the data cartridge during execution of the executable code.

12. The non-transitory computer-readable storage medium of claim 9 wherein the instructions that cause the computer system to compile the query comprise:

instructions that cause the computer system to categorize, based on mappings that map various functions to various categories, the first function into one of a first category, a second category, or a third category, wherein categorization of the first function into the first category indicates that the system provides an index for evaluation of the first function, wherein categorization of the first function into the second category indicates that the data cartridge provides an index for evaluation of the first function, and wherein categorization of the first function into the third category indicates that no index is provided for evaluation of the first function; and

instructions that cause the computer system to categorize, based on the mappings, a second function of the query into a different one of the first, second, and third categories than a category into which the first function was categorized.

13. The non-transitory computer-readable storage medium of claim 9 wherein the instructions that cause the computer system to execute the executable code comprise:

instructions that cause the computer system to use the index instance to evaluate a second function, wherein the collection is provided as an input argument for both the first function and the second function and the index instance is generated for the collection.

14. The non-transitory computer-readable storage medium of claim 9 wherein:

the collection comprising a plurality of elements is provided as an input argument to the first function;

the first function provided by the data cartridge is an index scan function using the index instance; and

the instructions that cause the computer system to execute the executable code comprise instructions that cause the computer system to receive a result set from execution of the first function provided by the data cartridge, the result set resulting from evaluation of the first function and comprising a set of items from the collection.

15. The non-transitory computer-readable storage medium of claim 9 wherein:

the collection comprising a plurality of elements is provided as an input argument to the first function;

the first function provided by the data cartridge is an index scan function using the index instance;

the instructions that cause the computer system to execute the executable code comprise instructions that cause the computer system to;

receive a result set from execution of the first function provided by the data cartridge, the result set comprising a set of items from the collection resulting from evaluation of the first function and also additional one or more items from the collection; and

determine the set of items resulting from evaluation of the first function from the result set.

16. The non-transitory computer-readable storage medium of claim 9 wherein the instructions that cause the computer system to execute the executable code comprise:

instructions that cause the computer system to execute the executable code upon receiving an event via an event stream, the event comprising data and a time stamp;

wherein compiling the query using the information provided by the data cartridge comprises compiling the query without using a user-defined function designed to interact specifically with a particular application that consumes results of execution of the executable code.

17. A method comprising:

determining that a query includes a first function for whose evaluation an index is not natively provided by an event processing system that is to compile the query;

providing a data cartridge to the event processing system, the data cartridge providing an indexing scheme for evaluating the first function of the query; and

using, by the event processing system, the indexing scheme provided by the data cartridge to evaluate the first function of the query, using the indexing scheme provided by the data cartridge further comprising;

determining an occurrence of a link definition in the query, the link definition identifying the first function and the data cartridge;

determining a list of input arguments to the first function, wherein the list of input arguments comprises a collection;

determining whether the data cartridge provides an index for evaluation of the first function with the set of input arguments;

compiling, by a computer system, the query using information provided by the data cartridge to generate executable code, the executable code comprising an index instance generated using the information provided by the data cartridge, the index instance being generated for evaluation of the first function; and

executing, by the computer system, the executable code, wherein the execution causes execution of the first function provided by the data cartridge related to the index instance.

18. The method of claim 17 wherein:

compiling the query comprises;

communicating information to the data cartridge identifying the first function and a position of the collection in a list of input arguments for the first function;

receiving metadata information from the data cartridge, the metadata information identifying an index type to be used for evaluating the first function for the collection and identifying the first function provided by the data cartridge;

generating the index instance based upon the index type; and

executing the executable code comprises;

communicating the information identifying the first function provided by the data cartridge to the data cartridge during execution of the executable code; and

executing the executable code upon receiving an event via an event stream, the event comprising data and a time stamp.