SUPPORT FOR USER DEFINED FUNCTIONS IN A DATA STREAM MANAGEMENT SYSTEM

• US 20120041934A1
• Filed: 09/23/2011
• Published: 02/16/2012
• Est. Priority Date: 10/18/2007
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Srinivasan, Anand, Jain, Namit, Mishra, Shailendra Kumar
• Granted Patent: US 8,543,558 B2
• Time in Patent Office: Days
• US Class Current: 707/702
• CPC Class Codes: G06F 8/20 Software design

Abstract

A data stream management system (DSMS) is designed to support a new user-defined function, by creating and using at least two structures as follows. A first structure (“metadata entry”) is created in response to a command for creation of the new function, and maps a single instance of a class to the function'"'"'s name. A second structure is created with creation of an operator on receipt of each new continuous query that uses the new function. The second structure (“operator specific data structure”) contains a path to the newly-created instance, which path is obtained by looking up the first structure. Additional second structures are created on receipt of additional continuous queries which use the new function, but all second structures contain the same path. All continuous queries use the same instance. Repeated use of a single instance to compile and execute multiple queries eliminates repeated instantiation of the same function.

Claims

1. A method implemented in a computer of processing a plurality of incoming streams of data, the method comprising:

processing the plurality of incoming streams, to execute thereon a plurality of continuous queries based on an existing global plan;

during said processing, receiving from a user, a command to create a function and identification of a set of instructions to be executed to perform said function;

during said processing, creating in a memory of said computer, an instance of said set of instructions and a first structure comprising a reference to said instance;

during said processing, receiving a new continuous query to be executed using said function;

during said processing, based on said first structure, creating in said memory an operator to invoke the instance, the operator comprising a second structure, the second structure comprising a first field to hold said reference to said instance of said set of instructions, and at least one additional field corresponding to at least one argument of said function;

during said processing, at least one processor in said computer modifying the existing global plan by adding thereto said operator, thereby to obtain a modified global plan;

altering said processing, to cause execution of the new continuous query in addition to said plurality of continuous queries, based on the modified global plan, thereby to perform said function; and

based at least partially on processing of at least a portion of said data by executing the new continuous query, outputting from said computer an output stream of data.

2. The method of claim 1 further comprising:

receiving with said command, identification of a name and a type of said at least one argument.

3. The method of claim 2 wherein:

said identification comprises a name of a class containing said function, and a name of a package containing said class;

the command comprises a name clause containing said identification;

the set of instructions is identified by a predetermined label within said package; and

said function is deterministic.

4. The method of claim 1 wherein:

said first structure further comprises a name of said function;

said first structure further comprises a data type of said argument of said function.

5. The method of claim 1 further comprising:

checking if a data type of said argument received in said new continuous query, matches said a corresponding data type of said argument received in said command.

6. The method of claim 1 wherein:

said function is identified in said new continuous query, in a WHEREIN clause and/or SELECT list.

7. The method of claim 1 wherein:

said second structure further comprises a count of the arguments of said function;

said second structure further comprises a data type of a result to be returned by said function.

8. The method of claim 1 wherein:

said instance of said function is repeatedly invoked, on receipt of each tuple of said data, without re-creating said instance;

said instance of said function is also invoked, for an additional query, without re-creating said instance.

9. The method of claim 1 wherein:

said second structure further comprises a second field, and the method further comprises copying into said second field a result returned by said function.

25. The method of claim 1 wherein:

the data in each incoming stream comprises a series of tuples, each tuple ‘

s’

being time-stamped with a time ‘

t’

.

26. The method of claim 25 wherein:

the execution of the new continuous query comprises checking if a new tuple of data has been received.

10. A data stream management system that processes streams of data using a plurality of continuous queries, the data stream management system comprising:

a store encoded with a plurality of tuples representing a relation, each tuple being time stamped;

a memory encoded with a query plan currently being used in execution of the continuous queries;

means, coupled to said store, for modifying the global plan in said memory in response receipt of a new continuous query that defines a new function defined by a user, if the global plan comprises an instance of the new function, by using a second operator to invoke the instance of the new function to implement the new continuous query; and

means, coupled to said means for modifying and to said store, for executing an instruction in the second operator, to perform the new function.

11. The data stream management system of claim 10 wherein:

said new function is identified in said new continuous query, in a WHEREIN clause and/or SELECT list.

12. The data stream management system of claim 10 wherein:

said instance of said new function is repeatedly invoked, on receipt of each tuple of said data, without re-creating said instance;

said instance of said new function is also repeatedly invoked, for each additional query, without re-creating said instance.

13. One or more computer readable non-transitory storage media encoded with a plurality of instructions to be executed in a computer, said instructions comprising:

instructions to process a plurality of streams of data, to execute thereon a plurality of continuous queries based on a global plan;

instructions to be executed concurrently with execution of said instructions to process, to receive a command to create a function and identification of a set of additional instructions to be executed to perform said function;

instructions to be executed concurrently with execution of said instructions to process, to create in a memory of said computer, an instance of said set of additional instructions and a first structure comprising a reference to said instance;

instructions to be executed concurrently with execution of said instructions to process, to receive a new continuous query to be executed using said function;

instructions to be executed concurrently with execution of said instructions to process, and based on said first structure, to create in said memory an operator to invoke the instance, the operator comprising a second structure, the second structure comprising a first field to hold said reference to said instance, and at least one additional field corresponding to at least one argument of said function;

instructions to be executed concurrently with execution of said instructions to process, to modify the global plan by adding thereto said operator, thereby to obtain a modified plan;

instructions to alter said instructions to process, to cause execution of the new continuous query in addition to said plurality of continuous queries, based on the modified plan, thereby to perform said function; and

instructions to output from said computer, a stream generated based at least partially on processing of said data by executing the new continuous query.

14. The one or more computer readable non-transitory storage media of claim 13 further comprising:

instructions to receive with said command, identification of a name and a type of said at least one argument.

15. The one or more computer readable non-transitory storage media of claim 14 wherein:

said identification comprises a name of a class containing said function, and a name of a package containing said class;

the command comprises a name clause containing said identification;

the set of additional instructions is identified by a predetermined label within said package; and

said function is deterministic.

16. The one or more computer readable non-transitory storage media of claim 13 wherein:

said first structure further comprises a name of said function;

said first structure further comprises a data type of said argument of said function.

17. The one or more computer readable non-transitory storage media of claim 13 further comprising:

instructions to check if a data type of said argument received in said new continuous query, matches said a corresponding data type of said argument received in said command.

18. The one or more computer readable non-transitory storage media of claim 13 wherein:

said function is identified in said new continuous query, in a WHEREIN clause and/or SELECT list.

19. The one or more computer readable non-transitory storage media of claim 13 wherein:

said second structure further comprises a count of the arguments of said function;

said second structure further comprises a data type of a result to be returned by said function.

20. The one or more computer readable non-transitory storage media of claim 13 wherein:

said instance of said function is repeatedly invoked, on receipt of each tuple of said data, without re-creating said instance;

said instance of said function is also repeatedly invoked, for each additional query, without re-creating said instance.

21. A method implemented in a computer, the method comprising:

processing a plurality of incoming streams of time-stamped tuples, to execute thereon a plurality of existing queries based on an existing global plan in a memory of said computer;

during said processing, receiving a command to create a function and identification of a set of instructions to be executed to perform said function;

during said processing, creating in said memory, an instance of said set of instructions and a metadata structure comprising a pointer to said instance;

during said processing, receiving a new query to be executed using said function;

during said processing, based on said metadata structure in said memory, creating in said memory a new operator to invoke the instance, the new operator comprising a operator-specific structure, the operator-specific structure comprising a first field to hold said pointer to said instance of said set of instructions, and at least one additional field corresponding to at least one argument of said function;

during said processing, at least one processor in said computer modifying the existing global plan in said memory by adding thereto said new operator, thereby to obtain a modified global plan in said memory;

altering said processing to use the modified global plan, thereby to cause execution of the new query in addition to execution of said plurality of existing queries; and

outputting from said computer, an output stream of data generated based at least partially on processing of time-stamped tuples in said incoming streams by execution of the new query.

22. The method of claim 21 wherein:

the modified global plan causes execution of a plurality of new queries including said new query, each new query in said plurality of new queries using said new operator; and

the method further comprising, in response to an error in said function, automatically disabling execution of said plurality of new queries while continuing execution of at least said plurality of existing queries.

23. The method of claim 21 wherein:

each incoming stream is infinitely long.

24. The method of claim 21 wherein:

the data in each incoming stream arrives indefinitely.