SUPPORT FOR USER DEFINED AGGREGATIONS IN A DATA STREAM MANAGEMENT SYSTEM

• US 20110270879A1
• Filed: 07/16/2011
• Published: 11/03/2011
• Est. Priority Date: 10/20/2007
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Jain, Namit, Srinivasan, Anand, Mishra, Shailendra Kumar
• Granted Patent: US 8,204,875 B2
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries

Abstract

A computer is programmed to accept a command to create a new aggregation defined by a user during execution of continuous queries on streams of data. The computer is further programmed to thereafter accept and process new continuous queries using the new aggregation, in a manner similar to built-in aggregations. The user typically writes a set of instructions to perform the new aggregation, and identifies in the command, a location of the set of instructions. In response to such a command, the computer creates metadata identifying the new aggregation. The metadata is used to instantiate one aggregation for each group of data in a current window, grouped by an attribute identified in a new query.

Claims

1. A data stream management system that processes a plurality of streams of data using a plurality of continuous queries, the data stream management system comprising:

a store comprising a plurality of tuples representing a relation, each tuple being time stamped;

a memory comprising a global plan currently being used in execution of the continuous queries on the plurality of streams;

means, coupled to said store, for modifying said global plan in said memory in response receipt of a new continuous query that uses a new aggregation defined by a user, by using an operator to invoke a first instance in a plurality of instances of the new aggregation based on a first value of an attribute of said data;

wherein said new aggregation groups data in said streams by said attribute; and

means, coupled to said means for modifying and to said store, for executing an instruction in the operator, to perform the new aggregation.

2. The data stream management system of claim 1 wherein:

a first count corresponds to said first value of said attribute;

said means for executing operates only on data of said first value of said attribute; and

said first count is changed only for data of said first value.

3. The data stream management system of claim 2 wherein:

said operator comprises a plurality of additional counts for a corresponding plurality of additional values of said attribute.

4. The data stream management system of claim 2 further comprising:

means for creating the first instance when the first count is changed from an initial value; and

means for deleting the first instance when the first count is changed to return to said initial value.

5. The data stream management system of claim 1 further comprising:

means for checking to ensure said new aggregation is identified in said new continuous query in a select list.

6. The data stream management system of claim 1 wherein:

the operator comprises a plurality of structures to hold a corresponding plurality of paths to said plurality of instances of the new aggregation.

7. The data stream management system of claim 1 wherein:

each instance in said plurality of instances of the new aggregation holds state information specific to a corresponding value of the attribute.

8. The data stream management system of claim 1 further comprising:

means for creating and deleting each instance in said plurality of instances in a just-in-time manner, when a window comprising data to be processed by said each instance becomes non-empty and empty respectively.

9. One or more non-transitory computer-readable storage media comprising software including instructions to process a plurality of streams of data, the instructions comprising:

instructions to store in a store a plurality of tuples representing a relation, each tuple being time stamped;

instructions to use a global plan in a memory to execute a plurality of continuous queries on the plurality of streams;

instructions responsive to receipt of a new continuous query that uses a new aggregation defined by a user, to modify in said memory said global plan currently being used in execution of the continuous queries, by using an operator to invoke a first instance in a plurality of instances of the new aggregation based on a first value of an attribute of said data;

wherein on execution of said instructions to modify, said new aggregation groups data in said streams by said attribute; and

instructions to perform the new aggregation when using said operator.

10. The one or more non-transitory computer-readable storage media of claim 9 wherein:

a first count corresponds to said first value of said attribute;

said instructions to perform operate only on data of said first value of said attribute; and

said first count is changed only for data of said first value.

11. The one or more non-transitory computer-readable storage media of claim 10 wherein:

said operator comprises a plurality of additional counts for a corresponding plurality of additional values of said attribute.

12. The one or more non-transitory computer-readable storage media of claim 10 further comprising:

instructions to create the first instance when the first count is changed from an initial value; and

instructions to delete the first instance when the first count is changed to return to said initial value.

13. The one or more non-transitory computer-readable storage media of claim 9 further comprising:

instructions to check to ensure said new aggregation is identified in said new continuous query in a select list.

14. The one or more non-transitory computer-readable storage media of claim 9 wherein:

the operator comprises a plurality of structures to hold a corresponding plurality of paths to said plurality of instances of the new aggregation.

15. The one or more non-transitory computer-readable storage media of claim 9 wherein:

each instance in said plurality of instances of the new aggregation holds state information specific to a corresponding value of the attribute.

16. The one or more non-transitory computer-readable storage media of claim 9 further comprising:

instructions to create and delete each instance in said plurality of instances in a just-in-time manner, when a window comprising data to be processed by said each instance becomes non-empty and empty respectively.

17. A method implemented in a computer of processing a plurality of streams of data, the method comprising:

storing in a store a plurality of tuples representing a relation, each tuple being time stamped;

using a global plan in a memory to execute a plurality of continuous queries on the plurality of streams;

in response receipt of a new continuous query that uses a new aggregation defined by a user, modifying in said memory said global plan currently being used in execution of the continuous queries, by using an operator to invoke a first instance in a plurality of instances of the new aggregation based on a first value of an attribute of said data;

wherein said new aggregation groups data in said streams by said attribute; and

executing at least one instruction in the operator, to perform the new aggregation.

18. The method of claim 17 wherein:

a first count corresponds to said first value of said attribute;

said executing of at least one instruction operates only on data of said first value of said attribute; and

said first count is changed only for data of said first value.

19. The method of claim 18 wherein:

said operator comprises a plurality of additional counts for a corresponding plurality of additional values of said attribute.

20. The method of claim 18 further comprising:

creating the first instance when the first count is changed from an initial value; and

deleting the first instance when the first count is changed to return to said initial value.