System and Method for Querying a Data Stream

• US 20130191370A1
• Filed: 10/11/2010
• Published: 07/25/2013
• Est. Priority Date: 10/11/2010
• Status: Abandoned Application

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Inventors: Chen, Qiming, Hsu, Meichun
• Application Number:
  US13/825,019
• Publication Number:
  US 20130191370A1
• Time in Patent Office: Days
• US Class Current: 707/718
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries

Abstract

There is provided a method (200) for querying a data stream. The method includes receiving a query plan based on a query specifying the data stream and a window. The method (200) further includes receiving one or more stream elements from the data stream during the window. Additionally, the method (200) includes applying the query to the one or more stream elements by passing the one or more stream elements from a scan operator at a leaf of the query plan to an upper layer of the query plan on a tuple-by-tuple basis. The method (200) also includes committing a result of the query based on the one or more stream elements.

Claims

1. A method (200) for querying a data stream, comprising:

receiving a query plan based on a query (528) specifying the data stream and a window;

receiving one or more stream elements from the data stream during the window;

applying the query (528) to the one or more stream elements by passing the one or more stream elements from a scan operator at a leaf of the query plan to an upper layer of the query plan on a tuple-by-tuple basis; and

committing a result of the query (528) based on the one or more stream elements.

2. The method (200) recited in claim 1, comprising:

providing the result to a client application (106); and

persisting the result to a database table.

3. The method (200) recited in claim 2, wherein the query (528) specifies one of:

an insert into operation; and

a select into operation.

4. The method (200) recited in claim 1, comprising:

initiating a transaction based on a user defined function specified in the query (528);

performing, during the transaction, a plurality of cycles corresponding to a plurality of windows, wherein the plurality of windows comprise the window, and wherein each of the cycles comprises;

receiving the one or more stream elements;

applying the query (528) to the one or more stream elements; and

committing the result.

5. The method (200) recited in claim 4, comprising performing a vacuum operation on obsolete data for the transaction periodically, wherein a period comprises a predetermined number of the plurality of cycles.

6. The method (208) recited in claim 1, comprising storing an intermediate result of the query (528) based on a user defined function.

7. The method (200) recited in claim 1, wherein the query (528) specifies a join of the data stream and at least one of the following:

a database table; and

another data stream.

8. A computer system (500) for querying a data stream, the computer system comprising a processor (512) configured to:

receive a query plan based on a query (528) specifying the data stream, a window, and a user defined function;

receive one or more stream elements from the data stream during the window;

apply the query (528) to the one or more stream elements by passing the one or more stream elements from a scan operator at a leaf of the query plan to an upper layer of the query plan on a tuple-by-tuple basis; and

commit a result of the query (528) based on the one or more stream elements.

9. The computer system (500) recited in claim 8, wherein the processor (512) is configured to:

provide the result to a client application 106); and

persist the result to a database table.

10. The computer system (500) recited in claim 9, wherein the query (528) specifies one of:

an insert into operation; and

a select into operation.

11. The computer system (500) recited in claim 8, wherein the processor (512) is configured to:

initiate a transaction based on the user defined function, wherein the user defined function is configured with an extended function call handle accessible to both the user defined function and a database management system execution engine, and wherein the execution engine and the user defined function are configured to interact to allocate initial memory for the transaction;

perform, during the transaction, a plurality of cycles corresponding to a plurality of windows, wherein the plurality of windows comprise the window, and wherein, during each of the cycles, the processor is configured to;

receive the one or more stream elements;

apply the query to the one or more stream elements; and

commit the result.

12. The computer system (500) recited in claim 11, comprising performing a vacuum operation on obsolete data for the transaction periodically, wherein a period comprises a predetermined number of the plurality of cycles.

13. The computer system (500) recited in claim 8, wherein the processor is configured to store an intermediate result of the query (528) based on a second user defined function.

14. The computer system (500) recited in claim 8, wherein the query (528) specifies a join of the data stream and at least one of the following:

a database table; and

another data stream.

15. A non-transitory, computer-readable medium (622) comprising machine-readable instructions executable by a processor (612) to query a data stream, the non-transitory, computer-readable medium (622) comprising:

computer-readable instructions (624) that, when executed by the processor (612), receive a query plan based on a query specifying an update operation, the data stream, a window, and a user defined function;

computer-readable instructions (626) that, when executed by the processor (612), receive one or more stream elements from the data stream during the window;

computer-readable instructions (628) that, when executed by the processor (612), initiate a transaction based on the user defined function;

computer-readable instructions (628) that, when executed by the processor (612), perform, during the transaction, a plurality of cycles corresponding to a plurality of windows, wherein the plurality of windows comprise the window, and wherein, during each of the cycles, the processor (612) is configured to;

receive the one or more stream elements;

apply the update operation to the one or more stream elements by passing the one or more stream elements from a scan operator at a leaf of the query plan to an upper layer of the query plan on a tuple-by-tuple basis; and

commit the result;

computer-readable instructions (630) that, when executed by the processor (612), provide the result to a client application; and

computer-readable instructions (630) that, when executed by the processor (612), persist the result to a database table.