Systems and methods of block computation

• US 9,594,573 B2
• Filed: 01/14/2011
• Issued: 03/14/2017
• Est. Priority Date: 01/14/2011
• Status: Active Grant

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Inventors: Chen, Qiming, Hsu, Meichun
• Primary Examiner(s): CHANNAVAJJALA, SRIRAMA T
• Application Number:
  US13/007,487
• Publication Number:
  US 20120185460A1
• Time in Patent Office: 2,251 Days
• Field of Search: 707/718, 707/737, 707/762, 707/770, 707/966, 711/133, 711/134, 711/159, 711/160, 717/124, 717/130
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/284 Relational databases
• CPC Class Codes: G06F 16/832 Query formulation
• CPC Class Codes: G06F 9/448 Execution paradigms, e.g. implementations of programming paradigms

Abstract

Systems and methods of block computation are disclosed. An example of a method includes buffering a plurality of tuples in a build phase. The method also includes, after buffering a last of the plurality of tuples, applying a computation to all of the buffered tuples in a compute phase. The method also includes returning results of the computation one tuple at a time to a query processing pipeline in a stream-out phase.

Claims

1. A method of block computation, the method implemented by program code stored on a non-transient computer-readable medium and executable by a processor, the method comprising:

buffering, via the processor, a chunk having a plurality of tuples received in a pipeline manner in a build phase;

after buffering a last of the plurality of tuples of the chunk, applying, via the processor, a block computation to all of the buffered tuples in the chunk during a compute phase; and

returning, via the processor, results of the block computation on the chunk, the results returned one tuple at a time to a query processing pipeline in a stream-out phase.

2. The method of claim 1, further comprising:

calling a function for each tuple in the chunk during the build phase; and

returning a null for each function called until each tuple in the chunk is buffered, and then entering the compute phase.

3. The method of claim 1, wherein the compute phase is executed external to a query engine.

4. The method of claim 1, wherein buffering is dynamic along the query processing pipeline.

5. The method of claim 1, wherein the chunk is less than a portion of data upon which an entire query is operative.

6. The method of claim 1, wherein the chunk is selected based on number of tuples.

7. The method of claim 1, wherein the chunk is selected based on a time window.

8. The method of claim 1, wherein the chunk is selected based on representing individual objects.

9. The method of claim 1, wherein buffering is by multiple calls with each of the multiple calls delivering a single return tuple.

10. A system of block computation implemented by program code stored on a non-transient computer-readable medium and executable by a processor, the system, comprising:

a buffer for holding a chunk having a plurality of tuples received in a pipeline manner during a build phase;

a computation engine for applying a block computation to all of the chunk during a compute phase after the build phase finishes; and

a query processing pipeline for returning results from the block computation on the chunk, the results returned one tuple at a time during a stream-out phase.

11. The system of claim 10, wherein the build phase finishes after a last of the plurality of tuples for the chunk is in the buffer.

12. The system of claim 10, further comprising a Set-In, Set-Out (SISO) user defined function (UDF), the UDF defining operations for a set of tuples representing a single object.

13. The system of claim 12, wherein the SISO UDF is for batch analytic computing.

14. The system of claim 12, wherein the SISO UDF receives vector input data for batch processing.

15. The system of claim 12, wherein the SISO UDF includes set-in, set-out block computation operators.

16. The system of claim 10, wherein a data source of the plurality of tuples is a table.

17. The system of claim 10, wherein a data source of the plurality of tuples is a stream.

18. A system of block computation implemented by program code stored on a non-transient computer-readable medium and executable by a processor, the system comprising:

a computation engine for buffering a chunk having a plurality of tuples received in a pipeline manner, and for, after buffering a last of the plurality of tuples in the chunk, applying a block computation to all of the buffered tuples in the chunk;

a user defined function (UDF) defining the block computation for the computation engine, the UDF defining operations for a tuple-set corresponding to a single time window; and

a query processing pipeline for receiving results from the computation engine one tuple at a time.

19. The system of claim 18, wherein the UDF is set-in, set-out (SISO) modeling applications definable on tuple-sets defined by the chunk.

20. The system of claim 18, wherein the UDF has four levels of accessible states:

per-function state, per-chunk state, per-tuple state, and per-return state.