Multi-streams analytics

• US 8,880,493 B2
• Filed: 09/28/2011
• Issued: 11/04/2014
• Est. Priority Date: 09/28/2011
• Status: Active Grant

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Chen, Qiming, Hsu, Meichun
• Primary Examiner(s): OBISESAN, AUGUSTINE KUNLE
• Application Number:
  US13/246,947
• Publication Number:
  US 20130080413A1
• Time in Patent Office: 1,133 Days
• Field of Search: None
• US Class Current: 707/706
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning

Abstract

Multi-stream analytics is disclosed. An example method of multi-stream analytics with a query engine includes punctuating unbounded streaming data into data chunks, each of the data chunks representing a bounded data set in the unbounded streaming data. The method also includes processing one of the data chunks. The method also includes rewinding a query instance for processing another of the data chunks.

Claims

1. A method of multi-stream analytics with a query engine, comprising:

punctuating unbounded streaming data into data chunks, each of the data chunks representing a bounded data set in the unbounded streaming data;

processing one of the data chunks,

cutting query execution based on cycle specification; and

rewinding a state of a query instance without shutting down the query instance between execution cycles for processing another of the data chunks in a next execution cycle.

2. The method of claim 1, further comprising executing a sequence of query cycles on a cycle-by-cycle basis on a sequence of the data chunks in a chunk-by-chunk manner.

3. The method of claim 1, further comprising joining multiple streams and self-joining a single stream using a data chunk-based window or sliding window, wherein self-joining, joins a most recent chunk and a predetermined past chunk.

4. The method of claim 1, further comprising joining multiple input streams by buffering consecutive data chunks in a sliding window.

5. The method of claim 1, further comprising joining a stream window and a static table.

6. The method of claim 1, further comprising window-joining multiple input streams.

7. The method of claim 1, further comprising joining streams in sliding windows.

8. The method of claim 1, further comprising self-joining a stream in a sliding window.

9. The method of claim 1, wherein the query uses structured query language (SQL).

10. The method of claim 1, further comprising when an end-of-cycle event is signaled from a stream capture function (SCF), the query engine completes a current query execution cycle and then rewinds the query instance for the next execution cycle.

11. The method of claim 10, wherein the end-of-cycle event is determined when a first stream element belonging to the next execution cycle is received.

12. The method of claim 11, wherein the query instance remains active even after the end-of-cycle event.

13. The method of claim 12, further comprising discarding intermediate results of SQL operators associated with a current chunk of data, and maintaining application context at the end-of-cycle event.

14. The method of claim 1, further comprising maintaining continuity of the query instance for retaining the buffered data for history-sensitive applications.

15. A system haying a query engine executing program code for multi-stream analytics, the program code stored on a non-transient computer readable medium and executed to:

punctuate unbounded streaming data into data chunks representing bounded data sets;

cutting query execution based on a cut point specified in cycle specification; and

execute a query instance on one of the data chunks, and then rewind a state of the query instance without shutting down the query instance between execution cycles for processing another of the data chunks in a next cycle; and

execute a sequence of query cycles on a sequence of the data chunks.

16. The system of claim 15, wherein the program code is further executed to join multiple streams and self-join a single stream using a data chunk-based window or sliding window.

17. The system of claim 15, wherein the program code is further executed to join multiple input streams.

18. The system of claim 15, wherein the program code is further executed to join a stream window and a static table.

19. The system of claim 15, wherein the program code is further executed to join streams in sliding windows.

20. The system of claim 15, wherein the program code is further executed to self-join a stream in a sliding window.