SUSPENDING AND RESUMING CONTINUOUS QUERIES OVER DATA STREAMS

• US 20140280325A1
• Filed: 03/15/2013
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Fernando, Rex E., Krishnamurthy, Sailesh, Metz, Chris, Bhattacharya, Jisu
• Granted Patent: US 9,910,896 B2
• Time in Patent Office: Days
• US Class Current: 707/770
• CPC Class Codes: G06F 16/24557 Efficient disk access during query execution
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/2471 Distributed queries

Abstract

In an embodiment, a method comprises processing an input data stream as the data stream is streamed and producing a derived stream therefrom; storing the input data stream in an input archive; suspending processing of the input data stream; subsequent to suspending processing, resuming processing of the input data stream, wherein resuming comprises: storing newly received data in the input data stream in a buffer, as the input data stream is streamed; determining a first timestamp; determining a second timestamp; searching the input archive to find a data item that matches the first timestamp of the last processed data item; processing data in the input archive having timestamps that are greater than the first timestamp until arriving at data with a third timestamp that is greater than the second timestamp; processing the input data stream from the buffer; continuing processing the input data stream as the input stream is streamed.

Claims

1. A method comprising:

processing an input data stream and producing a derived stream therefrom;

storing the input data stream in an input archive;

suspending processing of the input data stream; and

subsequent to suspending processing, resuming processing of the input data stream, wherein resuming comprises;

storing newly received data in the input data stream in a buffer, as the input data stream is streamed;

determining a first timestamp of a last processed data item;

determining a second timestamp from the buffered data stream;

searching the input archive to find a data item that matches the first timestamp of the last processed data item;

processing particular data in the input archive having timestamps that are greater than the first timestamp until arriving at data with a third timestamp that is greater than the second timestamp;

processing the input data stream from the buffer; and

continuing processing the input data stream as the input stream is streamed;

wherein the method is performed by one or more computing devices.

2. The method of claim 1 wherein processing the input data stream comprises processing a continuous query on the input data stream.

3. The method of claim 1 comprising storing the derived data in a derived data archive.

4. The method of claim 3, wherein the derived data is transactionally stored in the derived data archive based, at least in part, on a window interval defined by a continuous query.

5. The method of claim 3 wherein:

a datum in both the input stream and the derived stream comprises a timestamp;

the first timestamp is the greatest timestamp stored in the derived data archive.

6. The method of claim 1 wherein the buffer contains data from the input data stream, beginning from when processing is resumed and ending when data in the buffer is finished being processed.

7. The method of claim 1 wherein processing the input data stream from the buffer comprises removing and processing the input data stream until the buffer is empty.

8. The method of claim 1 wherein processing of the input data stream as the input stream is streamed continues subsequent to processing all data stored in the buffer.

9. The method of claim 1 wherein the second timestamp is the least timestamp in the buffer before processing data in the buffer.

10. The method of claim 1 comprising:

determining an amount of network congestion;

suspending processing automatically based, at least in part, on the amount of network congestion;

resuming processing automatically based, at least in part, on the amount of network congestion.

11. The method of claim 1 wherein:

suspending processing of the input data stream also suspends processing of downstream nodes;

resuming processing of the input data stream also resumes processing of downstream nodes.

12. The method of claim 1 wherein input data stream is any of:

raw stream, derived stream.

13. One or more non-transitory computer-readable media storing one or more sequences of instructions which, when executed by one or more computing devices, cause:

processing an input data and producing a derived stream therefrom;

storing the input data stream in an input archive;

suspending processing of the input data stream; and

subsequent to suspending processing, resuming processing of the input data stream, wherein resuming comprises;

storing newly received data in the input data stream in a buffer, as the input data stream is streamed;

determining a first timestamp of a last processed data item;

determining a second timestamp from the buffered data stream;

searching the input archive to find a data item that matches the first timestamp of the last processed data item;

processing particular data in the input archive having timestamps that are greater than the first timestamp until arriving at data with a third timestamp that is greater than the second timestamp;

processing the input data stream from the buffer; and

continuing processing the input data stream as the input stream is streamed;

wherein the method is performed by one or more computing devices.

14. The one or more non-transitory computer-readable media of claim 13 wherein processing the input data stream comprises processing a continuous query on the input data stream.

15. The one or more non-transitory computer-readable media of claim 13 comprising storing the derived data in a derived data archive.

16. The one or more non-transitory computer-readable media of claim 15, wherein the derived data is transactionally stored in the derived data archive based, at least in part, on a window interval defined by a continuous query.

17. The one or more non-transitory computer-readable media of claim 15 wherein:

a datum in both the input stream and the derived stream comprises a timestamp;

the first timestamp is the greatest timestamp stored in the derived data archive.

18. The one or more non-transitory computer-readable media of claim 13 wherein the buffer contains data from the input data stream, beginning from when processing is resumed and ending when data the buffer is finished being processed.

19. The one or more non-transitory computer-readable media of claim 13 wherein processing the input data stream from the buffer comprises removing and processing the input data stream until the buffer is empty.

20. The one or more non-transitory computer-readable media of claim 13 wherein processing of the input data stream as the input stream is streamed continues subsequent to processing all data stored in the buffer.

21. The one or more non-transitory computer-readable media of claim 13 wherein the second timestamp is the least timestamp in the buffer before processing data in the buffer.

22. The one or more non-transitory computer-readable media of claim 13 comprising:

determining an amount of network congestion;

suspending processing automatically based, at least in part, on the amount of network congestion;

resuming processing automatically based, at least in part, on the amount of network congestion.

23. The one or more non-transitory computer-readable media of claim 13 wherein:

suspending processing of the input data stream also suspends processing of downstream nodes;

resuming processing of the input data stream also resumes processing of downstream nodes.

24. The one or more non-transitory computer-readable media of claim 13 wherein input data stream is any of:

raw stream, derived stream.