STREAM DATA PROCESSING METHOD ON RECURSIVE QUERY OF GRAPH DATA

• US 20140181144A1
• Filed: 12/20/2012
• Published: 06/26/2014
• Est. Priority Date: 12/20/2012
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: KASHIYAMA, Toshihiko
• Granted Patent: US 9,400,823 B2
• Time in Patent Office: Days
• US Class Current: 707/773
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries

Abstract

A stream data processing computer comprises: a query execution module to receive an input stream of stream data, manage execution order of operators for executing a query, and output an output stream, the operators including static recursive execution modules, a judge operator, and a dynamic recursive execution module; and a memory including a temporal store to store data generated in executing the query. The static recursive execution modules are configured to conduct a recursive number of static recursive execution of the query. The judge operator is configured to determine whether a preset condition is met after the recursive number of static recursive execution of the query. The dynamic recursive execution module is configured, if the preset condition is not met, to continue execution of the query by dynamic recursive execution from a last result of a last module of the static recursive execution modules, based on a dynamic recursive value.

Claims

1. A stream data processing computer comprising:

a processor;

a query execution module to receive an input stream of stream data, manage execution order of operators for executing a query, and output an output stream of stream data, the operators including static recursive execution modules, a judge operator, and a dynamic recursive execution module; and

a memory including a temporal store to store data generated in executing the query;

wherein the static recursive execution modules are configured to conduct a recursive number of static recursive execution of the query;

wherein the judge operator is configured to determine whether a preset condition is met after the recursive number of static recursive execution of the query; and

wherein the dynamic recursive execution module is configured, if the preset condition is not met, to continue execution of the query by dynamic recursive execution from a last result of a last static recursive execution module of the static recursive execution modules, based on a dynamic recursive value.

2. The stream data processing computer according to claim 1,

wherein the preset condition is met if the static recursive execution modules generate a new result after the recursive number of static recursive execution of the query.

3. The stream data processing computer according to claim 2,

wherein the judge operator is configured to determine whether the static recursive execution modules generate a new result after the recursive number of static recursive execution of the query;

and

wherein the dynamic recursive execution module is configured, if the new result is not generated, to continue execution of the query by dynamic recursive execution from the last result of the last static recursive execution module of the static recursive execution modules, for a recursive times equal to the dynamic recursive value.

4. The stream data processing computer according to claim 1, further comprising:

a query parser which calls a static recursive query generating module in the memory to generate a number of the static recursive execution modules corresponding to the recursive number of static recursive execution of the query, a judge operator generating module in the memory to generate the judge operator; and

a dynamic recursive query generating module in the memory to generate the dynamic recursive execution module, in an execution tree.

5. The stream data processing computer according to claim 4,

wherein the query parser is configured to generate, based on the input stream and by parsing the query, the execution tree which includes an input tuple set of multiple streams;

wherein the operators include sliding window operators each cutting out data rows from the multiple streams by specifying a time interval for cutting out data rows;

wherein the static recursive execution modules are placed after the sliding window operators for the multiple streams;

wherein the judge operator is placed after the last static recursive execution module;

wherein the dynamic recursive execution module is placed after the judge operator.

6. The stream data processing computer according to claim 1,

wherein the dynamic recursive execution module includes a recursive times judge to judge whether to perform a recursive number of dynamic recursive execution of the query from the last result of the last static recursive execution module, based on the dynamic recursive value and a recursive times of dynamic recursive execution.

7. The stream data processing computer according to claim 1,

wherein the preset condition relates to at least one of path weight or path number.

8. The stream data processing computer according to claim 1,

wherein the recursive number of static recursive execution of the query and the dynamic recursive value are preset.

9. The stream data processing computer according to claim 1, further comprising:

a resource utilization monitoring module to monitor resource utilization of the stream data processing computer; and

a recursive number adjusting module to change at least one of the recursive number of static recursive execution of the query or the dynamic recursive value, based on the monitored resource utilization.

10. The stream data processing computer according to claim 9, further comprising:

a recursive number adjust setting module to update a recursive number adjusting definition table based on a recursive number adjust setting command;

wherein the resource utilization monitoring module is configured to monitor resource utilization of the stream data processing computer based on the recursive number adjusting definition table; and

wherein the recursive number adjusting module is configured to change at least one of the recursive number of static recursive execution of the query or the dynamic recursive value, based on the monitored resource utilization and the recursive number adjusting definition table.

11. A non-transitory computer-readable storage medium storing a plurality of instructions for controlling a data processor to perform stream data processing that receives stream data input information as primary information and generates secondary information by acquiring primary information within a predetermined period from among the received primary information, in a stream data processing computer which includes a memory having a temporal store, the plurality of instructions comprising:

instructions that cause the data processor to receive an input stream of stream data, manage execution order of operators for executing a query, and output an output stream of stream data, the operators including static recursive execution modules, a judge operator, and a dynamic recursive execution module;

instructions that cause the data processor to store in the temporal store data generated in executing the query;

instructions that cause the static recursive execution modules to conduct a recursive number of static recursive execution of the query;

instructions that cause the judge operator to determine whether a preset condition is met after the recursive number of static recursive execution of the query; and

instructions that cause the dynamic recursive execution module, if the preset condition is not met, to continue execution of the query by dynamic recursive execution from a last result of a last static recursive execution module of the static recursive execution modules, based on a dynamic recursive value.

12. The non-transitory computer-readable storage medium according to claim 11,

wherein the preset condition is met if the static recursive execution modules generate a new result after the recursive number of static recursive execution of, the query.

13. The non-transitory computer-readable storage medium according to claim 12,

wherein the instructions that cause the judge operator to determine whether a preset condition is met comprise instructions that cause the judge operator to determine whether the static recursive execution modules generate a new result after the recursive number of static recursive execution of the query;

and

wherein the instructions that cause the dynamic recursive execution module to continue execution of the query comprise instructions that cause the dynamic recursive execution module, if the new result is not generated, to continue execution of the query by dynamic recursive execution from the last result of the last static recursive execution module of the static recursive execution modules, for a recursive times equal to the dynamic recursive value.

14. The non-transitory computer-readable storage medium according to claim 11, wherein the plurality of instructions further comprise:

instructions that cause the data processor to generate a number of the static recursive execution modules corresponding to the recursive number of static recursive execution of the query, in an execution tree;

instructions that cause the data processor to generate the judge operator in the execution tree; and

instructions that cause the data processor to generate the dynamic recursive execution module in the execution tree.

15. The non-transitory computer-readable storage medium according to claim 14, wherein the plurality of instructions further comprise:

instructions that cause the data processor to generate, based on the input stream and by parsing the query, the execution tree which includes an input tuple set of multiple streams, wherein the operators include sliding window operators each cutting out data rows from the multiple streams by specifying a time interval for cutting out data rows;

instructions that cause the data processor to place the static recursive execution modules after the sliding window operators for the multiple streams;

instructions that cause the data processor to place the judge operator after the last static recursive execution module; and

instructions that cause the data processor to place the dynamic recursive execution module after the judge operator.

16. The non-transitory computer-readable storage medium according to claim 11, wherein the instructions that cause the dynamic recursive execution module, if the preset condition is not met, to continue execution of the query by dynamic recursive execution comprise:

instructions that cause the dynamic recursive execution module to judge whether to perform a recursive number of dynamic recursive execution of the query from the last result of the last static recursive execution module, based on the dynamic recursive value and a recursive times of dynamic recursive execution.

17. The non-transitory computer-readable storage medium according to claim 11,

wherein the preset condition relates to at least one of path weight or path number.

18. The non-transitory computer-readable storage medium according to claim 11,

wherein the recursive number of static recursive execution of the query and the dynamic recursive value are preset.

19. The non-transitory computer-readable storage medium according to claim 11, wherein the plurality of instructions further comprise:

instructions that cause the data processor to monitor resource utilization of the stream data processing computer; and

instructions that cause the data processor to change at least one of the recursive number of static recursive execution of the query or the dynamic recursive value, based on the monitored resource utilization.

20. The non-transitory computer-readable storage medium according to claim 19, wherein the plurality of instructions further comprise:

instructions that cause the data processor to update a recursive number adjusting definition table based on a recursive number adjust setting command;

instructions that cause the data processor to monitor resource utilization of the stream data processing computer based on the recursive number adjusting definition table; and

instructions that cause the data processor to change at least one of the recursive number of static recursive execution of the query or the dynamic recursive value, based on the monitored resource utilization and the recursive number adjusting definition table.