Recognizing operational options for stream operators at compile-time

• US 9,110,681 B2
• Filed: 12/11/2013
• Issued: 08/18/2015
• Est. Priority Date: 12/11/2013
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Branson, Michael J., Santosuosso, John M.
• Primary Examiner(s): Rampuria, Satish
• Application Number:
  US14/102,614
• Publication Number:
  US 20150160932A1
• Time in Patent Office: 615 Days
• Field of Search: 717140-167
• US Class Current: 1/1
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 8/443 Optimisation

Abstract

A source code that includes an operator graph that includes a plurality of processing elements, each processing element having one or more stream operators is received. A metadata tag describing a customization of at least one of the one or more stream operators having a windowing processing operation is parsed from the source code. The source code of the streaming application having the windowing processing operation based on the metadata tag is compiled.

Claims

1. A system for initializing a streaming application, the system comprising:

a compiler, wherein the compiler includes a processor and a memory, wherein the compiler is configured to;

receive a source code that includes an operator graph, the operator graph including a plurality of processing elements configured to execute the streaming application, each processing element having one or more stream operators, wherein the streaming application is executed on one or more compute nodes, each compute node adapted to execute the one or more stream operators;

parse, from the source code, a metadata tag describing a customization of at least one of the one or more stream operators having a windowing processing operation, wherein the customization is a delay operation within the windowing processing operation, wherein the customization changes from a first customization to a second customization in response to a change in central processing unit capacity on the one or more compute nodes, wherein the windowing procession operation includes one or more windows, and wherein the one or more windows are a logical container for tuples received by an input port of one of the one or more stream operators; and

compile the source code of the streaming application having the windowing processing operation, based on the metadata tag.

2. The system of claim 1, wherein the delay operation is designed to delay a stream operator from operating, based on a criterion that is selected from the group consisting of:

an amount of received tuples, an amount of elapsed time, a change in an attribute of the tuples, receiving a number of punctuations, establishing dynamic connections in the operator graph, and an amount of the tuples that have streamed through a portion of the operator graph.

3. The system of claim 1, wherein a second processing operation to be performed within the windowing processing operation is also based on the metadata tag.

4. The system of claim 3, wherein the second processing operation is one of partitioning, sorting, and aggregating.

5. The system of claim 1, wherein the customization changes from a first customization to a second customization in response to a change in time.

6. The system of claim 1, wherein the metadata tag is a keyword assigned to a piece of an information artifact.

7. The system of claim 6, wherein the information artifact is a stream operator.

8. The system of claim 1, wherein the one or more stream operators include a customized run-time behavior to perform an operation.

9. The system of claim 8, wherein the operation is selected from the group consisting of:

partitioning, sorting, and aggregating.

10. A computer program product for initializing a streaming, the computer program product comprising a computer readable hardware storage medium having program code embodied therewith, the program code comprising computer readable program code configured to:

receive a source code that includes an operator graph, the operator graph including a plurality of processing elements configured to execute the streaming application, each processing element having one or more stream operators, wherein the streaming application is executed on one or more compute nodes, each compute node adapted to execute the one or more stream operators;

parse, from the source code, a metadata tag describing a customization of at least one of the one or more stream operators having a windowing processing operation, wherein the customization is a delay operation within the windowing processing operation, wherein the customization changes from a first customization to a second customization in response to a change in central processing unit capacity on the one or more compute nodes, wherein the windowing procession operation includes one or more windows, and wherein the one or more windows are a logical container for tuples received by an input port of one of the one or more stream operators; and

compile the source code of the streaming application having the windowing processing operation, based on the metadata tag.

11. The computer program product of claim 10, wherein the delay operation is designed to delay a stream operator from operating, based on a criterion that is selected from the group consisting of:

an amount of received tuples, an amount of elapsed time, a change in an attribute of the tuples, receiving a number of punctuations, establishing dynamic connections in the operator graph, and an amount of the tuples that have streamed through a portion of the operator graph.

12. The computer program product of claim 10, wherein a second processing operation to be performed within the windowing processing operation is also based on the metadata tag.

13. The computer program product of claim 10, wherein the customization changes from a first customization to a second customization in response to a change in time.

14. The computer program product of claim 10, wherein the metadata tag is a keyword assigned to a piece of an information artifact.

15. The computer program product of claim 14, wherein the information artifact is a stream operator.

16. The computer program product of claim 10, wherein the one or more stream operators include a customized run-time behavior to perform an operation.

17. The computer program product of claim 16, wherein the operation is selected from the group consisting of:

partitioning, sorting, and aggregating.