STREAM OPERATOR MANAGEMENT

• US 20170269911A1
• Filed: 03/21/2016
• Published: 09/21/2017
• Est. Priority Date: 03/21/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Barsness, Eric L., Cook, Alexander, Orozco, Manuel, Santosuosso, John M.
• Granted Patent: US 10,042,611 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 8/20 Software design
• CPC Class Codes: G06F 8/36 Software reuse
• CPC Class Codes: H04L 65/60 Network streaming of media packets

Abstract

Disclosed aspects relate to stream operator management. A stream operator may be analyzed with respect to a set of usage indicators. The stream operator may operate on a computer processor and correspond to a processing element for processing a stream of tuples. Based on analyzing the stream operator, a set of profile data for the stream operator may be determined. The set of profile data for the stream operator may be established for utilization to develop a streaming application.

Claims

1. A computer-implemented method of stream operator management, the method comprising:

analyzing, with respect to a set of usage indicators, a stream operator which operates on a computer processor and corresponds to a processing element for processing a stream of tuples;

determining, based on the analyzing, a set of profile data for the stream operator; and

establishing, for utilization to develop a streaming application, the set of profile data for the stream operator.

2. The method of claim 1, wherein develop the streaming application includes construct implementation code.

3. The method of claim 1, wherein develop the streaming application includes configure a set of parameters.

4. The method of claim 1, wherein establishing, for utilization to develop the streaming application, the set of profile data for the stream operator includes:

providing, for utilization to develop the streaming application in an integrated development environment, the set of profile data for the stream operator to provide immediate feedback on a streams design in the integrated development environment.

5. The method of claim 1, wherein establishing, for utilization to develop the streaming application, the set of profile data for the stream operator includes:

capturing, in a streams manager for utilization to develop the streaming application, the set of profile data for the stream operator.

6. The method of claim 1, wherein the set of usage indicators include a set of values which correspond to a selection from a group consisting of at least one of:

a processor utilization factor,

a memory utilization factor,

a disk utilization factor, or

a throughput factor.

7. The method of claim 1, wherein analyzing, with respect to the set of usage indicators, the stream operator includes:

monitoring the stream operator;

identifying, based on the monitoring, a set of values for the set of usage indicators; and

collecting the set of values to facilitate determination of the set of profile data for the stream operator.

8. The method of claim 1, wherein the set of profile data for the stream operator includes:

a set of computing resource arrangements for utilization to develop the streaming application.

9. The method of claim 8, wherein the set of computing resource arrangements includes a selection from a group consisting of at least one of:

a threshold value for a set of processing resources,

a threshold value for a set of memory resources, or

a threshold value for a set of disk resources.

10. The method of claim 1, wherein the set of profile data for the stream operator includes a selection from a group consisting of at least one of:

a tuple type, a tuple quantity threshold value, a tuple size threshold value, a tuple processing rate threshold value, an attribute type, an attribute quantity threshold value, an attribute size threshold value, a window size threshold value, or a temporal period threshold value.

11. The method of claim 1, wherein determining the set of profile data for the stream operator includes:

computing a set of threshold factor values which correspond to the set of usage indicators.

12. The method of claim 1, wherein determining the set of profile data for the stream operator includes:

computing a set of target factor values which correspond to the set of usage indicators.

13. The method of claim 12, wherein a first throughput value using the set of target factor values exceeds a second throughput value using a set of original factor values.

14. The method of claim 1, further comprising:

introducing, using the set of profile data, a split operator to allocate the stream of tuples with respect to a plurality of processing elements.

15. The method of claim 1, further comprising:

introducing, using the set of profile data, a selection from a group consisting of at least one of;

a filter operator,

a functor operator,

a custom operator,

an ordering operator,

a barrier operator, or

a sync operator.

16. The method of claim 1, wherein the analyzing, the determining, and the establishing each occur in an automated fashion without user intervention, and wherein a set of patterns for improving performance is automatically applied.

17. The method of claim 1, further comprising:

receiving a stream of tuples to be processed by a set of processing elements which includes the processing element and operates on a set of compute nodes; and

processing, using the set of processing elements operating on the set of compute nodes, the stream of tuples.

18. A system for stream operator management, the system comprising:

a memory having a set of computer readable computer instructions, and

a processor for executing the set of computer readable instructions, the set of computer readable instructions including;

analyzing, with respect to a set of usage indicators, a stream operator which operates on a computer processor and corresponds to a processing element for processing a stream of tuples;

determining, based on the analyzing, a set of profile data for the stream operator; and

establishing, for utilization to develop a streaming application, the set of profile data for the stream operator.

19. A computer program product for stream operator management, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause the processor to perform a method comprising:

analyzing, with respect to a set of usage indicators, a stream operator which operates on a computer processor and corresponds to a processing element for processing a stream of tuples;

determining, based on the analyzing, a set of profile data for the stream operator; and

establishing, for utilization to develop a streaming application, the set of profile data for the stream operator.

20. The computer program product of claim 19, wherein at least one of:

the program instructions are stored in a computer readable storage medium in a data processing system, and wherein the program instructions were downloaded over a network from a remote data processing system;

or

the program instructions are stored in a computer readable storage medium in a server data processing system, and wherein the program instructions are downloaded over a network to a remote data processing system for use in a computer readable storage medium with the remote data processing system.