Smart tuple dynamic grouping of tuples

• US 10,733,209 B2
• Filed: 09/30/2015
• Issued: 08/04/2020
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cao, Bin, Eidem, Jessica R., Muras, Brian R., Sun, Jingdong
• Primary Examiner(s): Saeed, Usmaan
• Assistant Examiner(s): Perez-Arroyo, Raquel
• Application Number:
  US14/870,041
• Publication Number:
  US 20170091305A1
• Time in Patent Office: 1,770 Days
• Field of Search: 707737
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/285 Clustering or classification

Abstract

A stream application receives a stream of tuples to be processed by a plurality of processing elements. The plurality of processing elements operating on one or more compute nodes. Each processing element has one or more stream operators. The stream application assigns one or more processing cycles to one or more segments of software code. The segments of software code are embedded in a tuple of the stream of tuples. The software-embedded tuple identifies a set of target tuples based upon operation criteria. The set of target tuples are a part of the stream of tuples. The software-embedded tuple performs an operation based on the set of identified target tuples.

Claims

1. A system for processing a stream of tuples comprising:

a plurality of processing elements configured to receive a stream of tuples, each processing element having one or more stream operators, each tuple of the stream of tuples to be operated on by the processing elements before being written to one or more databases;

two or more processors; and

a memory containing an application that, when executed, causes at least one of the two or more processors to perform a method comprising;

embedding, by a first processor, a tuple of the stream of tuples with one or more segments of software code, wherein the logic of the processing elements and the stream operators are not directly updatable, wherein the one or more segments of software code temporarily change functionality of the plurality of processing elements of a stream application, and wherein the one or more segments of software code temporarily change functionality of the one or more stream operators of the stream application;

identifying, by a second processor and based on the one or more embedded segments of software code, a set of target tuples of the stream of tuples based upon operation criteria, wherein the one or more embedded segments of software code execute outside of the plurality of processing elements, and wherein the one or more embedded segments of software code execute outside of the one or more stream operators, wherein the identifying based on the embedded segments of software code occurs before the tuple is written to any of the one or more databases; and

performing, by the second processor and based on the set of identified target tuples and based on the one or more embedded segments of software code, an operation, wherein the operation includes bypassing one of the plurality of processing elements of the stream application.

2. The system of claim 1, wherein the operation criteria is included in a request, the system further comprising:

obtaining, by the second processor and based on the embedded segments of software code and before the identifying the set of target tuples, the request.

3. The system of claim 2, wherein the request is obtained from a second software-embedded tuple.

4. The system of claim 1, wherein the operation criteria includes an update value, and wherein the performed operation is an update of an attribute of each of the set of identified target tuples with the update value.

5. A computer program product for processing a stream of tuples, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a plurality of processing elements operating on one or more compute nodes, each processing element having one or more stream operators, the program instructions to perform a method before the stream of tuples is stored in a database, the method comprising:

embedding, by a first compute node, a tuple of the stream of tuples with one or more segments of software code, wherein the logic of the processing elements and the stream operators are not directly updatable, wherein the one or more segments of software code temporarily change functionality of the plurality of processing elements of a stream application, and wherein the one or more segments of software code temporarily change functionality of the one or more stream operators of the stream application;

identifying, by a second compute node and based on the one or more embedded segments of software code, a set of target tuples of the stream of tuples based upon operation criteria wherein the one or more embedded segments of software code execute outside of the plurality of processing elements, and wherein the one or more embedded segments of software code execute outside of the one or more stream operators, wherein the identifying based on the embedded segments of software code occurs before the tuple is written to any of the one or more databases; and

performing, by the second compute node and based on the set of identified target tuples and based on the one or more embedded segments of software code, an operation, wherein the operation includes bypassing one of the plurality of processing elements of the stream application.

6. The computer program product of claim 5, further comprising:

obtaining, by the second compute node and based on the one or more embedded segments of software code and before the performed operation, a request to perform the operation; and

processing, by the second compute node and based on the one or more embedded segments of software code and before the obtained request, the set of identified target tuples.

7. The computer program product of claim 6, wherein the processing is periodically sampling runtimes of the set of identified target tuples at each stream operator of the stream application, and wherein the performing the operation is averaging the runtimes.

8. The computer program product of claim 5, wherein the operation criteria is related to one or more attributes of each of the set of target tuples.

9. A system for processing a stream of tuples, the system comprising:

a first memory, the first memory containing a plurality of processing elements of a stream application, each processing element having one or more stream operators, wherein the logic of the processing elements and the stream operators are not updatable;

a first processor, the first processor communicatively coupled to the first memory, the first processor configured to;

receive a stream of tuples;

embed into at least one tuple of the stream of tuples one or more segments of software code, wherein the one or more segments of software code temporarily change functionality of the one or more processing elements or and the one or more stream operators of the stream application by bypassing a portion of the stream application;

process a subset of tuples of the stream of tuples based on the configuration of the one or more stream operators; and

store the subset of tuples of the stream of tuples in a database;

a second memory, the second memory including computer instructions; and

a second processor, the second processor communicatively coupled to the second memory, the second processor in response to the included computer instructions configured to;

identify, based on the one or more segments of software code embedded into the at least one tuple and before the storage of the processed tuples by the first processor, a set of target tuples of the stream of tuples based upon operation criteria, wherein the one or more embedded segments of software code execute outside of the plurality of processing elements, and wherein the one or more embedded segments of software code execute outside of the one or more stream operators; and

perform, based on the one or more segments of software code embedded into the at least one tuple and before the storage of the processed tuples by the first processor and based on the set of identified target tuples, an operation of the stream application, wherein the operation includes bypassing one of the plurality of processing elements.

10. The system of claim 9, wherein the first processor is further configured to:

remove, before the storage of the subset of tuples of the stream of tuples in the database and after the perform the operation by the second processor, the embedded one or more segments of software code from the at least one tuple.