Smart tuple dynamic grouping of tuples

• US 10,255,347 B2
• Filed: 11/06/2015
• Issued: 04/09/2019
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cao, Bin, Eidem, Jessica R., Muras, Brian R., Sun, Jingdong
• Primary Examiner(s): Alam, Shahid A
• Application Number:
  US14/934,292
• Publication Number:
  US 20170091286A1
• Time in Patent Office: 1,250 Days
• Field of Search: 707736
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/285 Clustering or classification

Abstract

A stream application receives a stream of tuples to be processed by a plurality of processing elements. The plurality of processing elements operating on one or more compute nodes. Each processing element has one or more stream operators. The stream application assigns one or more processing cycles to one or more segments of software code. The segments of software code are embedded in a tuple of the stream of tuples. The software-embedded tuple identifies a set of target tuples based upon operation criteria. The set of target tuples are a part of the stream of tuples. The software-embedded tuple performs an operation based on the set of identified target tuples.

Claims

1. A method for processing a stream of tuples, the stream of tuples located in a stream-based computing environment designed to process big-data, the tuples of the big-data set being processed before being recorded into a final destination, the method comprising:

receiving, by a stream application, a stream of tuples to be processed by a plurality of processing elements operating on one or more compute nodes, each processing element having one or more stream operators, wherein at least one tuple of the stream of tuples contains one or more embedded segments of software code;

assigning, by the stream application, one or more processing cycles to the one or more segments of software code embedded in the at least one tuple of the stream of tuples; and

executing, by the software-embedded tuple, the following operations;

identifying a set of target tuples of the stream of tuples based upon operation criteria; and

performing, based on the set of identified target tuples, an operation.

2. The method of claim 1, further comprising:

obtaining, by the software-embedded tuple and before the performed operation, a request to perform the operation; and

processing, by the software-embedded tuple and before the obtaining the request, the set of identified target tuples.

3. The method of claim 2, wherein the processing is periodically sampling runtimes of the set of identified target tuples at each stream operator of the stream application, and wherein the performing the operation is averaging the runtimes.

4. The method of claim 1, further comprising:

generating, by the software-embedded tuple, a summary from the performed operation; and

transmitting, by the software-embedded tuple, the summary to the stream application.

5. The method of claim 1, wherein the set of target tuples includes two or more tuples from the stream of tuples.

6. The method of claim 1, wherein the operation criteria is included within the one or more segments of software code at the time of creation of the software-embedded tuple.

7. The method of claim 1, wherein the operation criteria is included in a request, the method further comprising:

obtaining, by the software-embedded tuple and before the identifying the set of target tuples, the request.

8. The method of claim 7, wherein the request is obtained from the stream application.

9. The method of claim 7, wherein the request is obtained from a second software-embedded tuple.

10. The method of claim 1, wherein the operation criteria is related to the location of the set of target tuples in the stream application.

11. The method of claim 1, wherein the operation criteria is related to one or more attributes of each of the set of target tuples.

12. The method of claim 1, wherein the operation criteria includes an update value, and wherein the performed operation is an update of an attribute of each of the set of identified target tuples with the update value.