Smart tuple condition-based operation performance

• US 10,558,670 B2
• Filed: 09/30/2015
• Issued: 02/11/2020
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cao, Bin, Eidem, Jessica R., Muras, Brian R., Sun, Jingdong
• Primary Examiner(s): Alam, Shahid A
• Application Number:
  US14/870,042
• Publication Number:
  US 20170091283A1
• Time in Patent Office: 1,595 Days
• Field of Search: 707607
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 8/41 Compilation
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

A stream application receives a stream of tuples to be processed by a plurality of processing elements operating on one or more compute nodes. Each processing element has one or more stream operators. The stream application assigns one or more processing cycles to one or more segments of software code. The segments of software code are embedded in one or more tuples of the stream of tuples. The stream application determines that a condition is met based on the processing cycles. The stream application performs an operation based on the processing cycles.

Claims

1. A method for processing a stream of tuples, the stream of tuples located in a stream-based computing environment designed to process big-data continuously and in real-time, the method comprising:

receiving, by a stream application, a stream of tuples to be processed by a plurality of processing elements operating on one or more hardware compute nodes of the stream application, the tuples of the big-data set being processed before being recorded into a final destination, wherein at least one tuple of the stream of tuples contains one or more embedded segments of software code, wherein the stream application utilizes programmatic conventions to provide context to big-data using a structured language, wherein the programmatic conventions select a subset of information regarding millions of records as those records are generated, and wherein the stream application triggers actions every few milliseconds based on traditional program statements applied every hundred microseconds;

assigning, one or more processing cycles to the one or more segments of software code embedded in the at least one tuple of the stream of tuples;

determining, by the one or more processing cycles, a condition is met; and

performing, based on the determined condition and by the one or more processing cycles, an operation on the stream application.

2. The method of claim 1, wherein the condition is based on the processing elements of the stream application.

3. The method of claim 2, wherein the condition is selected from the group consisting of availability of a given processing element, activity of a given processing element, activity of a given stream operator, latency of a given processing element, and an association between the compute nodes and the processing elements.

4. The method of claim 1, wherein the condition includes two or more criteria, and wherein the determining the condition is met includes determining the two or more criteria.

5. The method of claim 1, wherein the condition is based on the stream of tuples to be processed by the plurality of processing elements.

6. The method of claim 1, wherein the condition is based on the one or more embedded tuples.

7. The method of claim 1 further comprising the following:

transmitting, by the one or more processing cycles, a notification of the performed operation to a user of the stream application.

8. The method of claim 1 further comprising the following:

transmitting, by the one or more processing cycles, a notification of the performed operation to an application other than the stream application.

9. The method of claim 1 wherein the stream application embeds the one or more tuples of the stream of tuples with the one or more segments of software code.

10. The method of claim 1, wherein the one or more processing cycles are from a computer outside of the stream application.

11. A system for processing a stream of tuples, the stream of tuples located in a stream-based computing environment designed to process big-data continuously and in real-time, the system comprising:

a plurality of processing elements configured to receive a stream of tuples, the tuples of the big-data set being processed before being recorded into a final destination, each processing element having one or more stream operators;

two or more processors; and

a memory containing an application that, when executed, causes at least one of the two or more processors to perform a method comprising;

embedding, by a first processor, at least one tuple of the stream of tuples with one or more segments of software code, wherein the stream application utilizes programmatic conventions to provide context to big-data using a structured language, wherein the programmatic conventions select a subset of information regarding millions of records as those records are generated, and wherein the stream application triggers actions every few milliseconds based on traditional program statements applied every hundred microseconds;

determining, by a second processor and based on the one or more segments of software code embedded within the at least one tuple, a condition is met; and

performing, by the second processor and based on the determined condition and based on the one or more segments of software code embedded within the at least one tuple, an operation.

12. The system of claim 11, wherein the first processor and the second processor are the same processor.

13. The system of claim 11, wherein the condition includes two or more criteria, and wherein the determining the condition is met includes determining the two or more criteria.

14. The system of claim 11, wherein the condition is based on the stream of tuples to be processed by the plurality of processing elements.

15. The system of claim 11, wherein the condition is based on the processing elements of the stream application.

16. A computer program product for processing a stream of tuples, the stream of tuples located in a stream-based computing environment designed to process big-data continuously and in real-time, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor, the program instructions cause the processor to perform a method comprising:

receiving, by a stream application, a stream of tuples including a plurality of data tuples embedded with non-software data and at least one software tuple embedded with segments of software code, wherein the non-software data is processed by software of stream operators operating on a portion of processing cycles of hardware of compute nodes assigned to the stream application, the tuples of the big-data set being processed before being recorded into a final destination, wherein the stream application utilizes programmatic conventions to provide context to big-data using a structured language, wherein the programmatic conventions select a subset of information regarding millions of records as those records are generated, and wherein the stream application triggers actions every few milliseconds based on traditional program statements applied every hundred microseconds;

assigning, by the stream application, a second portion of the hardware processing cycles to each embedded segment of software code of the at least one software tuple such that the at least one software tuple operates outside of the stream operators;

determining, by the stream application and based on the second portion of hardware processing cycles, a condition is met; and

performing, by the stream application and based on the second portion of hardware processing cycles and based on the determined condition, an operation.

17. The computer program product of claim 16, wherein the condition is based on the stream of tuples to be processed by the plurality of processing elements.

18. The computer program product of claim 16, wherein the program instructions further comprise the following:

transmitting a notification of the performed operation to a user.

19. The computer program product of claim 16, wherein the program instructions further comprise the following:

transmitting a notification of the performed operation to another application.