Moving a portion of a streaming application to a public cloud based on sensitive data

• US 10,129,311 B2
• Filed: 11/16/2017
• Issued: 11/13/2018
• Est. Priority Date: 08/21/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Barsness, Eric L., Branson, Michael J., Santosuosso, John M.
• Primary Examiner(s): Lynch, Sharon S
• Application Number:
  US15/815,701
• Publication Number:
  US 20180077213A1
• Time in Patent Office: 362 Days
• CPC Class Codes: G06F 21/60 Protecting data
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G06F 2221/2101 Auditing as a secondary aspect
• CPC Class Codes: H04L 41/042 comprising distributed management centres cooperatively managing the network
• CPC Class Codes: H04L 41/0896 Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities flow or congestion control using dynamic resource allocation, e.g. in-call renegotiation, H04L47/76
• CPC Class Codes: H04L 41/0897 by horizontal or vertical scaling of resources, or by migrating entities, e.g. virtual resources or entities
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 41/122 of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV]
• CPC Class Codes: H04L 43/026 using flow identification
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/205 involving negotiation or determination of the one or more network security mechanisms to be used, e.g. by negotiation between the client and the server or between peers or by selection according to the capabilities of the entities involved negotiation of communication capabilities H04L69/24
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H05K 999/99 dummy group

Abstract

A streams manager determines which portions of a streaming application process sensitive data, and when performance of the streaming application needs to be increased, selects based on the sensitive data which portion(s) of the streaming application can be moved to a public cloud. The streams manager then interacts with the public cloud manager to move the selected portion(s) of the streaming application to the public cloud. By taking sensitive data into account, the streams manager protects the integrity of sensitive data while still taking advantage of the additional resources available in a public cloud.

Claims

1. An apparatus comprising:

at least one processor;

a memory coupled to the at least one processor;

a streaming application residing in the memory and executed by the at least one processor, the streaming application comprising a flow graph that includes a plurality of processing elements that process a plurality of data tuples;

a performance monitor residing in the memory and executed by the at least one processor, the performance monitor monitoring performance of the plurality of processing elements in the flow graph;

a sensitive data monitor that identifies which of the plurality of processing elements that process a plurality of data tuples in the flow graph process sensitive data;

at least one sensitive data split criterion that specifies how to split the flow graph based on sensitive data so at least a portion of the flow graph can be moved to a public cloud in a manner that assures the sensitive data is not in the public cloud, wherein the at least one sensitive data split criterion specifies to move to the public cloud connected processing elements in the flow graph that process sensitive data when a data input to the connected processing elements in the flow graph that process sensitive data can be encrypted and a data output from the connected processing elements in the flow graph that process sensitive data can be decrypted; and

a streams manager residing in the memory and executed by the at least one processor, the streams manager managing the streaming application, the streams manager selecting based on the at least one sensitive data split criterion at least one portion of the flow graph to move to the public cloud when the performance monitor determines performance of the streaming application needs to be improved, and in response, the streams manager moving the selected at least one portion of the flow graph to the public cloud, adding a processing element in the streaming application to encrypt data tuples input to the selected at least one portion of the flow graph in the public cloud and adding a processing element in the streaming application to decrypt data tuples output from the selected at least one portion of the flow graph in the public cloud.

2. The apparatus of claim 1 wherein the at least one sensitive data split criterion specifies to move to the public cloud connected processing elements in the flow graph that do not process sensitive data.

3. The apparatus of claim 1 wherein the at least one sensitive data split criterion specifies to move to the public cloud individual processing elements in the flow graph that do not process sensitive data.

4. The apparatus of claim 1 wherein the at least one sensitive data split criterion specifies to unfuse one of the plurality of processing elements into a plurality of operators so at least one of the plurality of operators that does not process sensitive data is moved to the public cloud while at least one of the plurality of operators that process sensitive data is not moved to the public cloud.

5. The apparatus of claim 1 wherein the streams manager moves the selected at least one portion of the flow graph to the public cloud by requesting a public cloud manager to provision at least one virtual machine with logic to implement the selected at least one portion of the flow graph, and when the public cloud manager provisions the at least one virtual machine, the streams manager modifies the flow graph to include the at least one virtual machine in the flow graph of the streaming application.

6. The apparatus of claim 1 wherein the streams manager determines when the performance of the streaming application needs to be improved by comparing current performance of the streaming application to at least one performance threshold.

7. The apparatus of claim 1 wherein the streams manager determines when the performance of the streaming application needs to be improved by comparing current performance of the streaming application to historical performance of the streaming application.

8. A computer-implemented method executed by at least one processor for executing a streaming application, the method comprising:

executing the streaming application comprising a flow graph that includes a plurality of processing elements that process a plurality of data tuples;

monitoring performance of the plurality of processing elements in the flow graph;

identifying which of the plurality of processing elements in the flow graph process sensitive data;

defining at least one sensitive data split criterion that specifies how to split the flow graph based on sensitive data so at least a portion of the flow graph can be moved to a public cloud in a manner that assures the sensitive data is not in the public cloud, wherein the at least one sensitive data split criterion specifies to move to the public cloud connected processing elements in the flow graph that process sensitive data when a data input to the connected processing elements in the flow graph that process sensitive data can be encrypted and a data output from the connected processing elements in the flow graph that process sensitive data can be decrypted; and

selecting based on the at least one sensitive data split criterion at least one portion of the flow graph to move to the public cloud when performance of the streaming application needs to be improved, and in response;

moving the selected at least one portion of the flow graph to the public cloud;

adding a processing element in the streaming application to encrypt data tuples input to the selected at least one portion of the flow graph in the public cloud; and

adding a processing element in the streaming application to decrypt data tuples output from the selected at least one portion of the flow graph in the public cloud.

9. The method of claim 8 wherein the at least one sensitive data split criterion specifies to move to the public cloud connected processing elements in the flow graph that do not process sensitive data.

10. The method of claim 8 wherein the at least one sensitive data split criterion specifies to move to the public cloud individual processing elements in the flow graph that do not process sensitive data.

11. The method of claim 8 wherein the at least one sensitive data split criterion specifies to unfuse one of the plurality of processing elements into a plurality of operators so at least one of the plurality of operators that does not process sensitive data is moved to the public cloud while at least one of the plurality of operators that process sensitive data is not moved to the public cloud.

12. The method of claim 8 wherein moving the selected at least one portion of the flow graph to the public cloud comprises:

requesting a public cloud manager to provision at least one virtual machine with logic to implement the selected at least one portion of the flow graph;

modifying the flow graph to include the at least one virtual machine in the flow graph of the streaming application.

13. The method of claim 8 wherein determining the performance of the streaming application needs to be improved comprises comparing current performance of the streaming application to at least one performance threshold.

14. The method of claim 8 wherein determining the performance of the streaming application needs to be improved comprises comparing current performance of the streaming application to historical performance of the streaming application.