Moving a portion of a streaming application to a public cloud based on sensitive data

• US 10,061,938 B2
• Filed: 11/15/2017
• Issued: 08/28/2018
• Est. Priority Date: 09/10/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Barsness, Eric L., Branson, Michael J., Santosuosso, John M.
• Primary Examiner(s): Zaidi, Syed
• Application Number:
  US15/813,994
• Publication Number:
  US 20180082080A1
• Time in Patent Office: 286 Days
• Field of Search: 726 28
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 21/6245 Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G06F 21/6254 by anonymising data, e.g. decorrelating personal data from the owner's identification
• CPC Class Codes: G06F 9/455 Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: H04L 41/5025 by proactively reacting to service quality change, e.g. by reconfiguration after service quality degradation or upgrade
• CPC Class Codes: H04L 41/5054 Automatic deployment of services triggered by the service manager, e.g. service implementation by automatic configuration of network components
• CPC Class Codes: H04L 41/509 wherein the managed service relates to media content delivery, e.g. audio, video or TV
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display

Abstract

A streams manager determines which portions of a streaming application process sensitive data, and when performance of the streaming application needs to be increased, selects based on the sensitive data which portion(s) of the streaming application can be moved to a public cloud. The streams manager then interacts with the public cloud manager to move the selected portion(s) of the streaming application to the public cloud. This may include cloning of processing elements or operators to a public cloud, then splitting tuple attributes so tuple attributes that do not include sensitive data can be processed in the public cloud while tuple attributes that include sensitive data are processed in a secure system. The tuple attributes are then recombined into full tuples in the secure system. The streams manager thus protects the integrity of sensitive data while still taking advantage of the additional resources available in a public cloud.

Claims

1. An apparatus comprising:

at least one processor;

a memory coupled to the at least one processor;

a streaming application residing in the memory and executed by the at least one processor, the streaming application comprising a flow graph that includes a plurality of processing elements that process a plurality of data tuples;

a performance monitor residing in the memory and executed by the at least one processor, the performance monitor monitoring performance of the plurality of processing elements in the flow graph;

a sensitive data monitor that identifies which of the plurality of processing elements in the flow graph process sensitive data; and

a streams manager that manages the streaming application, the streams manager selecting based on which of the plurality of processing elements in the flow graph process sensitive data at least one portion of the flow graph to move to a public cloud when the performance monitor determines performance of the streaming application needs to be improved, and in response, the streams manager moves the selected at least one portion of the flow graph to the public cloud by creating at least one cloned portion of the flow graph in the public cloud, routes a first set of the plurality of tuple attributes to a portion of the flow graph in a secure system, routes a second set of the plurality of tuple attributes to the at least one cloned portion of the flow graph in the public cloud, and recombines the first set of the plurality of tuple attributes and the second set of the plurality of tuple attributes into the selected tuple in the secure system.

2. The apparatus of claim 1 further comprising at least one sensitive data split criterion that specifies how to split the flow graph based on sensitive data so at least a portion of the flow graph can be moved to the public cloud in a manner that assures the sensitive data is not in the public cloud, wherein the at least one sensitive data split criterion specifies to split a plurality of tuple attributes corresponding to a selected data tuple between the first set of the plurality of tuple attributes that include sensitive data and the second set of the plurality of tuple attributes that do not include sensitive data.

3. The apparatus of claim 2 wherein the at least one sensitive data split criterion specifies to move to the public cloud connected processing elements in the flow graph that do not process sensitive data.

4. The apparatus of claim 2 wherein the at least one sensitive data split criterion specifies to move to the public cloud individual processing elements in the flow graph that do not process sensitive data.

5. The apparatus of claim 2 wherein the at least one sensitive data split criterion specifies to move to the public cloud connected processing elements in the flow graph that process sensitive data that can have a data input encrypted and a data output decrypted.

6. The apparatus of claim 2 wherein the at least one sensitive data split criterion specifies to unfuse one of the plurality of processing elements into a plurality of operators so at least one of the plurality of operators that does not process sensitive data is moved to the public cloud while at least one of the plurality of operators that process sensitive data is not moved to the public cloud.

7. The apparatus of claim 1 wherein the streams manager moves the selected at least one portion of the flow graph to the public cloud by requesting a public cloud manager to provision at least one virtual machine with logic to implement the selected at least one portion of the flow graph, and when the public cloud manager provisions the at least one virtual machine, the streams manager modifies the flow graph to include the at least one virtual machine in the flow graph of the streaming application.

8. The apparatus of claim 1 wherein the streams manager determines when the performance of the streaming application needs to be improved by comparing current performance of the streaming application to at least one performance threshold.

9. The apparatus of claim 1 wherein the streams manager determines when the performance of the streaming application needs to be improved by comparing current performance of the streaming application to historical performance of the streaming application.

16. The method of claim 9 wherein moving the selected at least one portion of the flow graph to the public cloud is performed by requesting a public cloud manager to provision at least one virtual machine with logic to implement the selected at least one portion of the flow graph, and when the public cloud manager provisions the at least one virtual machine, the streams manager modifies the flow graph to include the at least one virtual machine in the flow graph of the streaming application.

17. The method of claim 9 wherein determining when the monitored performance indicates performance of the streaming application needs to be improved is performed by comparing current performance of the streaming application to at least one performance threshold.

18. The method of claim 9 wherein determining when the performance of the streaming application needs to be improved is performed by comparing current performance of the streaming application to historical performance of the streaming application.

10. A computer-implemented method executed by at least one processor for executing a streaming application, the method comprising:

executing the streaming application comprising a flow graph that includes a plurality of processing elements that process a plurality of data tuples;

monitoring performance of the plurality of processing elements in the flow graph;

identifying which of the plurality of processing elements in the flow graph process sensitive data; and

selecting based on which of the plurality of processing elements in the flow graph process sensitive data at least one portion of the flow graph to move to the public cloud when the monitored performance indicates performance of the streaming application needs to be improved, and in response, moving the selected at least one portion of the flow graph to the public cloud by creating at least one cloned portion of the flow graph in the public cloud, routing the first set of the plurality of tuple attributes to a portion of the flow graph in a secure system, routing the second set of the plurality of tuple attributes to the at least one cloned portion of the flow graph in the public cloud, and recombining the first set of the plurality of tuple attributes and the second set of the plurality of tuple attributes into the selected tuple in the secure system.

11. The method of claim 10 further comprising defining at least one sensitive data split criterion that specifies how to split the flow graph based on sensitive data so at least a portion of the flow graph can be moved to the public cloud in a manner that assures the sensitive data is not in the public cloud, wherein the at least one sensitive data split criterion specifies to split a plurality of tuple attributes corresponding to a selected data tuple between the first set of the plurality of tuple attributes that include sensitive data and the second set of the plurality of tuple attributes that do not include sensitive data.

12. The method of claim 11 wherein the at least one sensitive data split criterion specifies to move to the public cloud connected processing elements in the flow graph that do not process sensitive data.

13. The method of claim 11 wherein the at least one sensitive data split criterion specifies to move to the public cloud individual processing elements in the flow graph that do not process sensitive data.

14. The method of claim 11 wherein the at least one sensitive data split criterion specifies to move to the public cloud connected processing elements in the flow graph that process sensitive data that can have a data input encrypted and a data output decrypted.

15. The method of claim 11 wherein the at least one sensitive data split criterion specifies to unfuse one of the plurality of processing elements into a plurality of operators so at least one of the plurality of operators that does not process sensitive data is moved to the public cloud while at least one of the plurality of operators that process sensitive data is not moved to the public cloud.

19. An article of manufacture comprising software stored on a computer readable storage medium, the software comprising:

a streaming application comprising a flow graph that includes a plurality of processing elements that process a plurality of data tuples;

a performance monitor that monitors performance of the plurality of processing elements in the flow graph;

a sensitive data monitor that identifies which of the plurality of processing elements in the flow graph process sensitive data; and

a streams manager that manages the streaming application, the streams manager selecting based on which of the plurality of processing elements in the flow graph process sensitive data at least one portion of the flow graph to move to a public cloud when the performance monitor determines performance of the streaming application needs to be improved, and in response, the streams manager moves the selected at least one portion of the flow graph to the public cloud by creating at least one cloned portion of the flow graph in the public cloud, routes a first set of the plurality of tuple attributes to a portion of the flow graph in a secure system, routes a second set of the plurality of tuple attributes to the at least one cloned portion of the flow graph in the public cloud, and recombines the first set of the plurality of tuple attributes and the second set of the plurality of tuple attributes into the selected tuple in the secure system.

20. The article of manufacture of claim 19 further comprising at least one sensitive data split criterion that specifies how to split the flow graph based on sensitive data so at least a portion of the flow graph can be moved to the public cloud in a manner that assures the sensitive data is not in the public cloud, wherein the at least one sensitive data split criterion specifies to split a plurality of tuple attributes corresponding to a selected data tuple between the first set of the plurality of tuple attributes that include sensitive data and the second set of the plurality of tuple attributes that do not include sensitive data.