Deploying operators of a streaming application based on physical location attributes of a virtual machine

• US 10,075,515 B2
• Filed: 03/14/2017
• Issued: 09/11/2018
• Est. Priority Date: 03/06/2014
• Status: Expired due to Fees

• Current Assignee: AirBNB Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Bragstad, Lance, Branson, Michael J., Cao, Bin, Carey, James E., Odden, Mathew R.
• Primary Examiner(s): Tokuta, Shean
• Application Number:
  US15/458,466
• Publication Number:
  US 20170187796A1
• Time in Patent Office: 546 Days
• CPC Class Codes: H04L 41/083 for increasing network speed
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 41/342 between virtual entities, e.g. orchestrators, SDN or NFV entities
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5054 Automatic deployment of services triggered by the service manager, e.g. service implementation by automatic configuration of network components
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04L 67/1004 Server selection for load balancing
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1012 based on compliance of requirements or conditions with available server resources
• CPC Class Codes: H04L 67/1021 based on client or server locations
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

A streams manager monitors operator performance of a streaming application to determine when the performance of an operator needs to be improved or optimized. The streams manager in conjunction with a cloud manager automatically determines one or more preferred virtual machines in a cloud with a specified streams infrastructure that best meet the needs of the underperforming operator or application component based on physical location attributes of the preferred virtual machines. The cloud manager determines the physical location attributes of the candidate virtual machines. The streams manager or the cloud manager can then determine a preferred virtual machine of the candidates to deploy the operator based on the physical location attributes. The streams manager then modifies the flow graph so one or more operators of the streaming application are deployed to a preferred virtual machine determined according to the physical location attributes of the preferred virtual machine.

Claims

1. An apparatus comprising:

at least one processor;

a memory coupled to the at least one processor;

a streaming application residing in the memory and executed by the at least one processor, the streaming application comprising a flow graph that includes a plurality of operators that process a plurality of data tuples; and

a streams manager residing in the memory and executed by the at least one processor, the streams manager monitoring performance of the streaming application, and when the streams manager determines an under-performing operator relative to other operators and the under-performing operator of the streaming application can be optimized, sending a request to a cloud manager to provision at least one virtual machine, wherein the request specifies a location restriction that provides input for the cloud manger to determine a physical location for a preferred virtual machine based on physical location attributes of the preferred virtual machine;

the cloud manager determines physical locations attributes of candidate virtual machines and determines the preferred virtual machine from the candidate virtual machines based on the physical location attributes in the request, the cloud manager provisions the preferred virtual machine; and

wherein the streams manager modifies the flow graph to move the under-performing operator of the streaming application to the preferred virtual machine determined by the cloud manager based on physical location attributes of the preferred virtual machine meeting the location restriction sent in the request

wherein the streams manager sends a notification to the cloud manager that the flow graph has been modified to make the cloud manager aware of the change for load balancing; and

wherein the physical location attributes comprise;

climate, cooling cost, data regulation environment, and a security attribute of the physical machine hosting the virtual machine that indicates a high level of security for the physical location hosting the virtual machine.

2. The apparatus of claim 1 wherein the streams manager monitors the performance of the streaming application and determines the under-performing operator by comparing current performance of the streaming application to at least one defined performance threshold.

3. The apparatus of claim 1 wherein the location restriction indicates an operator processes politically sensitive data that must be performed in a specific political environment and instructs the cloud manager to select a VM with a physical location attribute that meets the location restriction for the specific nature of the sensitive data processed by the operator.

4. The apparatus of claim 1 wherein the streams manager determines the preferred virtual machine is not already available and requests the cloud manager to provision the preferred virtual machine in the cloud.

5. The apparatus of claim 1 wherein the streams manager communicates with a plurality of cloud managers in a multi-cloud environment.

6. A computer-implemented method executed by at least one processor for managing a streaming application, the method comprising:

executing a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples;

monitoring performance of the streaming application;

when an operator is determined to be under-performing relative to other operators and can be optimized, sending a request to a cloud manager to provision at least one virtual machine, wherein the request specifies a location restriction that provides input for the cloud manger to determine a physical location for a preferred virtual machine based on physical location attributes of the preferred virtual machine,

requesting a cloud manager to provision at least one virtual machine with logic to implement at least one of the plurality of operators;

the cloud manager determining physical locations attributes of candidate virtual machines;

the cloud manager determining the preferred virtual machine from the candidate virtual machines based on the physical location attributes;

the cloud manager provisioning the preferred virtual machine; and

modifying the flow graph to move the under-performing operator to a preferred virtual machine based on physical location attributes of the preferred virtual machine meeting the location restriction sent in the request

wherein the streams manager sends a notification to the cloud manager that the flow graph has been modified to make the cloud manager aware of the change for load balancing; and

wherein the physical location attributes comprise;

climate, cooling cost, data regulation environment, and a security attribute of the physical machine hosting the virtual machine that indicates a high level of security for the physical location hosting the virtual machine.

7. The method of claim 6 wherein the monitoring the performance of the streaming application comprises comparing current performance of the streaming application to at least one defined performance threshold.

8. The method of claim 6 wherein the location restriction indicates an operator processes politically sensitive data that must be performed in a specific political environment and instructs the cloud manager to select a VM with a physical location attribute that meets the location restriction for the specific nature of the sensitive data processed by the operator.

9. The method of claim 6 further comprising determining the preferred virtual machine is not already available and requesting the cloud manager to provision the preferred virtual machine in the cloud.

10. An apparatus comprising:

at least one processor;

a memory coupled to the at least one processor;

a streaming application residing in the memory and executed by the at least one processor, the streaming application comprising a flow graph that includes a plurality of operators that process a plurality of data tuples; and

a streams manager residing in the memory and executed by the at least one processor, the streams manager executing the streaming application, monitoring performance of the streaming application by comparing current performance of the streaming application to at least one defined performance threshold, determining an operator in the flow graph is under-performing relative to other operators and can be optimized, sending a request to a cloud manager to provision at least one virtual machine with logic to implement at least one of the plurality of operators, wherein the request specifies a location restriction that provides input for the cloud manger to determine a physical location for a preferred virtual machine based on physical location attributes of the preferred virtual machine, wherein the physical location attributes comprise climate, cooling cost, data regulation environment, and a security attribute of the physical machine hosting the virtual machine that indicates a high level of security for the physical location hosting the virtual machine, the cloud manager determining physical locations attributes of candidate virtual machines, the cloud manager determining the preferred virtual machine from the candidate virtual machines based on the physical location attributes, the cloud manager provisioning the preferred virtual machine at the determined physical location based on the physical location attributes of the preferred virtual machine, the streams manager modifying the flow graph to deploy the under-performing operator of the streaming application into the preferred virtual machine determined based on physical location attributes of the preferred virtual machine meeting the location restriction sent in the request, sending a notification to the cloud manager that the flow graph has been modified to make the cloud manager aware of the change for load balancing.