Using cloud resources to improve performance of a streaming application

• US 9,379,950 B2
• Filed: 11/07/2013
• Issued: 06/28/2016
• Est. Priority Date: 11/07/2013
• Status: Expired due to Fees

• Current Assignee: AirBNB Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Bragstad, Lance, Branson, Michael J., Cao, Bin, Carey, James E., Odden, Mathew R.
• Primary Examiner(s): Lazaro, David
• Assistant Examiner(s): Khurshid, Zia
• Application Number:
  US14/074,121
• Publication Number:
  US 20150127808A1
• Time in Patent Office: 964 Days
• Field of Search: 709/224
• US Class Current: 1/1
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5096 wherein the managed service relates to distributed or central networked applications
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

A streams manager monitors performance of a streaming application, and when the performance needs to be improved, the streams manager automatically requests virtual machines from a cloud manager. The streams manager specifies to the cloud manager streams infrastructure and one or more streams application components for the virtual machines. The cloud manager provisions one or more virtual machines in a cloud with the specified streams infrastructure and streams application components. The streams manager then modifies the flow graph so one or more portions of the streaming application are hosted by the virtual machines in the cloud.

Claims

1. A computer-implemented method executed by at least one processor for managing a streaming application, the method comprising:

executing a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples;

monitoring performance of the streaming application;

when performance of the streaming application needs to be improved, requesting a cloud manager to provision at least one virtual machine with logic to implement at least one of the plurality of operators, wherein the request to the cloud manager specifies;

a number of virtual machines to provision;

for each of the number of virtual machines to provision;

a specified streams infrastructure to provision; and

at least one specified component of the streams application to provision in the virtual machine; and

after the cloud manager provisions the at least one virtual machine, modifying the flow graph to include the at least one virtual machine in the flow graph of the streaming application.

2. The method of claim 1 wherein the monitoring the performance of the streaming application comprises comparing current performance of the streaming application to at least one defined performance threshold.

3. The method of claim 2 wherein comparing the performance of the streaming application to the at least one defined threshold determines when the performance of the streaming application needs to be improved.

4. The method of claim 1 wherein the logic to implement the at least one of the plurality of operators comprises:

software that provides a streaming infrastructure; and

software running in the streaming infrastructure that implements the at least one of the plurality of operators.

5. The method of claim 1 wherein the at least one virtual machine implements logic for a selected one of the plurality of operators, wherein the at least one virtual machine processes tuples in parallel with the selected one operator in the flow graph after the flow graph of the streaming application is modified.

6. The method of claim 5 further comprising modifying the flow graph to include a new operator that splits tuples from an existing operator to the selected one operator in the flow graph and to the at least one virtual machine that processes the tuples in parallel with the selected one operator in the flow graph.

7. A computer-implemented method executed by at least one processor for managing a streaming application, the method comprising:

executing a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples;

monitoring performance of the streaming application by comparing current performance of the streaming application to at least one defined performance threshold;

determining when performance of the streaming application needs to be improved by comparing the current performance of the streaming application with the at least one defined performance threshold;

submitting a request to a cloud manager to provision at least one virtual machine, the request comprising;

a number of virtual machines to provision;

a specified streams infrastructure to provision; and

at least one specified component of the streams application;

the cloud manager provisioning the at least one virtual machine to implement logic for a selected one of the plurality of operators; and

after the cloud manager provisions the at least one virtual machine, modifying the flow graph to include;

a new operator that splits tuples from an existing operator to the selected one operator in the flow graph and to the at least one virtual machine; and

the at least one virtual machine in the flow graph of the streaming application so the virtual machine processes tuples in parallel with the selected one operator in the flow graph after the flow graph of the streaming application is modified.