Replicating a virtual machine implementing parallel operators in a streaming application based on performance

• US 10,467,041 B2
• Filed: 05/15/2018
• Issued: 11/05/2019
• Est. Priority Date: 10/10/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bragstad, Lance, Branson, Michael J., Cao, Bin, Carey, James E., Odden, Mathew R.
• Primary Examiner(s): Zhao, Bing
• Application Number:
  US15/980,720
• Publication Number:
  US 20180267824A1
• Time in Patent Office: 539 Days
• Field of Search: None
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/45575 Starting, stopping, suspending or resuming virtual machine instances
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H05K 999/99 dummy group

Abstract

Performance of parallel portions of a streaming application implemented in multiple virtual machines (VMs) is monitored and logged. When there is a need to replicate operators in an existing virtual machine to a new virtual machine, the logged performance data is used to determine a desired configuration for the new virtual machine based on a past implementation of the existing virtual machine.

Claims

1. An apparatus comprising:

at least one processor;

a memory coupled to the at least one processor;

a streaming application residing in the memory and executed by the at least one processor, the streaming application comprising a flow graph that includes a plurality of operators that process a plurality of data tuples, wherein the flow graph comprises a first set of operators implemented in a first virtual machine and a second set of operators implemented in a second virtual machine, wherein the first set of operators and the second set of operators include similar operators and connections between operators and operate on data tuples from a first data stream in parallel and produce data tuples that are sent to a same operator in the streaming application; and

a streams manager residing in the memory and executed by the at least one processor, wherein the streams manager monitors performance of the streaming application, logs performance data for each virtual machine that includes operators in the flow graph, and when there is a need to replicate operators from the first virtual machine to a new virtual machine, uses the logged performance data to determine a desired configuration for the new virtual machine that implements a new set of operators in the streaming application that operate on data tuples from a second data stream in the streaming application in parallel with operators in at least one existing virtual machine,

wherein the streams manager determines from the performance data a past implementation for the first virtual machine that had the best performance, and allocates resources to the new virtual machine according to the past implementation for the first virtual machine that had the best performance to create the desired configuration for the new virtual machine.

3. The apparatus of claim 1 wherein the first data stream comprises the second data stream.

2. A computer-implemented method executed by at least one processor for managing a streaming application, the method comprising:

executing a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples, wherein the flow graph comprises a first set of operators implemented in a first virtual machine and a second set of operators implemented in a second virtual machine, wherein the first set of operators and the second set of operators include similar operators and connections between operators and operate on data tuples from a first data stream in parallel and produce data tuples that are sent to a same operator in the streaming application;

monitoring performance of the streaming application;

logging performance data for each virtual machine that includes operators in the flow graph;

when there is a need to replicate operators to a new virtual machine, using the logged performance data to determine a desired configuration for the new virtual machine that implements a new set of operators in the streaming application that operate on data tuples from a second data stream in the streaming application in parallel with operators in at least one existing virtual machine;

determining from the performance data a past implementation for the first virtual machine that had the best performance; and

allocating resources to the new virtual machine according to the past implementation for the first virtual machine that had the best performance to create the desired configuration for the new virtual machine.

4. The method of claim 2 wherein the first data stream comprises the second data stream.