System and Method for Hardware Based Dynamic Load Balancing of Message Passing Interface Tasks
First Claim
1. A method, in a multiple processor system, for balancing a Message Passing Interface (MPI) workload across a plurality of processors, comprising:
- receiving one or more MPI synchronization operation calls from one or more processors of the plurality of processors;
identifying a first processor, in the plurality of processors, having a fastest time of completion of a computation phase of a first associated MPI task based on the received one or more MPI synchronization operation calls;
identifying a second processor, in the plurality of processors, having a slowest time of completion of a computation phase of a second associated MPI task based on the received one or more MPI synchronization operation calls; and
modifying a first amount of data to be processed by the first processor in executing the first associated MPI task in a next computation cycle to thereby increase the amount of data.
1 Assignment
0 Petitions
Accused Products
Abstract
A system and method for providing hardware based dynamic load balancing of message passing interface (MPI) tasks are provided. Mechanisms for adjusting the balance of processing workloads of the processors executing tasks of an MPI job are provided so as to minimize wait periods for waiting for all of the processors to call a synchronization operation. Each processor has an associated hardware implemented MPI load balancing controller. The MPI load balancing controller maintains a history that provides a profile of the tasks with regard to their calls to synchronization operations. From this information, it can be determined which processors should have their processing loads lightened and which processors are able to handle additional processing loads without significantly negatively affecting the overall operation of the parallel execution system. As a result, operations may be performed to shift workloads from the slowest processor to one or more of the faster processors.
63 Citations
20 Claims
-
1. A method, in a multiple processor system, for balancing a Message Passing Interface (MPI) workload across a plurality of processors, comprising:
-
receiving one or more MPI synchronization operation calls from one or more processors of the plurality of processors; identifying a first processor, in the plurality of processors, having a fastest time of completion of a computation phase of a first associated MPI task based on the received one or more MPI synchronization operation calls; identifying a second processor, in the plurality of processors, having a slowest time of completion of a computation phase of a second associated MPI task based on the received one or more MPI synchronization operation calls; and modifying a first amount of data to be processed by the first processor in executing the first associated MPI task in a next computation cycle to thereby increase the amount of data. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A computer program product comprising a computer useable medium having a computer readable program, wherein the computer readable program, when executed on a data processing system, causes the data processing system to:
-
receive one or more Message Passing Interface (MPI) synchronization operation calls from one or more processors of a plurality of processors; identify a first processor, in the plurality of processors, having a fastest time of completion of a computation phase of a first associated MPI task based on the received one or more MPI synchronization operation calls; identify a second processor, in the plurality of processors, having a slowest time of completion of a computation phase of a second associated MPI task based on the received one or more MPI synchronization operation calls; and modify a first amount of data to be processed by the first processor in executing the first associated MPI task in a next computation cycle to thereby increase the amount of data. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19)
-
-
20. A data processing system, comprising:
-
a plurality of processors; and at least one load balancing controller associated with the plurality of processors, wherein the at least one load balancing controller balances a Message Passing Interface (MPI) workload across the plurality of processors by; receiving one or more MPI synchronization operation calls from one or more processors of the plurality of processors; identifying a first processor, in the plurality of processors, having a fastest time of completion of a computation phase of a first associated MPI task based on the received one or more MPI synchronization operation calls; identifying a second processor, in the plurality of processors, having a slowest time of completion of a computation phase of a second associated MPI task based on the received one or more MPI synchronization operation calls; and modifying a first amount of data to be processed by the first processor in executing the first associated MPI task in a next computation cycle to thereby increase the amount of data.
-
Specification