Contention free pipelined broadcasting within a constant bisection bandwidth network topology

• US 8,274,987 B2
• Filed: 03/22/2010
• Issued: 09/25/2012
• Est. Priority Date: 03/22/2010
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Jia, Bin
• Primary Examiner(s): Sheikh, Ayaz
• Assistant Examiner(s): LEE, ANDREW CHUNG CHEUNG
• Application Number:
  US12/728,747
• Publication Number:
  US 20110228789A1
• Time in Patent Office: 918 Days
• Field of Search: 370386-394, 370400-401
• US Class Current: 370/401
• CPC Class Codes: H04L 45/00 Routing or path finding of packets in data switching networks routing or path finding in wireless networks H04W40/00
• CPC Class Codes: H04L 49/101 using crossbar or matrix
• CPC Class Codes: H04L 49/201 Multicast operation; Broadcast operation

Abstract

In an interconnection network, multiple nodes are connected to one of a first layer of switches. The first layer of switches is connected to one another through a second layer of switches. Each of the nodes is connected through one of multiple shared links connecting the first layer switches and the second layer of switches. A pipelined broadcast manager schedules a hierarchical pipelined broadcast through at least one switch of the first layer switches comprising non-root nodes by selecting two nodes among the non-root nodes connected to the at least one switch and scheduling each of multiple broadcast steps for the pipelined broadcast with at least one of an inter-switch broadcast phase and an intra-switch broadcast phase using the selected two nodes.

Claims

1. A parallel computing system, comprising:

a plurality of nodes each of which comprises at least one processor and at least one adapter;

an interconnection network comprising a first plurality of switches wherein each of the plurality of nodes is connected to one of the first plurality of switches and wherein each of the first plurality of switches is connected through a second plurality of switches, wherein each of the plurality of nodes is connected through a plurality of shared links connecting the first plurality of switches and the second plurality of switches;

a user space layer distributed in at least one of the plurality of nodes comprising a message passing interface (MPI) layer for receiving a message passing interface broadcast command from an application, wherein the message passing interface broadcast command triggers an MPI broadcast for passing a plurality of data packets from one of the plurality of nodes ranked as a root node to a selection of non-root nodes from among the plurality of nodes;

the MPI layer for triggering a pipelined broadcast manager to schedule a hierarchical pipelined broadcast for the MPI broadcast; and

the pipelined broadcast manager for scheduling the hierarchical pipelined broadcast through at least one switch of the first plurality of switches from among a plurality of non-root nodes by selecting two nodes of the plurality of non-root nodes connected to the at least one switch and scheduling each of a plurality of broadcast steps for the hierarchical pipelined broadcast with at least one of an inter-switch broadcast phase with a first node of the two nodes receiving a first data packet from another switch from among the first plurality of switches and a second node of the two nodes sending a second data packet previously received from the another switch to one other switch from among the plurality of switches and an intra-switch broadcast phase with the first node acting as a source for sending a previously received data packet to at least one other non-root node and the second node acting as a sink for receiving the previously received data packet, wherein each of the non-root nodes sends and receives the previously received data packet once throughout the plurality of broadcast steps and the sink receives the previously received data packet last, wherein for each next step of the plurality of broadcast steps the first node and the second node alternate roles.

2. The parallel computing system according to claim 1 further comprising:

the interconnection network further comprising a subnet manager for setting a table comprising each separate single, static route set between each of the plurality of nodes through the plurality of shared links; and

the pipelined broadcast manager for scheduling each of the plurality of broadcast steps for the pipelined broadcast for routing according to the table.

3. The parallel computing system according to claim 1 further comprising:

the pipelined broadcast manager for scheduling the hierarchical pipelined broadcast for hardware unicast wherein each data packet of a plurality of data packets broadcast during the hierarchical pipelined broadcast is routed to reach one destination.

4. The parallel computing system according to claim 1 wherein the at least one adapter further comprises at least one of a host channel adapter for connecting a processing node to the interconnection network and a target channel adapter for connecting an input/output node to the interconnection network.

5. The parallel computing system according to claim 1, further comprising:

the pipelined broadcast manager, for the hierarchical pipelined broadcast through the at least one switch which is a non-root switch from among the first plurality of switches of a broadcast and which requires an even number of steps for a data packet to be transferred from the source to the sink during the intra-switch broadcast phase, for scheduling a delayed step in the sink receiving a particular packet, wherein during a broadcast step p from among the plurality of broadcast steps the sink receives the particular data packet that should have reached the sink node during a step p-1 from among the plurality of broadcast steps.

6. The parallel computing system according to claim 1, further comprising:

the pipelined broadcast manager, for the hierarchical pipelined broadcast through another switch of the first plurality of switches comprising only a single node attached to the another switch, for scheduling only the inter-switch broadcast phrase for each of the plurality of broadcast steps for the single node of the another switch.

7. The parallel computing system according to claim 1, further comprising:

the pipelined broadcast manager, for the hierarchical pipelined broadcast through a root switch from among the first plurality of switches comprising a selection of only two nodes attached from among the plurality of nodes, for scheduling an actual root node originating the hierarchical pipelined broadcast from among the selection of two nodes to send each separate data packet during a separate broadcast step from among the plurality of steps to the other non-root node from among the selection of two nodes and for scheduling the other non-root node for the inter-switch broadcast phases only and functioning as a root node.

8. The parallel computing system according to claim 1, further comprising:

the pipelined broadcast manager, for the hierarchical pipelined broadcast through a root switch from among the first plurality of switches comprising a selection of more than two nodes attached from among the plurality of nodes, for scheduling a root node originating the hierarchical pipelined broadcast from among the selection of more than two nodes for the inter-switch broadcast phase only during the plurality of broadcast steps as if the root node is connecting to one switch and scheduling the other non-root nodes from among the selection of more than two nodes as if the other non-root nodes are connected to another switch separate from the root node.

9. A method for pipelined broadcasting within an interconnection network, comprising:

connecting communicatively a plurality of nodes each of which comprises at least one adapter through an interconnection network comprising a first plurality of switches, wherein each of the plurality of nodes is connected to one of the first plurality of switches and wherein each of the first plurality of switches is connected through a second plurality of switches, wherein each of the plurality of nodes is connected through a plurality of shared links connecting the first plurality of switches and the second plurality of switches;

implementing a user space layer distributed in at least one of the plurality of nodes comprising a message passing interface (MPI) layer for receiving a message passing interface broadcast command from an application, wherein the message passing interface broadcast command triggers an MPI broadcast for passing a plurality of data packets from one of the plurality of nodes ranked as a root node to a selection of non-root nodes from among the plurality of nodes;

triggering, by the MPI layer, scheduling of a hierarchical pipelined broadcast for the MPI broadcast; and

scheduling, using a processor, the hierarchical pipelined broadcast through at least one switch of the first plurality of switches from among a plurality of non-root nodes by;

selecting, using the processor, two nodes of the plurality of non-root nodes connected to the at least one switch; and

scheduling, using the processor, each of a plurality of broadcast steps for the hierarchical pipelined broadcast with at least one of an inter-switch broadcast phase with a first node of the two nodes receiving a first data packet from another switch from among the first plurality of switches and a second node of the two nodes sending a second data packet previously received from the another switch to one other switch from among the plurality of switches and an intra-switch broadcast phase with the first node acting as a source for sending a previously received data packet to at least one other non-root node and the second node acting as a sink for receiving the previously received data packet, wherein each of the non-root nodes sends and receives the previously received data packet once throughout the plurality of broadcast steps and the sink receives the previously received data packet last, wherein for each next step of the plurality of broadcast steps the first node and the second node alternate roles.

10. The method according to claim 9 further comprising:

setting a table comprising each separate single, static route set between each of the plurality of nodes through the plurality of shared links by a subnet manager within the interconnection network; and

scheduling each of the plurality of broadcast steps for the pipelined broadcast for routing according to the table.

11. The method according to claim 9 further comprising:

scheduling the hierarchical pipelined broadcast for hardware unicast wherein each data packet of a plurality of data packets broadcast during the hierarchical pipelined broadcast is routed to reach one destination.

12. The method according to claim 9, wherein the at least one adapter further comprises at least one of a host channel adapter for connecting a processing node to the interconnection network and a target channel adapter for connecting an input/output node to the interconnection network.

13. The method according to claim 9, further comprising:

scheduling, for the hierarchical pipelined broadcast through the at least one switch which is a non-root switch from among the first plurality of switches of a broadcast and which requires an even number of steps for a data packet to be transferred from the source to the sink during the intra-switch broadcast phase, a delayed step in the sink receiving a particular packet, wherein during a broadcast step p from among the plurality of broadcast steps the sink receives the particular data packet that should have reached the sink node during a step p-1 from among the plurality of broadcast steps.

14. The method according to claim 9, further comprising:

scheduling, for the hierarchical pipelined broadcast through another switch of the first plurality of switches comprising only a single node attached to the another switch, only the inter-switch broadcast phrase for each of the plurality of broadcast steps for the single node of the another switch.

15. The method according to claim 9, further comprising:

scheduling, for the hierarchical pipelined broadcast through a root switch from among the first plurality of switches comprising a selection of only two nodes attached from among the plurality of nodes, an actual root node originating the hierarchical pipelined broadcast from among the selection of two nodes to send each separate data packet during a separate broadcast step from among the plurality of steps to the other non-root node from among the selection of two nodes and for scheduling the other non-root node for the inter-switch broadcast phases only and functioning as a root node.

16. The method according to claim 9, further comprising:

scheduling, for the hierarchical pipelined broadcast through a root switch from among the first plurality of switches comprising a selection of more than two nodes attached from among the plurality of nodes, a root node originating the hierarchical pipelined broadcast from among the selection of more than two nodes for the inter-switch broadcast phase only during the plurality of broadcast steps as if the root node is connecting to one switch and scheduling the other non-root nodes from among the selection of more than two nodes as if the other non-root nodes are connected to another switch separate from the root node.

17. A computer program product for pipelined broadcasting within an interconnection network, the computer program product comprising:

one or more computer-readable, tangible storage devices;

program instructions, stored on at least one of the one or more storage devices, to connect communicatively a plurality of nodes each of which comprises at least one adapter through an interconnection network comprising a first plurality of switches, wherein each of the plurality of nodes is connected to one of the first plurality of switches and wherein each of the first plurality of switches is connected through a second plurality of switches, wherein each of the plurality of nodes is connected through a plurality of shared links connecting the first plurality of switches and the second plurality of switches;

program instructions, stored on at least one of the one or more storage devices, to implement a user space layer distributed in at least one of the plurality of nodes comprising a message passing interface (MPI) layer for receiving a message passing interface broadcast command from an application, wherein the message passing interface broadcast command triggers an MPI broadcast for passing a plurality of data packets from one of the plurality of nodes ranked as a root node to a selection of non-root nodes from among the plurality of nodes;

program instructions, stored on at least one of the one or more storage devices, to trigger, by the MPI layer, scheduling of a hierarchical pipelined broadcast for the MPI broadcast; and

program instructions, stored on at least one of the one or more storage devices, to schedule the hierarchical pipelined broadcast through at least one switch of the first plurality of switches from among a plurality of non-root nodes by selecting two nodes of the plurality of non-root nodes connected to the at least one switch and scheduling each of a plurality of broadcast steps for the hierarchical pipelined broadcast with at least one of an inter-switch broadcast phase with a first node of the two nodes receiving a first data packet from another switch from among the first plurality of switches and a second node of the two nodes sending a second data packet previously received from the another switch to one other switch from among the plurality of switches and an intra-switch broadcast phase with the first node acting as a source for sending a previously received data packet to at least one other non-root node and the second node acting as a sink for receiving the previously received data packet, wherein each of the non-root nodes sends and receives the previously received data packet once throughout the plurality of broadcast steps and the sink receives the previously received data packet last, wherein for each next step of the plurality of broadcast steps the first node and the second node alternate roles.