Coupling processors to each other for high performance computing (HPC)

• US 8,335,909 B2
• Filed: 04/15/2004
• Issued: 12/18/2012
• Est. Priority Date: 04/15/2004
• Status: Active Grant

• Current Assignee: Raytheon Company (Rtx Corporation)
• Original Assignee: Raytheon Company (Rtx Corporation)
• Inventors: Ballew, James D., Early, Gary R.
• Primary Examiner(s): Huisman, David
• Assistant Examiner(s): Geib, Benjamin P
• Application Number:
  US10/824,874
• Publication Number:
  US 20050235092A1
• Time in Patent Office: 3,169 Days
• Field of Search: 712/10, 712/11, 710/316
• US Class Current: 712/10
• CPC Class Codes: G06F 13/4022 using switching circuits, e.g. switching matrix, connection or expansion network G06F13/4009 takes precedence
• CPC Class Codes: G06F 15/80 comprising an array of processing units with common control, e.g. single instruction multiple data processors G06F15/82 takes precedence ; for correlation function computation G06F17/15
• CPC Class Codes: G06F 15/803 Three-dimensional arrays or hypercubes
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451

Abstract

A High Performance Computing (HPC) node comprises a motherboard, a switch comprising eight or more ports integrated on the motherboard, and at least two processors operable to execute an HPC job, with each processor communicably coupled to the integrated switch and integrated on the motherboard.

Claims

1. A system comprising a plurality of interconnected nodes, each node comprising:

a first motherboard;

at least two first processors integrated onto the first motherboard and configured to communicate with each other via a direct link between them; and

a first switch integrated onto the first motherboard, the first processors communicably coupled to the first switch, the first switch configured to communicably couple the first processors to at least six second motherboards that each comprise at least two second processors integrated onto the second motherboard and a second switch integrated onto the second motherboard configured to communicably couple the second processors to the first motherboard and at least five third motherboards that each comprise at least two third processors integrated onto the third motherboard and a third switch integrated onto the third motherboard;

the first processors configured to communicate with particular second processors on a particular second motherboard via the first switch and the second switch on the particular second motherboard;

the first processors configured to communicate with particular third processors on a particular third motherboard via the first switch, the second switch on an intermediate second motherboard between the first motherboard and the particular third motherboard, and the third switch on the particular third motherboard without communicating via either of the second processors on the intermediate second motherboard.

2. The system of claim 1, wherein a Host Channel Adapter (HCA) couples the first processors to the first switch.

3. The system of claim 2, wherein a peripheral component interconnect (PCI) bridge further couples the first processors to the first switch.

4. The system of claim 1, wherein, at each of one or more of the nodes, a link supporting processor-to-processor communication communicably couples the first processors directly to each other.

5. The system of claim 1, wherein a Northbridge communicably couples the first processors to the first switch.

6. The system of claim 5, each node comprising at least two Northbridges, each Northbridge communicably coupling one of the first processors to the first switch.

7. The system of claim 1, wherein the first switch is configured to communicate input/output (I/O) messages at a bandwidth that is equal to a processing speed of the first processors.

8. The system of claim 1, wherein the first switch comprises twenty-four ports and enables a toroidal topology comprising four dimensions.

9. The system of claim 1, wherein the plurality of nodes are arranged in a topology enabled by the first switch of each of the plurality of nodes.

10. The system of claim 9, wherein the topology comprises a hypercube.

11. The system of claim 9, wherein the topology comprises a folded topology.

12. The system of claim 1, wherein a first node of the plurality of nodes is interconnected to a second node of the plurality of nodes along an X axis, a third node of the plurality of nodes along a Y axis that is perpendicular to the X axis, a fourth node of the plurality of nodes along a Z axis that is perpendicular to the X and Y axes, and a fifth node along a diagonal axis that is oblique to one or more of the X, Y, or Z axes.

13. The system of claim 12, wherein the connection between the first node and the fifth node is configured to reduce message jumps among the plurality of nodes.

14. The system of claim 1, wherein a first node of the plurality of nodes is interconnected to a second node, a third node, a fourth node, and a fifth node, the first node being identical to the second, third, fourth, and fifth nodes, the first node being configured to communicate with each of the second, third, fourth, and fifth nodes via the interconnections.

15. A method comprising:

integrating at least two first processors onto a first motherboard, the first processors being configured to communicate with each other via a direct link between them; and

integrating a first switch onto the first motherboard and coupling the first switch to the first processors, the first processors communicably coupled to the first switch, the first switch configured to communicably couple the first processors to at least six second motherboards that each comprise at least two second processors integrated onto the second motherboard and a second switch integrated onto the second motherboard configured to communicably couple the second processors to the first motherboard and at least five third motherboards that each comprise at least two third processors integrated onto the third motherboard and a third switch integrated onto the third motherboard, the first processors configured to communicate with particular second processors on a particular second motherboard via the first switch and the second switch on the particular second motherboard, the first processors configured to communicate with particular third processors on a particular third motherboard via the first switch, the second switch on an intermediate second motherboard between the first motherboard and the particular third motherboard, and the third switch on the particular third motherboard without communicating via either of the second processors on the intermediate second motherboard.

16. The method of claim 15, wherein coupling the first switch to the first processors comprises coupling each one of the first processors to the first switch through a Host Channel Adapter (HCA).

17. The method of claim 16, wherein coupling the first switch to the first processors comprises coupling each one of the first processors to the first switch through a peripheral component interconnect (PCI) bridge.

18. The method of claim 15, further comprising coupling at least two of the first processors directly to each other via a link supporting processor-to-processor communication.

19. The method of claim 15, wherein coupling the first processors to the first switch comprises coupling each one of the first processors to the first switch through a Northbridge.

20. The method of claim 19, wherein the first motherboard comprises at least two Northbridges, each Northbridge communicably coupling one of the first processors to the first switch.

21. The method of claim 15, wherein the first switch is configured to communicate input/output (I/O) messages at a bandwidth that is equal to a processing speed of the first processors.

22. The method of claim 15, wherein the first switch comprises twenty-four ports and enables a toroidal topology comprising four dimensions.

23. A node comprising:

a first motherboard;

at least two first processors integrated onto the first motherboard and configured to communicate with each other via a direct link between them; and

a first switch integrated onto the first motherboard, the first processors communicably coupled to the first switch, the first switch configured to communicably couple the first processors to at least six second motherboards that each comprise at least two second processors integrated onto the second motherboard and a second switch integrated onto the second motherboard configured to communicably couple the second processors to the first motherboard and at least five third motherboards that each comprise at least two third processors integrated onto the third motherboard and a third switch integrated onto the third motherboard;

the first processors configured to communicate with particular second processors on a particular second motherboard via the first switch and the second switch on the particular second motherboard;

the first processors configured to communicate with particular third processors on a particular third motherboard via the first switch, the second switch on an intermediate second motherboard between the first motherboard and the particular third motherboard, and the third switch on the particular third motherboard without communicating via either of the second processors on the intermediate second motherboard.

24. The node of claim 23, further comprising an input port and an output port.

25. The node of claim 23, wherein a Host Channel Adapter (HCA) couples the first processors to the first switch.

26. The node of claim 25, wherein a peripheral component interconnect (PCI) bridge further couples the first processors to the first switch.

27. The node of claim 23, wherein a link supporting processor-to-processor communication communicably couples the first processors directly to each other.

28. The node of claim 23, wherein a Northbridge communicably couples the first processors to the first switch.

29. The node of claim 28, comprising at least two Northbridges, each Northbridge communicably coupling one of the first processors to the first switch.

30. The node of claim 23, wherein the first switch is configured to communicate input/output (I/O) messages at a bandwidth that is equal to a processing speed of the first processors.

31. The node of claim 23, wherein the first switch comprises twenty-four ports and enables a toroidal topology comprising four dimensions.

32. The node of claim 23, wherein the first switch is configured to communicate a first message from a first one of the first processors and a second message from a second one of the first processors in parallel.