Dragonfly processor interconnect network

• US 9,614,786 B2
• Filed: 12/27/2014
• Issued: 04/04/2017
• Est. Priority Date: 08/20/2008
• Status: Active Grant

• Current Assignee: Intel Corporation, Board of Trustees of the Leland Stanford Junior University (Stanford University)
• Original Assignee: Intel Corporation, Board of Trustees of the Leland Stanford Junior University (Stanford University)
• Inventors: Kim, John, Abts, Dennis C., Scott, Steven L., Dally, William J.
• Primary Examiner(s): Korsak, Oleg
• Application Number:
  US14/583,588
• Publication Number:
  US 20150186318A1
• Time in Patent Office: 829 Days
• CPC Class Codes: G06F 13/4027 using bus bridges G06F13/4022 takes precedence
• CPC Class Codes: G06F 13/4221 being an input/output bus, e.g. ISA bus, EISA bus, PCI bus, SCSI bus
• CPC Class Codes: G06F 15/17375 One dimensional, e.g. linear array, ring
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: H04L 45/28 using route fault recovery
• CPC Class Codes: H04L 45/58 Association of routers
• CPC Class Codes: H04L 49/15 Interconnection of switching modules
• CPC Class Codes: H04L 49/1515 Non-blocking multistage, e.g. Clos
• CPC Class Codes: H04L 49/70 Virtual switches

Abstract

A multiprocessor computer system comprises a dragonfly processor interconnect network that comprises a plurality of processor nodes, a plurality of routers, each router directly coupled to a plurality of terminal nodes, the routers coupled to one another and arranged into a group, and a plurality of groups of routers, such that each group is connected to each other group via at least one direct connection.

Claims

1. An apparatus comprising:

a routing device to support adaptive routing of data in a multiprocessor network comprising a Dragonfly network topology, wherein the routing of the data is to be based on one or more routing tables, the Dragonfly network topology is to comprise a plurality of groups, one or more of the groups is to comprise a respective plurality of routers and is to couple by respective global channels to the other groups in the plurality of groups, and a respective one of the routers is to connect to one or more processor nodes in the network,

wherein the number of processor nodes per router in the Dragonfly network topology is greater than the number of global channels per router.

2. The apparatus of claim 1, wherein the apparatus is a switch or a router and the routing device is a chip.

3. The apparatus of claim 1, wherein at least one of the plurality of groups is to function as a virtual router in the network.

4. The apparatus of claim 3, wherein each of the plurality of groups is to function as a respective virtual router in the network.

5. The apparatus of claim 1, wherein the routing device is to further monitor congestion associated with at least one channel in the network.

6. The apparatus of claim 5, wherein adaptive routing decisions are to be based at least in part on the congestion monitoring.

7. The apparatus of claim 6, wherein the routing device is to route data using credit round-trip latency as an indicator of channel congestion.

8. The apparatus of claim 6, wherein the channel comprises a global channel.

9. The apparatus of claim 1, wherein the Dragonfly network topology comprises at least one virtual channel.

10. The apparatus of claim 1, wherein the network interconnects at least two sub-networks.

11. The apparatus of claim 1, wherein the effective radix of each group is the product of the number of routers in each group times the sum of the number of processor nodes connected to each router plus the number of global channels in the network.

12. The apparatus of claim 1, wherein the number of routers per group is equal to twice the number of processor nodes per router, and the number of processor nodes per router is equal to the number of channels per router connected to other groups.

13. The apparatus of claim 1, wherein the network comprises a number of processor nodes and the virtual router has a radix of approximately 2 times the square root of the number of processor nodes.

14. The apparatus of claim 1, wherein the plurality of routers in at least one of the groups is greater than twice the number of global channels per router.

15. The apparatus of claim 1, wherein the group comprises a sub-network having a flattened butterfly network topology.

16. The apparatus of claim 1, wherein the routing device is to route data using selective virtual channel discrimination.

17. The apparatus of claim 1, wherein the one or more tables comprises at least one global table for routing the data between groups of the plurality of groups.

18. The apparatus of claim 1, wherein the one or more tables comprises at least one local table used for routing the data within a group of the plurality of groups.

19. A method comprising:

detecting congestion of at least one channel in a multiprocessor network comprising a Dragonfly network topology;

performing adaptive routing of data within the network based on the congestion; and

using one or more routing tables corresponding to the Dragonfly network,

wherein the Dragonfly network topology comprises a plurality of groups, each respective group in the plurality of groups comprises a plurality of routers and is coupled by respective global channels to the other groups in the plurality of groups, and respective of the routers are connected to one or more processor nodes in the network,

wherein the number of processor nodes per router in the Dragonfly network topology is greater than the number of global channels per router.

20. A system comprising:

a plurality of processor nodes; and

a plurality of router devices, wherein each router is coupled to a respective subset of the plurality of processor nodes, each router is included in a respective one of a plurality of router groups, each group has a respective group topology and is connected via a respective global channel to each other group in the plurality of router groups, at least one of the routers is to perform routing within the system using adaptive routing, and the routing is to be performed based at least in part on one or more routing tables,

wherein the number of processor nodes per router in the Dragonfly network topology is greater than the number of global channels per router.

21. The system of claim 20, wherein each of the plurality of router groups is to function as a virtual router in the network.

22. The system of claim 20, further comprising one or more local channels to interconnect routers within a respective router group.

23. The system of claim 22, wherein the adaptive routing is based on congestion determined for the at least one global channel.

24. The system of claim 23, wherein the congestion of the global channel is determined from congestion detected at a local channel.

25. The system of claim 20, wherein the plurality of processor nodes comprises at least one hundred thousand processor nodes.

26. The system of claim 20, wherein each of the router groups comprises a sub-network with a respective topology.

27. The system of claim 26, wherein the topology of at least one of the router groups comprises a flattened butterfly topology.

28. The system of claim 20, wherein the one or more tables comprises at least one global table for routing the data between groups of the plurality of groups.

29. The system of claim 20, wherein the one or more tables comprises at least one local table used for routing the data within a group of the plurality of groups.