Point backbones for network deployment

• US 8,880,739 B1
• Filed: 05/19/2010
• Issued: 11/04/2014
• Est. Priority Date: 05/19/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Brar, Jagwinder Singh, Marr, Michael David, Lamoreaux, Tyson J., Kelly, Mark N.
• Primary Examiner(s): Tiv, Backhean
• Application Number:
  US12/783,281
• Time in Patent Office: 1,630 Days
• Field of Search: 709249-252, 370256-258
• US Class Current: 709/250
• CPC Class Codes: H04Q 3/0062 Provisions for network management

Abstract

The efficiency of scaling of a network of computing devices can be improved by grouping highly-connected portions into deployment units wherein devices only have to be aware of other devices in the respective deployment unit. The various deployment units can be connected by a backbone mechanism. In some embodiments, the backbone utilizes a torroidal connection scheme to connect the backbone switches and pass routing protocol information. Host traffic can still utilize connections between tiers, such that standard routing and networking protocols can be utilized. In other embodiments, logical interfaces can be initialized in each backbone switch, such that each switch itself can function as a point backbone. The devices of the various deployment units then can communicate with each other via the point backbones using standard networking and routing protocols.

Claims

1. A computer-implemented method of connecting computing devices, comprising:

under control of one or more computer systems configured with executable instructions,

grouping a plurality of switches of a network deployment into a plurality of deployment units, each deployment unit structured as an individual highly-connected network comprising a first tier of edge switches each connected to each of a second tier of spine switches, each edge switch being further connected to a respective set of host computing devices, the edge switches and spine switches being connected using only connections between tiers;

connecting each of the deployment units to an external network using a common backbone tier comprising a plurality of backbone switches, connections for network traffic between the host computing devices and the external network being only between-tier connections; and

connecting all of the plurality of deployment units using a logical interface on each of the backbone switches within the common backbone tier, wherein the logical interface enables one or more host computing devices of any deployment unit to communicate with at least one host computing device in each of all other deployment units using only between-tier connections according to conventional routing protocols, such that at least a portion of the network traffic between the host computing devices and the external network is routed along a path that enters the host computing devices of at least two of the plurality of the deployment units by using the logical interface.

2. The computer-implemented method of claim 1, wherein each logical interface comprises a tunneling-based interface.

3. The computer-implemented method of claim 1, wherein at least one deployment unit comprises a Clos network, fat tree network, high radix network, butterfly network, or dragonfly network.

4. A computer-implemented method of connecting computing devices, comprising:

under control of one or more computer systems configured with executable instructions,

grouping switches into a plurality of deployment units, each deployment unit structured as an individual network comprising a first tier of edge switches each connected to each of a second tier of spine switches, each edge switch comprising ports for connection to a respective set of computing devices;

connecting the plurality of deployment units to an external network using a backbone tier comprising a plurality of backbone switches; and

connecting the plurality of deployment units using at least one logical interface on at least one of the plurality of backbone switches, the logical interface on the at least one backbone switch operable to act as a point backbone wherein one or more computing devices of any deployment unit are able to communicate with at least one computing device in each of all other deployment units via the point backbone using conventional routing protocols, such that at least a portion of network traffic between the computing devices and the external network enters the host computing devices of at least two of the plurality of the deployment units by using the logical interface.

5. The computer-implemented method of claim 4, wherein each of the plurality of edge switches and the plurality of spine switches is a commodity network switch or application-specific integrated circuit (ASIC).

6. The computer-implemented method of claim 4, wherein the edge switches and spine switches of each deployment unit are connected using only connections between tiers.

7. The computer-implemented method of claim 4, wherein connections for network traffic between the computing devices and the external network are only between-tier connections.

8. The computer-implemented method of claim 4, wherein each of the plurality of backbone switches comprises at least one logical interface for connecting the deployment units.

9. The computer-implemented method of claim 4, wherein each edge switch connected to a set of host devices is a top of rack (TOR) switch connected to a plurality of host devices in a network rack.

10. The computer-implemented method of claim 4, wherein a number of spine and edge switches for a first deployment unit is different than a number of spine and edge switches for a second deployment unit.

11. The computer-implemented method of claim 4, wherein the backbone tier is connected to the external network using a high radix interconnection network.

12. The computer-implemented method of claim 4, wherein at least one of the deployment units is a Clos network, fat tree network, high radix network, butterfly network, or dragonfly network.

13. A data center, comprising:

a plurality of computing devices each operable to receive a request and perform a computing operation;

a plurality of switches grouped into a plurality of deployment units, each deployment unit structured as an individual network comprising a first tier of edge switches each connected to each of a second tier of spine switches, each edge switch being further connected to a respective set of the plurality of computing devices, the edge switches and spine switches being connected using only connections between tiers; and

a backbone tier comprising a plurality of backbone switches, the backbone switches connecting the plurality of deployment units to an external network such that network traffic passes between the computing devices and the external network using only between-tier connections, the plurality of backbone switches each comprising at least one logical interface for connecting the plurality of deployment units, each logical interface of each backbone switch operable to act as a point backbone wherein one or more computing devices of any deployment unit are able to communicate with at least one computing device in each of all other deployment units via the point backbone using conventional routing protocols, such that the logical interface is configured to route at least a portion of network traffic between the computing devices and the external network to enter at least two of the plurality of the deployment units.

14. The data center of claim 13, wherein each logical interface comprises a tunneling-based interface.

15. The data center of claim 13, wherein at least one of the deployment units is a Clos network, fat tree network, high radix network, butterfly network, or dragonfly network.

16. The data center of claim 13, further comprising a highly-connected aggregation fabric connecting the backbone tier to the external network.

17. A system for determining connections for computing devices, comprising:

a processor; and

a memory device including instructions that, when executed by the processor, cause the processor to;

group switches into a plurality of deployment units, each deployment unit structured as an individual network comprising a first tier of edge switches each connected to each of a second tier of spine switches, each edge switch comprising ports for connection to a respective set of computing devices;

determine first connections to connect the plurality of deployment units to an external network using a backbone tier comprising a plurality of backbone switches; and

provide at least one logical interface on each of the backbone switches enabling each of the backbone switches to function as a point backbone connecting the plurality of logical areas, wherein one or more computing devices of any deployment unit are able to communicate with at least one computing device in each of all other deployment units via each point backbone using conventional routing protocols, such that the logical interface is configured to route at least a portion of network traffic between the computing devices and the external network to enter the computing devices of at least two of the plurality of the deployment units.

18. The system of claim 17, wherein each of the plurality of edge switches and the plurality of spine switches is a commodity network switch or application-specific integrated circuit (ASIC).

19. The system of claim 17, wherein each logical interface comprises a tunneling-based interface.

20. The system of claim 17, wherein at least one of the deployment units is a Clos network.

21. A non-transitory computer-readable storage medium storing instructions for determining connections for computing devices, the instructions when executed by a processor causing the processor to:

group switches into a plurality of deployment units, each deployment unit structured as an individual network comprising a first tier of edge switches each connected to each of a second tier of spine switches, each edge switch comprising ports for connection to a respective set of computing devices;

determine first connections to connect the plurality of deployment units to an external network using a backbone tier comprising a plurality of backbone switches; and

provide at least one logical interface on each of the backbone switches enabling each of the backbone switches to function as a point backbone connecting the plurality of deployment units, wherein one or more computing devices of any deployment unit are able to communicate with at least one computing device in each of all other deployment units via each point backbone using conventional routing protocols, wherein the logical interface is configured to route at least a portion of network traffic between the computing devices and the external network to enter the computing devices of at least two of the plurality of the deployment units.

22. The non-transitory computer-readable storage medium of claim 21, wherein each of the plurality of edge switches and the plurality of spine switches is a commodity network switch or application-specific integrated circuit (ASIC).

23. The non-transitory computer-readable storage medium of claim 21, wherein each logical interface comprises a tunneling-based interface.

24. The non-transitory computer-readable storage medium of claim 21, wherein at least one of the deployment units is a Clos network.