Connecting multiple customer sites over a wide area network using an overlay network

• US 9,621,460 B2
• Filed: 01/08/2014
• Issued: 04/11/2017
• Est. Priority Date: 01/14/2013
• Status: Active Grant

• Current Assignee: Versa Networks, Inc.
• Original Assignee: Versa Networks, Inc.
• Inventors: Mehta, Apurva, Shenoy, Shivaprakash, Ramakrishnan, Ramanarayanan, Sankaran, Krishna
• Primary Examiner(s): Sefcheck, Gregory
• Assistant Examiner(s): Esmaeilian, Majid
• Application Number:
  US14/150,640
• Publication Number:
  US 20140198794A1
• Time in Patent Office: 1,189 Days
• Field of Search: 370392
• CPC Class Codes: H04L 12/4633 Interconnection of networks using encapsulation techniques, e.g. tunneling
• CPC Class Codes: H04L 45/04 Interdomain routing, e.g. hierarchical routing
• CPC Class Codes: H04L 45/28 using route fault recovery
• CPC Class Codes: H04L 45/74 Address processing for routing
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]

Abstract

A method and apparatus for connecting multiple customer sites over a wide area network (WAN) using an overlay network is described. In one embodiment of the invention, each one of multiple customer edge (CE) routers establishes a Border Gateway Protocol (BGP) session with one or more BGP route reflectors and announces their private IP network prefixes and one or more transport IP addresses to reach that CE router. The BGP route reflector(s) reflect those IP network prefixes and the one or more transport IP addresses to reach that specific CE router to the other CE routers. The CE routers receive those reflected IP network prefixes and the corresponding transport IP address(es) to reach that CE router in which those IP network prefixes belong and register them in their corresponding routing/forwarding data structures. In this way, the CE routers learn how to reach each other.

Claims

1. A method for connecting a plurality of customer sites over a wide area network (WAN) of a service provider, wherein each of the plurality of customer sites includes a customer edge router, the method comprising:

each of the customer edge routers establishing a Border Gateway Protocol (BGP) peer session with one or more route reflectors;

each of the customer edge routers announcing the following to the one or more route reflectors;

one or more local routes that are local to the customer site in which that customer edge router belongs;

one or more tunnel IP addresses of one or more interfaces of that customer edge router, wherein the one or more interfaces of that customer edge router respectively connect that customer edge router to a set of one or more provider edge routers in the WAN; and

a preferred tunnel type and tunnel specific information to reach that customer edge router, wherein the tunnel specific information includes one or more of an encryption key and a password, and wherein the preferred tunnel type and tunnel specific information facilitate automatically setting up secure communication between that customer edge router and other customer edge routers;

the one or more route reflectors reflecting the one or more local routes, one or more tunnel IP addresses, and the preferred tunnel type and tunnel specific information received from each of the customer edge routers to the other ones of the customer edge routers;

each of the customer edge routers installing the following that is received from the one or more route reflectors from a particular customer edge router;

the one or more local routes, the corresponding one or more tunnel IP addresses of one or more interfaces of that particular customer edge router for which those one or more local routes belong, and the preferred tunnel type and tunnel specific information to reach that particular customer edge router;

a first customer edge router performing the following in response to receiving a first packet from a first customer device that corresponds to a first local route in a first one of the plurality of customer sites, wherein the first packet is destined for a second customer device that corresponds to a second local route in a second one of the plurality of customer sites;

determining that a second customer edge router provides reachability for the second local route,

encapsulating the first packet within a second packet resulting in a first encapsulated packet for traversing the WAN of the service provider to reach the second customer edge router, wherein the second packet has a destination IP address that is one of the one or more tunnel IP addresses of one or more interfaces of the second customer edge router, and wherein the second packet has a source IP address that is one of the one or more tunnel IP addresses of one or more interfaces of the first customer edge router, and

transmitting the first encapsulated packet over the WAN of the service provider towards the second customer edge router for decapsulation and delivery to the second customer device; and

the second customer edge router performing the following in response to receiving the first encapsulated packet;

decapsulating the first encapsulated packet to reveal the first packet,

determining, from the first packet, that the second customer device is a destination of the first packet, and

transmitting the first packet towards the second customer device.

2. The method of claim 1, wherein the one or more tunnel IP addresses of one or more interfaces of that customer edge router are announced as one of:

a Network Layer Reachability Information (NLRI) utilizing a Sub Address Family Indicator (SAFI), an Extended Community, and a transitive BGP attribute.

3. The method of claim 1, wherein the one or more tunnel IP addresses announced by at least one of the customer edge routers includes a primary address and one or more secondary addresses.

4. The method of claim 3, wherein the at least one of the customer edge routers indicates a priority of the one or more tunnel IP addresses.

5. A method in a first customer edge router for connecting a first customer site with a set of one or more other customer sites over a wide area network (WAN) of a service provider, wherein the first customer edge router is connected to a set of one or more provider edge routers of the service provider, and wherein the first customer edge router is within the first customer site, the method comprising:

transmitting reachability information to one or more route reflectors, wherein the transmitted reachability information includes the following;

one or more tunnel IP addresses of one or more interfaces of the first customer edge router, wherein the one or more interfaces of the first customer edge router respectively connect the first customer edge router to the set of one or more provider edge routers of the service provider;

one or more network prefixes that are reachable through the first customer edge router; and

a preferred tunnel type and tunnel specific information to reach the first customer edge router, wherein the tunnel specific information includes one or more of an encryption key and a password, and wherein the preferred tunnel type and tunnel specific information facilitate automatically setting up secure communication between the first customer edge router with a set of one or more different customer edge routers in the set of one or more other customer sites over the WAN;

receiving, from the one or more route reflectors, reachability information to reach one or more of the set of one or more other customer sites, wherein each of the set of one or more other customer sites includes the set of one or more different customer edge routers, wherein each of the received reachability information includes the following;

one or more tunnel IP addresses of one or more interfaces of a different customer edge router, wherein the one or more interfaces of the different customer edge router respectively connect the different customer edge router to a set of one or more provider edge routers of the service provider;

one or more network prefixes that are reachable through that different customer edge router; and

the preferred tunnel type and tunnel specific information to reach that different customer edge router; and

installing the reachability information received from the one or more route reflectors in one or more forwarding structures;

receiving a first packet from a first customer device in the first customer site that is destined for a second customer device in one of the set of one or more other customer sites;

determining, through accessing the one or more forwarding structures, which one of the set of one or more different customer edge routers provides reachability for the second customer device;

encapsulating the first packet within a second packet resulting in a first encapsulated packet for traversing the WAN of the service provider to reach the customer edge router determined to provide reachability for the second customer device, wherein the second packet has a destination IP address that is one of the one or more tunnel IP addresses of one or more interfaces of the customer edge router determined to provide reachability for the second customer device that was received from the one or more route reflectors, and wherein the second packet has a source IP address that is one of the one or more tunnel IP addresses of one or more interfaces of the first customer edge router;

transmitting the first encapsulated packet over the WAN of the service provider towards the customer edge router determined to provide reachability for the second customer device for decapsulation and delivery to the second customer device;

receiving a second encapsulated packet, wherein the second encapsulation packet includes a third packet encapsulated within a fourth packet, wherein the fourth packet has a destination IP address that is one of the one or more tunnel IP addresses of one or more interfaces of the first customer edge router, and wherein the fourth packet has a source IP address that is one of the one or more tunnel IP addresses of one or more interfaces of the one of the set of one or more different customer edge routers;

decapsulating the second encapsulated packet to reveal the third packet;

determining a destination network prefix from the third packet, wherein the destination network prefix is one of the one or more network prefixes that are reachable through the first customer edge router; and

transmitting the third packet towards the destination network prefix.

6. The method of claim 5, further comprising:

establishing a Border Gateway Protocol (BGP) peer session with the one or more route reflectors prior to transmitting the reachability information to the one or more route reflectors.

7. The method of claim 5, wherein the one or more tunnel IP addresses of one or more interfaces of the first customer edge router included in the transmitted reachability information are announced as one of:

a Network Layer Reachability Information (NLRI) utilizing a Sub Address Family Indicator (SAFI), an Extended Community, and a transitive BGP attribute.

8. The method of claim 5, further comprising:

monitoring the one or more interfaces of the first customer edge router for failure;

responsive to determining that one of the one or more interfaces of the first customer edge router is experiencing a failure, transmitting a reachability update to the one or more route reflectors that indicates that the tunnel IP address of that interface is experiencing a failure and should be withdrawn.

9. The method of claim 8, wherein the reachability update is one of a Border Gateway Protocol (BGP) withdraw message, and a BGP update message.

10. The method of claim 5, further comprising:

wherein the received reachability information for at least one of the set of one or more different customer edge routers includes at least two tunnel IP addresses and specifies at least a primary tunnel IP address and a secondary tunnel IP address;

receiving, from one of the one or more route reflectors, an update that reflects a change affecting reachability to the at least one of the set of one or more different customer edge routers; and

responsive to determining that the at least one of the set of one or more different customer edge routers is not reachable through its primary tunnel IP address, updating the one or more forwarding structures to use the secondary tunnel IP address to reach that customer edge router.

11. The method of claim 10, wherein the received update is one of a Border Gateway Protocol (BGP) withdraw message, and a BGP update message.

12. The method of claim 5, further comprising:

monitoring, with an end-to-end monitoring mechanism, a primary tunnel between a first one of the one or more interfaces of the first customer edge router and a first one of the one or more interfaces of one of the set of different customer edge routers; and

responsive to determining that the tunnel is experiencing a failure, using a secondary tunnel between a second one of the one or more interfaces of the first customer edge router and a second one of the one or more interfaces of one of the set of different customer edge routers.

13. A customer edge router to connect a first customer site with a set of one or more other customer sites over a wide area network (WAN) of a service provider, wherein the customer edge router is to be connected to a set of one or more provider edge routers of the service provider, and wherein the customer edge router is to be within the first customer site, the customer edge router comprising:

a processor to execute instructions, and

at least one non-transitory machine-readable storage medium coupled with the processor, the medium storing one or more instructions that, when executed, cause the processor to perform the following;

transmit reachability information to one or more route reflectors, wherein the reachability information includes the following;

one or more tunnel IP addresses of one or more interfaces of the customer edge router, wherein the one or more interfaces of the customer edge router respectively connect the customer edge router to the set of one or more provider edge routers of the service provider,

one or more network prefixes that are reachable through the customer edge router, and

a preferred tunnel type and tunnel specific information to reach the customer edge router, wherein the tunnel specific information includes one or more of an encryption key and a password, and wherein the preferred tunnel type and tunnel specific information facilitate automatically setting up secure communication between the customer edge router with a set of one or more different customer edge routers in the set of one or more other customer sites over the WAN;

receive, from the one or more route reflectors, reachability information to reach one or more of the set of one or more other customer sites, wherein each of the set of one or more other customer sites includes the set of one or more different customer edge routers, wherein each of the received reachability information includes the following;

one or more tunnel IP addresses of one or more interfaces of a different customer edge router, wherein the one or more interfaces of the different customer edge router respectively connect the different customer edge router to a set of one or more provider edge routers of the service provider;

one or more network prefixes that are reachable through that different customer edge router; and

the preferred tunnel type and tunnel specific information to reach that different customer edge router; and

install the reachability information to be received from the one or more route reflectors in one or more forwarding structures;

receive a first packet from a first customer device in the first customer site that is destined for a second customer device in one of the set of one or more other customer sites;

determine, through an access of the one or more forwarding structures, which one of the set of one or more different customer edge routers provides reachability for the second customer device;

encapsulate the first packet within a second packet resulting in a first encapsulated packet for traversing the WAN of the service provider to reach the customer edge router determined to provide reachability for the second customer device, wherein the second packet has a destination IP address that is one of the one or more tunnel IP addresses of one or more interfaces of the customer edge router determined to provide reachability for the second customer device that was received from the one or more route reflectors, and wherein the second packet has a source IP address that is one of the one or more tunnel IP addresses of one or more interfaces of the customer edge router;

transmit the first encapsulated packet over the WAN of the service provider towards the customer edge router determined to provide reachability for the second customer device for decapsulation and delivery to the second customer device;

receive a second encapsulated packet, wherein the second encapsulation packet includes a third packet encapsulated within a fourth packet, wherein the fourth packet has a destination IP address that is one of the one or more tunnel IP addresses of one or more interfaces of the customer edge router, and wherein the fourth packet has a source IP address that is one of the one or more tunnel IP addresses of one or more interfaces of the one of the set of one or more different customer edge routers;

decapsulate the second encapsulated packet to reveal the third packet;

determine a destination network prefix from the third packet, wherein the destination network prefix is one of the one or more network prefixes that are reachable through the customer edge router; and

transmit the third packet towards the destination network prefix.

14. The customer edge router of claim 13, wherein the medium further stores one or more instructions that, when executed, cause the processor to perform the following:

establish a Border Gateway Protocol (BGP) peer session with the one or more route reflectors prior to transmission of the reachability information to the one or more route reflectors.

15. The customer edge router of claim 13, wherein the one or more tunnel IP addresses of one or more interfaces of the customer edge router included in the reachability information to be transmitted are announced as one of:

a Network Layer Reachability Information (NLRI) utilizing a Sub Address Family Indicator (SAFI), an Extended Community, and a transitive BGP attribute.