Using MPLS for virtual private cloud network isolation in openflow-enabled cloud computing

• US 8,560,663 B2
• Filed: 09/30/2011
• Issued: 10/15/2013
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: Telefonaktiebolaget LM Ericsson
• Original Assignee: Telefonaktiebolaget LM Ericsson
• Inventors: Baucke, Stephan, Green, Howard, Kempf, James, Tatipamula, Mallik
• Primary Examiner(s): Nguyen, Tammy
• Application Number:
  US13/251,107
• Publication Number:
  US 20130086236A1
• Time in Patent Office: 746 Days
• Field of Search: 709/223, 709/225, 709/238, 709/246, 370/255, 370/389
• US Class Current: 709/223
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 45/306 Route determination based on the nature of the carried application
• CPC Class Codes: H04L 45/50 using label swapping, e.g. multi-protocol label switch [MPLS]
• CPC Class Codes: H04L 45/54 Organization of routing tables
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 49/70 Virtual switches
• CPC Class Codes: H04L 61/103 across network layers, e.g. resolution of network layer into physical layer addresses or address resolution protocol [ARP]
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]

Abstract

Embodiments of the invention include a method performed by a cloud network manager flow entries in a cloud network. The CNM is coupled to virtualized servers for hosting virtual machines (“VM”) that each comprise a virtual switch coupled to a top of rack switch (“TORS”). The CNM receives notification messages that indicate virtual machines have been scheduled for activation on virtualized servers. The CNM determines a VM media access control (“MAC”) address associated with the VM and a virtual switch MAC address associated with that virtualized server'"'"'s virtual switch. The CNM records an association between the VM MAC address and the virtual switch MAC address. The CNM further determines a label that associates the TORS with the virtual switch. The CNM sends a modifies flow entries in the virtual switch to indicate data packets matching the MPLS label and the VM MAC address should be forwarded to the VM.

Claims

1. A method performed by a cloud network manager (“

CNM”

) to manage multi-protocol label switching (“

MPLS”

) flow entries in a cloud network, the CNM coupled to a first virtualized server for hosting one or more virtual machines (“

VM”

), wherein the first virtualized server comprises a first virtual switch that supports MPLS and the first virtual switch is coupled to a first top of rack switch (“

TORS”

) that supports MPLS, the method comprising the steps of;

receiving a first notification message, wherein the first notification message indicates that a first VM has been scheduled for activation on the first virtualized server; and

in response to receiving the first notification message, performing the following steps;

determining a first VM media access control (“

MAC”

) address associated with the first VM,

determining a first virtual switch MAC address associated with the first virtual switch,

recording, in an address mapping table, an association between the first VM MAC address and the first virtual switch MAC address,

determining a first MPLS label that associates the first TORS with the first virtual switch, and

sending a first flow entry modification message to the first virtual switch indicating that data packets matching the first MPLS label and the first VM MAC address should be forwarded to the first VM.

2. The method of claim 1, further comprising sending a second flow entry modification message to the first virtual switch in response to receiving the first notification message, the second flow entry modification message indicating that dynamic host control protocol (“

DHCP”

) messages originating from the first VM MAC address should be forwarded to the CNM.

3. The method of claim 2, further comprising the steps of:

receiving a DHCP request originating from the first VM;

relaying the DHCP request to a DHCP server;

receiving a DHCP reply from the DHCP server;

recording, in the address mapping table, an association between the first VM MAC address and a first internet protocol (“

IP”

) address included within the DHCP reply; and

forwarding the DHCP reply to the first VM.

4. The method of claim 1, further comprising sending a second flow entry modification message to the first virtual switch in response to receiving the first notification message, the second flow entry modification message indicating that address resolution protocol (“

ARP”

) messages originating from the first VM MAC address should be forwarded to the CNM.

5. The method of claim 4, further comprising the steps of:

receiving an ARP request originating from the first VM;

determining a destination IP address from the ARP request;

determining, from the address mapping table, a destination VM MAC address and a destination virtual switch MAC address each corresponding with the destination IP address;

determining a destination MPLS label corresponding with the destination virtual switch MAC address;

sending a third flow entry modification message to the first virtual switch indicating that data packets matching the destination VM MAC address should have the destination MPLS label pushed on and forwarded to the first TORS;

generating an ARP reply indicating the destination VM MAC address corresponds with the destination IP address; and

forwarding the ARP reply to the first VM.

6. The method of claim 1, further comprising the steps of:

receiving a second notification message, the second notification message indicating that the first VM is moving to a second virtualized server and is unable to service data packets, wherein the second virtualized server comprises a second virtual switch that supports MPLS and the second virtual switch is coupled to a second TORS that supports MPLS;

sending, in response to the second notification message, a second flow entry modification message to the first virtual switch indicating that data packets matching the first MPLS label and the first VM MAC address should be forwarded to the CNM;

receiving a move complete indication that indicates that the first VM has moved to the second virtualized server and is able to service data packets; and

in response to the move complete indication, performing the steps of;

determining a second virtual switch MAC address associated with the second virtual switch,

recording, in the address mapping table, an association between the first VM MAC address and the second virtual switch MAC address,

determining a second MPLS label that links the second TORS with the second virtual switch,

sending a third flow entry modification message to the second virtual switch indicating that data packets matching the second MPLS label and the first VM MAC address should be forwarded the first VM, and

updating the address mapping table to associate the first VM with the second virtual switch and disassociate the first VM from the first virtual switch.

7. The method of claim 6 further comprising the steps of:

causing one or more data packets destined for the first VM MAC address to be buffered in response to the second notification message; and

causing the one or more buffered data packets to be forwarded to the first VM in response to the move complete indication.

8. The method of claim 6, further comprising the steps of:

receiving a peer data packet matching the first MPLS label and the first VM MAC address, wherein the peer data packet originated from a peer VM hosted by a peer virtualized server comprising a peer virtual switch that supports MPLS and that is coupled to a peer TORS that supports MPLS;

determining a peer VM MAC address from the peer data packet;

determining the peer virtual switch from the peer VM MAC address;

forwarding the peer data packet to the first VM; and

sending a fourth flow entry modification message to the peer virtual switch indicating that data packets destined for the first VM MAC address should have the second MPLS label pushed on and forwarded to the peer TORS, whereby subsequent data packets from the peer VM destined for the first VM will be forwarded to the second virtual switch.

9. A system for managing a multi-protocol label switching (“

MPLS”

) cloud network comprising;

a plurality of top of rack switches (“

TORS”

);

a plurality of virtualized servers, each virtualized server coupled to one of the plurality of TORSs and comprising;

a hypervisor configured to execute one or more virtual machines (“

VM”

) on the virtualized server, and

a virtual switch configured to support MPLS and to transmit data packets from the one or more VMs to the TORS coupled with this virtualized server;

a cloud execution manager (“

CEM”

) coupled to the plurality of virtualized servers and configured to;

schedule execution of the one or more VMs, each VM to be executed on one of the plurality of virtualized servers,

suspend execution of the one or more VMs, and

move execution of the one or more VMs from one of the plurality of virtualized servers to another one of the plurality of virtualized servers; and

a cloud network manager (“

CNM”

) coupled to the plurality of TORSs, the plurality of virtualized servers, and the CEM, the CNM configured to;

receive a first notification message from the CEM, wherein the first notification message is to indicate that a first VM has been scheduled for activation on a first of the plurality of virtualized servers;

in response to the first notification message;

determine a first VM media access control (“

MAC”

) address associated with the first VM,

determine a first virtual switch MAC address associated with the first virtual switch in the first virtualized server,

record, in an address mapping table, an association between the first VM MAC address and the first virtual switch MAC address,

determine a first MPLS label to associate a first TORS with the first virtual switch, wherein the first TORS is to be coupled with the first virtualized server, and

send a first flow entry modification message to the first virtual switch to indicate that data packets that match the first MPLS label and the first VM MAC address should be forwarded the first VM.

10. The system of claim 9, wherein the CNM is further configured to send a second flow entry modification message to the first virtual switch in response to the first notification message, the second flow entry modification message to indicate that dynamic host control protocol (“

DHCP”

) messages that originate from the first VM MAC address should be forwarded to the CNM.

11. The system of claim 10, wherein the CNM is further configured to:

receive a DHCP request that originated from the first VM;

relay the DHCP request to a DHCP server;

receive a DHCP reply from the DHCP server;

record, in the address mapping table, an association between the first VM MAC address and a first internet protocol (“

IP”

) address included within the DHCP reply; and

forward the DHCP reply to the first VM.

12. The system of claim 9, wherein the CNM is further configured to send a second flow entry modification message to the first virtual switch in response to the first notification message, the second flow entry modification message to indicate that address resolution protocol (“

ARP”

) messages that originate from the first VM MAC address should be forwarded to the CNM.

13. The system of claim 12, wherein the CNM is further configured to:

receive an ARP request that originated from the first VM;

determine a destination IP address from the ARP request;

determine, from the address mapping table, a destination VM MAC address and a destination virtual switch MAC address each to correspond with the destination IP address;

determine a destination MPLS label to correspond with the destination virtual switch MAC address;

send a third flow entry modification message to the first virtual switch to indicate that data packets that match the destination VM MAC address should have the destination MPLS label pushed on and be forwarded to the first TORS;

generate an ARP reply to indicate the destination VM MAC address is to correspond with the destination IP address; and

forward the ARP reply to the first VM.

14. The system of claim 9, wherein the CNM is further configured to:

receive a second notification message, the second notification message to indicate that the first VM is to be moved to a second of the plurality of virtualized servers and will be unable to service data packets;

send, in response to the second notification message, a second flow entry modification message to the first virtual switch to indicate that data packets that match the first MPLS label and the first VM MAC address should be forwarded to the CNM;

receive a move complete indication to indicate that the first VM moved to the second virtualized server and will be able to service data packets; and

in response to the move complete indication, the CNM further configured to;

determine a second virtual switch MAC address to be associated with a second virtual switch,

record, in the address mapping table, an association between the first VM MAC address and the second virtual switch MAC address,

determine a second MPLS label to associate a second TORS with the second virtual switch, wherein the second TORS is to be coupled with the second virtualized server,

send a third flow entry modification message to the second virtual switch to indicate that data packets that match the second MPLS label and the first VM MAC address should be forwarded the first VM, and

update the address mapping table to associate the first VM with the second virtual switch and disassociate the first VM from the first virtual switch.

15. The system of claim 14, wherein the CNM is further configured to:

cause one or more data packets destined for the first VM MAC address to be buffered in response to the second notification message; and

cause the one or more buffered data packets to be forwarded to the first VM in response to the move complete indication.

16. The system of claim 14, wherein the CNM is further configured to:

receive a peer data packet that matches the first MPLS label and the first VM MAC address, wherein the peer data packet originated from a peer VM hosted by a third of the plurality of virtualized servers;

determine a peer VM MAC address from the peer data packet;

determine a peer virtual switch from the peer VM MAC address, wherein the third virtualized server comprises the peer virtual switch;

forward the peer data packet to the first VM; and

send a fourth flow entry modification message to the peer virtual switch to indicate that data packets destined for the first VM MAC address should have the second MPLS label pushed on and forwarded to a peer TORS, whereby subsequent data packets from the peer VM destined for the first VM will be forwarded to the second virtual switch.

17. A cloud network manager (“

CNM”

) for managing multi-protocol label switching (“

MPLS”

) flow entries in a cloud network, the CNM coupled to a first virtualized server for hosting one or more virtual machines (“

VM”

) wherein the first virtualized server comprises a first virtual switch that supports MPLS and the first virtual switch is coupled to a first top of rack switch (“

TORS”

) that supports MPLS, the CNM configured to;

receive a first notification message, wherein the first notification message is to indicate that a first VM has been scheduled for activation on the first virtualized server; and

in response to the first notification message, the CNM further configured to;

determine a first VM media access control (“

MAC”

) address associated with the first VM,

determine a first virtual switch MAC address associated with the first virtual switch,

record, in an address mapping table, an association between the first VM MAC address and the first virtual switch MAC address,

determine a first MPLS label to associate the first TORS with the first virtual switch, and

send a first flow entry modification message to the first virtual switch to indicate that data packets that match the first MPLS label and the first VM MAC address should be forwarded the first VM.

18. The CNM of claim 17, wherein the CNM is further configured to send a second flow entry modification message to the first virtual switch in response to the first notification message, the second flow entry modification message to indicate that dynamic host control protocol (“

DHCP”

) messages that originate from the first VM MAC address should be forwarded to the CNM.

19. The CNM of claim 18, wherein the CNM is further configured to:

receive a DHCP request that originated from the first VM;

relay the DHCP request to a DHCP server;

receive a DHCP reply from the DHCP server;

record, in the address mapping table, an association between the first VM MAC address and a first internet protocol (“

IP”

) address included within the DHCP reply; and

forward the DHCP reply to the first VM.

20. The CNM of claim 17, wherein the CNM is further configured to send a second flow entry modification message to the first virtual switch in response to the first notification message, the second flow entry modification message to indicate that address resolution protocol (“

ARP”

) messages that originate from the first VM MAC address should be forwarded to the CNM.

21. The CNM of claim 20, wherein the CNM is further configured to:

receive an ARP request that originated from the first VM;

determine a destination IP address from the ARP request;

determine, from the address mapping table, a destination VM MAC address and a destination virtual switch MAC address each to correspond with the destination IP address;

determine a destination MPLS label to correspond with the destination virtual switch MAC address;

send a third flow entry modification message to the first virtual switch to indicate that data packets that match the destination VM MAC address should have the destination MPLS label pushed on and be forwarded to the first TORS;

generate an ARP reply to indicate the destination VM MAC address corresponds to the destination IP address; and

forward the ARP reply to the first VM.

22. The CNM of claim 17, wherein the CNM is further configured to:

receive a second notification message, the second notification message to indicate that the first VM is to move to a second virtualized server and will be unable to service data packets, wherein the second virtualized server is to comprise a second virtual switch that supports MPLS and the second virtual switch is to be coupled to a second TORS that supports MPLS;

send, in response to the second notification message, a second flow entry modification message to the first virtual switch to indicate that data packets that match the first MPLS label and the first VM MAC address should be forwarded to the CNM;

receive a move complete indication to indicate that the first VM has moved to the second virtualized server and will be able to service data packets; and

in response to the move complete indication, the CNM configured to;

determine a second virtual switch MAC address associated with the second virtual switch,

record, in the address mapping table, an association between the first VM MAC address and the second virtual switch MAC address,

determine a second MPLS label to link the second TORS with the second virtual switch,

send a third flow entry modification message to the second virtual switch to indicate that data packets that match the second MPLS label and the first VM MAC address should be forwarded the first VM, and

update the address mapping table to associate the first VM with the second virtual switch and disassociate the first VM from the first virtual switch.

23. The CNM of claim 22, wherein the CNM is further configured to:

cause one or more data packets destined for the first VM MAC address to be buffered in response to the second notification message; and

cause the one or more buffered data packets to be forwarded to the first VM in response to the move complete indication.

24. The CNM of claim 22, wherein the CNM is further configured to:

receive a peer data packet that matches the first MPLS label and the first VM MAC address, wherein the peer data packet originated from a peer VM hosted by a peer virtualized server that comprises a peer virtual switch that supports MPLS and is coupled to a peer TORS that supports MPLS;

determine a peer VM MAC address from the peer data packet;

determine the peer virtual switch from the peer VM MAC address;

forward the peer data packet to the first VM; and

send a fourth flow entry modification message to the peer virtual switch to indicate that data packets destined for the first VM MAC address should have the second MPLS label pushed on and be forwarded to the peer TORS, whereby subsequent data packets from the peer VM destined for the first VM will be forwarded to the second virtual switch.