Fabric-backplane enterprise servers with VNICs and VLANs

• US 7,860,097 B1
• Filed: 02/12/2005
• Issued: 12/28/2010
• Est. Priority Date: 02/13/2004
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Habanero Holdings Incorporated
• Inventors: Nicolaou, Cosmos, Shah, Shreyas B., Jagannathan, Rajesh K., Lovett, Thomas Dean, Saraiya, Nakul Pratap, Shingane, Mangesh, Mehrotra, Sharad, White, Myron H.
• Primary Examiner(s): Sheikh, Ayaz R
• Assistant Examiner(s): Moutaouakil, Mounir
• Application Number:
  US11/057,116
• Time in Patent Office: 2,145 Days
• Field of Search: 370/235, 370389-392, 709/232
• US Class Current: 370/392
• CPC Class Codes: H04L 12/4641 Virtual LANs, VLANs, e.g. virtual private networks [VPN] LAN interconnection over a bridge based backbone H04L12/462; encapsulation techniques H04L12/4633; routing of packets H04L45/00; packet switches H04L49/00; virtual private networks for security H04L63/0272
• CPC Class Codes: H04L 49/357 Fibre channel switches
• CPC Class Codes: H04L 49/45 Arrangements for providing or supporting expansion
• CPC Class Codes: H04L 49/602 Multilayer or multiprotocol switching, e.g. IP switching
• CPC Class Codes: H04L 49/70 Virtual switches

Abstract

Virtual Network Interface Controllers (vNICs) provide for communication among modules of Enterprise Server (ES) embodiments via a switch fabric dataplane. Processes executing on compute complexes of the servers exchange data as packets or messages by interfaces made available through vNICs. The vNICs further provide for transparent communication with network and storage interfaces. vNIC provisioning capabilities include programmable bandwidth, priority scheme selection, and detailed priority control (such as round-robin weights). In some embodiments, vNICs are implemented in Virtual Input/Output Controllers (VIOCs). In another aspect, Virtual Local Area Networks (VLANs) enable access to layer-2 and selected layer-3 network functions while exchanging the packets and messages. VLAN identification is provided in each vNIC, and VLAN processing is partially performed in VIOCs implementing vNICs. The compute complexes and interfaces are typically configured as pluggable modules inserted into a backplane included in a chassis.

Claims

1. A method of transmitting data over a switching fabric from a symmetric multiprocessor having a memory, the method comprising:

managing a plurality of transmit queues organized as a plurality of logical network interfaces, each transmit queue referencing a corresponding respective plurality of transmit buffers in the memory, the managing performed at least in part by at least one network interface driver process executing on the symmetric multiprocessor;

assigning a media access control address that is shared by each transmit queue within each logical network interface;

storing the data into the transmit buffers by at least one application process executing on the symmetric multiprocessor;

within egress logic, scheduling processing of the transmit queues;

via a bus interface of the egress logic, reading the data into the egress logic in accordance with the scheduling;

within the egress logic, providing cells to an included fabric interface in accordance with selective groupings of the data, each cell including a fabric egress port address corresponding to a destination network address of each data grouping and the fabric egress port address further corresponding to a physical fabric egress port of the fabric; and

via the fabric interface, sending the cells to the fabric.

2. The method of claim 1, wherein the destination network address is a media access control destination address, and the corresponding fabric egress port address is obtained via a lookup based at least in part upon the media access control destination address.

3. The method of claim 1, wherein the scheduling includes evaluation at least one of:

bandwidth consumption of the transmit queues, bandwidth allocation of the transmit queues, priorities assigned to the transmit queues, resource utilization by the transmit queues, and past processing order of the transmit queues.

4. The method of claim 1, wherein at least some of the data groupings correspond to packets.

5. The method of claim 1, wherein at least some of the data groupings correspond to byte-granular variable length groupings sent in accordance with a reliable delivery protocol.

6. The method of claim 1, wherein at least some of the data groupings correspond to messages.

7. The method of claim 6, wherein the messages have a byte-granular variable length and are sent in accordance with a reliable delivery protocol.

8. The method of claim 1, wherein at least some of the data groupings correspond to packets and at least some other of the data groupings correspond to messages.

9. The method of claim 8, wherein at least some of the transmit queues are packet transmit queues and at least some other of the transmit queues are message transmit queues.

10. The method of claim 9, wherein each logical network interface includes some of the packet transmit queues and some of the message transmit queues.

11. The method of claim 10, wherein each data grouping sent via the fabric interface is read according to an associated transmit queue of the transmit queues, the associated transmit queue being a member of an associated logical network interface of the logical network interfaces, each data grouping sent via the fabric comprising a source network address field containing the media access control address assigned to the associated logical network interface.

12. The method of claim 9, wherein each of a plurality of network interfaces has an associated respective plurality of the packet transmit queues and an associated respective plurality of the message transmit queues.

13. The method of claim 12, wherein the scheduling uses a multiple-level selecting and the selecting at a first level selects one of the packet transmit queues associated with each network interface.

14. An apparatus for transmitting data over a switching fabric from a symmetric multiprocessor having a memory, the apparatus comprising:

means for organizing a plurality of transmit queues as a plurality of logical network interfaces, each transmit queue referencing a corresponding respective plurality of transmit buffers in the memory;

wherein the transmit queues are managed by at least one network interface driver process executing on the symmetric multiprocessor, each transmit queue within each logical network interface shares a media access control address assignment, and the data is stored into the transmit buffers by at least one application process executing on the symmetric multiprocessor;

egress logic comprising a bus interface, a fabric interface, means for scheduling processing of the transmit queues, means for reading the data into the egress logic via the bus interface in accordance with the processing, means for providing cells to the fabric interface in accordance with selective groupings of the data, and means for sending the cells to the fabric via the fabric interface; and

wherein each cell comprises a fabric egress port address corresponding to a destination network address of each data grouping and the fabric egress port address further corresponds to a physical fabric egress port of the fabric.

15. The apparatus of claim 14, wherein the destination network address is a media access control destination address, and the corresponding fabric egress port address is obtained via a lookup based at least in part upon the media access control destination address.

16. The apparatus of claim 14, wherein the means for scheduling comprises evaluation at least one of:

bandwidth consumption of the transmit queues, bandwidth allocation of the transmit queues, priorities assigned to the transmit queues, resource utilization by the transmit queues, and past processing order of the transmit queues.

17. The apparatus of claim 14, wherein at least some of the data groupings correspond to packets.

18. The apparatus of claim 14, wherein at least some of the data groupings correspond to byte-granular variable length groupings sent in accordance with a reliable delivery protocol.

19. The apparatus of claim 14, wherein at least some of the data groupings correspond to messages.

20. The apparatus of claim 19, wherein the messages have a byte-granular variable length and are sent in accordance with a reliable delivery protocol.

21. The apparatus of claim 14, wherein at least some of the data groupings correspond to packets and at least some other of the data groupings correspond to messages.

22. The apparatus of claim 21, wherein at least some of the transmit queues are packet transmit queues and at least some other of the transmit queues are message transmit queues.

23. The apparatus of claim 22, wherein each logical network interface comprises some of the packet transmit queues and some of the message transmit queues.

24. The apparatus of claim 23, wherein each data grouping sent via the fabric interface is read according to an associated transmit queue of the transmit queues, the associated transmit queue being a member of an associated logical network interface of the logical network interfaces, each data grouping sent via the fabric comprising a source network address field containing the media access control address assigned to the associated logical network interface.

25. The apparatus of claim 22, wherein each of a plurality of network interfaces has an associated respective plurality of the packet transmit queues and an associated respective plurality of the message transmit queues.

26. The apparatus of claim 25, wherein the means for scheduling uses a multiple-level selection and the selection at a first level selects one of the packet transmit queues associated with each network interface.