Fabric address and sub-address resolution in fabric-backplane enterprise servers

• US 7,561,571 B1
• Filed: 02/12/2005
• Issued: 07/14/2009
• Est. Priority Date: 02/13/2004
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Habanero Holdings Incorporated
• Inventors: Shingane, Mangesh, Lovett, Thomas Dean, Nicolaou, Cosmos, Shah, Shreyas B., Jagannathan, Rajesh K., Mehrotra, Sharad, White, Myron H., Saraiya, Nakul Pratap
• Primary Examiner(s): Kizou, Hassan
• Assistant Examiner(s): Moutaouakil, Mounir
• Application Number:
  US11/057,119
• Time in Patent Office: 1,613 Days
• Field of Search: 370/392, 370/395.21, 370/395.31, 370/396, 709238-244
• US Class Current: 370/392
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways

Abstract

A hybrid server and multi-layer switch system architecture, referred to hereinafter as the Enterprise Fabric (EF) architecture, forms the basis for a number of Enterprise Server (ES) chassis embodiments. Each ES embodiment generally includes one or more Processor Memory Modules (PMMs, each generally having one or more symmetric multiprocessor complexes), one or more Network Modules, and a System Control Module (SCM). The SCM includes a cellified switching-fabric core (SF) and a System Intelligence Module (SIM). Each PMM has one or more resident Virtual IO Controller (VIOC) adapters. Each VIOC is a specialized I/O controller that includes embedded layer-2 forwarding and filtering functions and tightly couples the PMM to the SF. Thus the layer-2 switch functionality within the ES chassis is distributed over all of the SCM, NM, and PMM modules. Through the use of VIOC/VNIC device drivers, host operating system software (Host O/S) running on the PMMs is presented with a plurality of Virtual Network Interface Cards (VNICs). In some embodiments, each VNIC behaves as a high-performance Ethernet interface at the full disposal of the Host O/S. In other embodiments, at least some of the VNICs behave as high-performance Fiber Channel Host Bus Adapters.

Claims

1. An apparatus for use with one or more external networks, the apparatus comprising:

at least one network module having one or more physical network interfaces enabled to couple with respective links of the external networks;

at least one processor-memory module enabled to run software processes and providing software process visible state for each of a plurality of logical network interfaces;

a multi-ported switching fabric, each network and processor-memory module being enabled to couple to a respective port of the switching fabric at a respective fabric address;

wherein the plurality of logical network interfaces share one or more couplings to the switching fabric;

wherein the physical and logical network interfaces are enabled to be associated with respective fabric sub-addresses and Data Link Layer MAC addresses and to selectively exchange Data Link Layer frames among the links and the processes; and

wherein for at least some of the frames received by the network interfaces of each network and processor-memory module, the module is enabled to use at least a destination MAC address of the frame to search a repository of forwarding information and to present the frame to the switch fabric encapsulated with a search-result-provided destination fabric address and destination fabric sub-address.

2. The apparatus of claim 1, wherein:

the repository of forwarding information includes a layer-2 (L2) forwarding information base (FIB) in each network and processor-memory module, each physical and logical network interface is associated with at least one virtual local area network (VLAN), each VLAN has a respective VLAN identifier (VLANid), and communications among the physical and logical network interfaces is controlled in part by assignment of VLANids to each physical and logical network interface.

3. The apparatus of claim 2, wherein:

L2 communication among a group of two or more of the physical and logical network interfaces is enabled by assignment of identical VLANids to the physical and logical network interfaces of the group.

4. The apparatus of claim 3, wherein:

unless specified to the contrary, each physical and logical network interface is assigned an identical VLANid by default.

5. The apparatus of claim 3, wherein:

those of the physical and logical network interfaces having identical VLANid assignments are on a common VLAN and appear as ports of a virtualized L2 switch respective to the VLAN and operating in accordance with the forwarding information in the repository.

6. The apparatus of claim 1, wherein:

at least part of the logical network interface software process visible state is the programmer visible state of an Ethernet compatible network interface card.

7. The apparatus of claim 1, wherein:

at least part of the logical network interface software process visible state is the programmer visible state of a fibre channel compatible host bus adapter.

19. The apparatus of claim 2, wherein:

L2 communications is precluded for physical and logical network interfaces having different VLANid assignments.

20. The apparatus of claim 2, wherein:

the specification of VLANid assignments requires network administrator privileges.

21. The apparatus of claim 2, wherein:

each of the entries in the L2 FIBs corresponding to a known MACDA are always associated will a respective VLANid of the respective destination.

22. The apparatus of claim 2, wherein:

at least some keys used to search at least some of the L2 FIBs include the MACDA of a source packet.

8. An apparatus for use with one or more external networks, the apparatus comprising:

at least one network module having one or more physical network interfaces enabled to couple with respective links of the external networks;

at least one processor-memory module enabled to run software processes and providing software process visible state for each of a plurality of logical network interfaces;

a multi-ported switching fabric, each network and processor-memory module being enabled to couple to a respective port of the switching fabric at a respective fabric address;

wherein the physical and logical network interfaces are enabled to be associated with respective fabric sub-addresses and Data Link Layer MAC addresses and to selectively exchange Data Link Layer frames among the links and the processes;

wherein for at least some of the frames received by the network interfaces of each module, the module is enabled to present the frame to the switch fabric encapsulated with a destination fabric address and destination fabric sub-address;

wherein at least some of the destination fabric address and sub-address are determined in each logical network interface by using a destination MAC address of the frame to search a processor-memory module repository of forwarding information;

wherein at least some of the destination fabric addresses and sub-addresses are determined in at least some of the physical network interfaces in accordance with one of at least a first and second mode, the first mode using a destination MAC address of the frame to obtain the destination fabric address and sub-address by searching a network module repository of forwarding information, the destination fabric address and sub-address being set to a programmable static destination fabric address and sub-address in the second mode, each of the at least some of the physical network interfaces being capable of using a different programmable static destination fabric address and sub-address when in the second mode.

9. The apparatus of claim 8, wherein:

the static destination fabric address is a VIOCid and the static destination fabric sub-address is a VNICid and the VNIC appears to receive frames directly from one of the links.

10. The apparatus of claim 9, wherein:

in a sub-mode of the second mode, all of the frames received on the physical network interface are forwarded to a selected VIOC and VNIC.

11. The apparatus of claim 9, wherein:

in a sub-mode of the second mode, some of the frames received on the physical network interface are forwarded to a selected VIOC and VNIC corresponding respectively to the VIOCid and VNICid and other of the frames are discarded in accordance with a predetermined criteria.

12. The apparatus of claim 11, wherein:

the predetermined criteria includes evaluation of the destination MAC address of the frame.

13. The apparatus of claim 12, wherein:

the evaluation includes an L2 FIB lookup.

14. The apparatus of claim 12, wherein:

the evaluation includes comparison with a programmable static MAC address.

15. The apparatus of claim 8, wherein:

wherein at least some of the logical network interfaces are configured to have a special VLANid that when used to search the processor-memory module repository retrieves a programmable static network module destination fabric address and programmable static physical network interface sub-address independent of the destination MAC address of the frame.

16. The apparatus of claim 15, wherein:

the physical network interface at the programmable static physical network interface sub-address conditionally accepts the frames in accordance with a predetermined criteria.

17. The apparatus of claim 15, wherein:

the predetermined criteria includes comparison of the source fabric address and sub-address with a programmable static source fabric address and sub-address.

18. The apparatus of claim 15, wherein:

the physical network interface appears to receive frames directly from just one of the logical network interfaces.