Processor-memory module performance acceleration in fabric-backplane enterprise servers

• US 8,868,790 B2
• Filed: 02/12/2005
• Issued: 10/21/2014
• Est. Priority Date: 02/13/2004
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Lovett, Thomas Dean, Mehrotra, Sharad, Nicolaou, Cosmos, Saraiya, Nakul Pratap, Shah, Shreyas B., White, Myron H., Jagannathan, Rajesh K., Shingane, Mangesh
• Primary Examiner(s): Burgess, Glenton B
• Assistant Examiner(s): Nguyen, Angela
• Application Number:
  US11/057,036
• Publication Number:
  US 20130107872A1
• Time in Patent Office: 3,538 Days
• Field of Search: 709/250
• US Class Current: 709/250
• CPC Class Codes: H04L 1/0061 Error detection codes
• CPC Class Codes: H04L 12/4645 Details on frame tagging routing of packets H04L45/00; support for virtual LAN H04L49/354
• CPC Class Codes: H04L 49/00 Packet switching elements
• CPC Class Codes: H04L 49/10 characterised by the switching fabric construction
• CPC Class Codes: H04L 49/111 Switch interfaces, e.g. port details
• CPC Class Codes: H04L 49/116 Transferring a part of the packet through each plane, e.g. by bit-slicing
• CPC Class Codes: H04L 49/118 Address processing within a device, e.g. using internal ID or tags for routing within a switch
• CPC Class Codes: H04L 49/357 Fibre channel switches
• CPC Class Codes: H04L 49/45 Arrangements for providing or supporting expansion
• CPC Class Codes: H04L 49/602 Multilayer or multiprotocol switching, e.g. IP switching
• CPC Class Codes: H04L 49/70 Virtual switches

Abstract

A hybrid server and multi-layer switch system architecture, referred to hereinafter as the Enterprise Fabric (EF) architecture, forms the basis for a number of Enterprise Server (ES) chassis embodiments. Each ES embodiment generally includes one or more Processor Memory Modules (PMMs, each generally having one or more symmetric multiprocessor complexes), one or more Network Modules, and a System Control Module (SCM). The SCM includes a cellified switching-fabric core (SF) and a System Intelligence Module (SIM). Each PMM has one or more resident Virtual IO Controller (VIOC) adapters. Each VIOC is a specialized I/O controller that includes embedded layer-2 forwarding and filtering functions and tightly couples the PMM to the SF. Thus the layer-2 switch functionality within the ES chassis is distributed over all of the SCM, NM, and PMM modules. Through the use of VIOC/VNIC device drivers, host operating system software (Host O/S) running on the PMMs is presented with a plurality of Virtual Network Interface Cards (VNICs). In some embodiments, each VNIC behaves as a high-performance Ethernet interface at the full disposal of the Host O/S. In other embodiments, at least some of the VNICs behave as high-performance Fibre Channel Host Bus Adapters.

Claims

1. An apparatus for use with one or more external networks having links and devices, the apparatus comprising:

at least one network module having one or more physical network interfaces enabled to couple with the links of the external networks;

a plurality of processor-memory modules enabled to run software processes and to provide interface state to one or more of the software processes via one or more dedicated hardware registers of the processor-memory module, the interface state corresponding to a plurality of Ethernet compatible logical network interfaces;

a multi-ported internally non-blocking switching fabric, each of the network modules and the processor-memory modules being coupled via point-to-point links to a respective port of the switching fabric at a respective fabric address, the switching fabric being enabled to support multiple simultaneous data exchanges between the network modules and the processor-memory modules;

wherein the physical network interfaces and the Ethernet compatible logical network interfaces are enabled to selectively exchange frames among the links and the processes;

wherein each network module comprises logic to cellify each switch-bound frame received by the physical network interfaces of each network module;

wherein each processor-memory module comprises a plurality of physical partitions and logic to

convert each switch bound frame into cells, each cell specifying a destination address corresponding to a physical partition of a destination processor-memory module and transfer each cell through the switching fabric according to the destination address; and

wherein each of the network modules and the processor-memory modules comprises logic to reassemble the cells into the frame.

2. The apparatus of claim 1, further wherein:

each of the network modules and the processor-memory modules is enabled, for each of the switch-bound frames, to use the destination address of the frame prior to cellification to search a repository of forwarding information and to encapsulate the frame with at least a destination fabric address that is based at least in part on a result of the search.

3. The apparatus of claim 1, further wherein in a lossy response to oversubscription of bandwidth incoming to each processor-memory module from the switching fabric, frames eligible to be discarded are dropped by the processor-memory module.

4. The apparatus of claim 1, further wherein in a lossless response to oversubscription of bandwidth incoming to each processor-memory module from the switching fabric, fabric back pressure is applied by the processor-memory module to limit traffic sent by the other network and processor-memory modules.

5. The apparatus of claim 2, further wherein:

the physical network interfaces and the Ethernet compatible logical network interfaces are communication interfaces, and each communication interface has a respective destination fabric sub-address; and

when the destination address of the frame corresponds to one of the communication interfaces, the search result further comprises the destination fabric sub-address of the corresponding communication interface.

6. The apparatus of claim 5, further wherein for at least some destination fabric sub-addresses there is more than one associated destination logical interface and at least some of the search results further comprise a destination logical interface identifier.