Input/output controller for coupling the processor-memory complex to the fabric in fabric-backplane interprise servers

• US 7,664,110 B1
• Filed: 02/12/2005
• Issued: 02/16/2010
• Est. Priority Date: 02/07/2004
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Habanero Holdings Incorporated
• Inventors: Shingane, Mangesh, Lovett, Thomas Dean, Nicolaou, Cosmos, Shah, Shreyas B., Jagannathan, Rajesh K., Mehrotra, Sharad, White, Myron H., Saraiya, Nakul Pratap
• Primary Examiner(s): Sheikh, Ayaz R
• Assistant Examiner(s): Moutaouakil, Mounir
• Application Number:
  US11/057,112
• Time in Patent Office: 1,830 Days
• Field of Search: 370/392, 370/395.21, 370/395.31, 370/396, 709238-244
• US Class Current: 370/392
• CPC Class Codes: H04L 49/201 Multicast operation; Broadcast operation
• CPC Class Codes: H04L 49/254 Centralised controller, i.e. arbitration or scheduling
• CPC Class Codes: H04L 49/3045 Virtual queuing
• CPC Class Codes: H04L 49/3072 Packet splitting
• CPC Class Codes: H04L 49/351 for local area network [LAN], e.g. Ethernet switches
• CPC Class Codes: H04L 49/354 for supporting virtual local area networks [VLAN]
• CPC Class Codes: H04L 49/356 for storage area networks
• CPC Class Codes: H04L 49/357 Fibre channel switches
• CPC Class Codes: H04L 49/506 Backpressure
• CPC Class Codes: H04L 49/557 Error correction, e.g. fault recovery or fault tolerance
• CPC Class Codes: H04L 49/70 Virtual switches
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/32 Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
• CPC Class Codes: H04L 69/324 in the data link layer [OSI layer 2], e.g. HDLC
• CPC Class Codes: H04L 69/326 in the transport layer [OSI layer 4] H04L69/16 takes precedence
• CPC Class Codes: H04L 69/328 in the presentation layer [OSI layer 6]
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A hybrid server and multi-layer switch system architecture, referred to hereinafter as the Enterprise Fabric (EF) architecture, forms the basis for a number of Enterprise Server (ES) chassis embodiments. Each ES embodiment generally includes one or more Processor Memory Modules (PMMs, each generally having one or more symmetric multiprocessor complexes), one or more Network Modules, and a System Control Module (SCM). The SCM includes a cellified switching-fabric core (SF) and a System Intelligence Module (SIM). Each PMM has one or more resident Virtual IO Controller (VIOC) adapters. Each VIOC is a specialized I/O controller that includes embedded layer-2 forwarding and filtering functions and tightly couples the PMM to the SF. Thus the layer-2 switch functionality within the ES chassis is distributed over all of the SCM, NM, and PMM modules. Through the use of VIOC/VNIC device drivers, host operating system software (Host O/S) running on the PMMs is presented with a plurality of Virtual Network Interface Cards (VNICs). In some embodiments, each VNIC behaves as a high-performance Ethernet interface at the full disposal of the Host O/S. In other embodiments, at least some of the VNICs behave as high-performance Fiber Channel Host Bus Adapters.

Claims

1. An apparatus for use with one or more external networks having links and devices, the apparatus including:

at least one network module having one or more physical network interfaces adapted to couple with the links of the external networks;

at least one processor-memory module adapted to run software processes and provide programmable registers for each of a plurality of logical network interfaces;

a multi-ported switching fabric, each module being adapted to couple to a respective port of the switching fabric at a respective fabric address;

wherein the network interfaces are adapted to selectively exchange frames among the links and the software processes;

wherein for each frame received by the network interfaces of each module, the module is adapted to use a destination address of the frame to search a repository of forwarding information and to present the frame to the switch fabric encapsulated with a search-result-provided destination fabric address; and

wherein each processor-memory module has a processor-memory portion and an input-output portion, each processor-memory portion providing circuitry in support of the running of the software processes, the input-output portion providing the coupling to the switching fabric for the processor-memory module via one or more virtual input-output controllers, each virtual input-output controller providing circuitry in support of the programmable registers of the logical network interfaces.

2. The apparatus of claim 1, wherein:

each virtual input-output controller includes a processor-memory interface, a fabric interface, a transmit sub-system, a receive sub-system, and a lookup sub-system.

3. The apparatus of claim 2, wherein:

the transmit sub-system fetches transmit descriptors from host memory.

4. The apparatus of claim 3, wherein:

each transmit sub-system includes at least one shared transmit DMA engine adapted to copy the frames from transmit-descriptor-specified memory buffers within the processor-memory portion into a transmit memory shared by the logical network interfaces of the virtual input-output controller.

5. The apparatus of claim 4, wherein:

each transmit sub-system includes at least two transmit DMA engines.

6. The apparatus of claim 4, wherein:

the transmit sub-system is adapted to perform the search via the lookup sub-system, the lookup sub-system having local access to at least part of a copy of the repository.

7. The apparatus of claim 6, wherein:

the transmit sub-system has control logic adapted to encapsulate lookup results with the frame.

8. The apparatus of claim 7, wherein:

each transmit sub-system includes shared transmit segmentation logic adapted to segment the encapsulated frames in the shared transmit memory into cells.

9. The apparatus of claim 8, wherein:

each transmit sub-system selectively transfers the cellifed frames from the shared transmit memory into virtual output queues in the fabric interface.

10. The apparatus of claim 9, wherein:

the fabric interface selectively transfers the cells in the virtual output queues to the switching fabric in accord with a virtual output queue scheduling policy.

11. The apparatus of claim 2, wherein:

the receive sub-system fetches receive descriptors from host memory.

12. The apparatus of claim 11, wherein:

the receive sub-system receives cells from the fabric.

13. The apparatus of claim 12, wherein:

the receive sub-system parses incoming encapsulated cellified frames to determine the destination fabric address and a fabric sub-address as well as optional queue and destination logical interface specifiers.

14. The apparatus of claim 13, wherein:

the receive sub-system reassembles cells into frames within receive memory shared by the logical network interfaces.

15. The apparatus of claim 14, wherein:

the receive sub-system includes at least one shared receive DMA engine adapted to copy the frames from the receive shared memory into receive-descriptor-specified buffers in the host memory.

16. The apparatus of claim 15, wherein:

the receive sub-system includes at least two receive DMA engines.

17. The apparatus of claim 15, wherein:

the receive sub-system interrupts at least some of the software processes for every N packets reassembled in host memory per logical network interface.

18. The apparatus of claim 17, wherein:

N is separately programmable for each logical network interface.

19. The apparatus of claim 2, wherein:

each virtual input-output controller is implemented on a single integrated circuit.

20. The apparatus of claim 2, wherein:

each virtual input-output controller is packaged as a single electronic component.