Hierarchical transport protocol stack for data transfer between enterprise servers

• US 8,848,727 B2
• Filed: 12/11/2009
• Issued: 09/30/2014
• Est. Priority Date: 02/13/2004
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Saraiya, Nakul Pratap, Sundaresan, Ganesh, Fisher, William E.
• Primary Examiner(s): Chriss, Andrew
• Assistant Examiner(s): Rivas, Salvador E
• Application Number:
  US12/636,437
• Publication Number:
  US 20130117621A1
• Time in Patent Office: 1,754 Days
• Field of Search: None
• US Class Current: 370/428
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: H04L 49/00 Packet switching elements
• CPC Class Codes: H04L 49/254 Centralised controller, i.e. arbitration or scheduling
• CPC Class Codes: H04L 49/3045 Virtual queuing
• CPC Class Codes: H04L 49/3072 Packet splitting
• CPC Class Codes: H04L 49/356 for storage area networks
• CPC Class Codes: H04L 49/357 Fibre channel switches
• CPC Class Codes: H04L 49/503 Policing
• CPC Class Codes: H04L 49/552 by ensuring the integrity of packets received through redundant connections
• CPC Class Codes: H04L 49/70 Virtual switches
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/326 in the transport layer [OSI layer 4] H04L69/16 takes precedence

Abstract

A Small Computer System Interface (SCSI) transport for fabric backplane enterprise servers provides for local and remote communication of storage system information between storage sub-system elements of an ES system and other elements of an ES system via a storage interface. The transport includes encapsulation of information for communication via a reliable transport implemented in part across a cellifying switch fabric. The transport may optionally include communication via Ethernet frames over any of a local network or the Internet. Remote Direct Memory Access (RDMA) and Direct Data Placement (DDP) protocols are used to communicate the information (commands, responses, and data) between SCSI initiator and target end-points. A Fiber Channel Module (FCM) may be operated as a SCSI target providing a storage interface to any of a Processor Memory Module (PMM), a System Control Module (SCM), and an OffLoad Module (OLM) operated as a SCSI initiator.

Claims

1. A method comprising:

managing transport of storage data between an initiator and a target according to a protocol stack, the protocol stack comprising a plurality of hierarchical layers, the plurality of hierarchical layers comprising Small Computer System Interface (SCSI) protocol as an upper layer protocol of the hierarchical layers, Remote Direct Memory Access (RDMA) protocol or Direct Data Placement (DDP) protocol as a remote data protocol layer, the remote data protocol layer being below the upper layer protocol in the hierarchical layers, Enterprise Fabric Message Protocol (EFMP) as a reliable connection protocol layer, the reliable connection protocol layer being below the remote data protocol layer in the hierarchical layers, and Ethernet as a transmission protocol layer being below the reliable connection protocol layer;

via logic circuitry, transferring commands of the storage transport protocol and portions of the storage data as cells from the initiator to the target via a switch fabric in based on the protocol stack and in accordance with flow control information associated with the reliable connection protocol;

via application and storage processors, accessing the storage data in respective memories associated with each of the processors; and

further via the logic circuitry, accessing the respective memories in accordance with respective direct memory access protocols that are in accordance with the transferring.

2. The method of claim 1, further comprising:

via an application processor portion of the logic circuitry, reading a portion of the storage data from the memory associated with the application processor and supplying a portion of results of the reading to the switch fabric as cells addressed to a storage processor portion of the logic circuitry; and

via the storage processor portion of the logic circuitry, writing at least some of the portion of results from the switch fabric into the memory associated with the storage processor.

3. The method of claim 1, further comprising:

via a storage processor portion of the logic circuitry, reading a portion of the storage data from the memory associated with the storage processor and supplying a portion of results of the reading to the switch fabric as cells addressed to an application processor portion of the logic circuitry; and

via the application processor portion of the logic circuitry, writing at least some of the portion of results from the switch fabric into the memory associated with the application processor.

4. The method of claim 1, further comprising further via the logic circuitry, determining additional flow control information as a function of congestion in the logic circuitry and wherein the transferring is further in accordance with the additional flow control information.

5. The method of claim 1, further comprising via the logic circuitry, detecting one of the portions being received out of sequence and retransmitting the one of the portions.

6. The method of claim 1, further comprising communicating the storage data between the memory associated with the storage processor and a storage device.

7. A system comprising:

a plurality of processors; and

a memory coupled with each of the processors and storing a set of instructions which, when executed by the processors, causes the processors to;

manage transport of storage data between an initiator and a target according to a protocol stack, the protocol stack comprising a plurality of hierarchical layers, the plurality of hierarchical layers comprising Small Computer System Interface (SCSI) protocol as an upper layer protocol of the hierarchical layers, Remote Direct Memory Access (RDMA) protocol or Direct Data Placement (DDP) protocol as a remote data protocol layer, the remote data protocol layer being below the upper layer protocol in the hierarchical layers, Enterprise Fabric Message Protocol (EFMP) as a reliable connection protocol layer, the reliable connection protocol layer being below the remote data protocol layer in the hierarchical layers, and Ethernet as a transmission protocol layer being below the reliable connection protocol layer;

transfer commands of the storage transport protocol and portions of the storage data as cells from the initiator to the target via a switch fabric in based on the protocol stack and in accordance with flow control information associated with the reliable connection protocol;

access the storage data in respective memories associated with each of the processors; and

access the respective memories in accordance with respective direct memory access protocols that are in accordance with the transferring.

8. The system of claim 7, further comprising:

reading a portion of the storage data;

supplying a portion of results of the reading to the switch fabric as cells addressed to a storage; and

writing at least some of the portion of results from the switch fabric into the storage.

9. The system of claim 7, further comprising:

reading a portion of the storage data from the memory;

supplying a portion of results of the reading to the switch fabric as cells addressed to an application; and

via the application, writing at least some of the portion of results from the switch fabric into the memory.

10. A computer-readable memory device comprising a set of instructions stored therein which, when executed by a plurality of processors, causes the processors to:

manage transport of storage data between an initiator and a target according to a protocol stack, the protocol stack comprising a plurality of hierarchical layers, the plurality of hierarchical layers comprising Small Computer System Interface (SCSI) protocol as an upper layer protocol of the hierarchical layers, Remote Direct Memory Access (RDMA) protocol or Direct Data Placement (DDP) protocol as a remote data protocol layer, the remote data protocol layer being below the upper layer protocol in the hierarchical layers, Enterprise Fabric Message Protocol (EFMP) as a reliable connection protocol layer, the reliable connection protocol layer being below the remote data protocol layer in the hierarchical layers, and Ethernet as a transmission protocol layer being below the reliable connection protocol layer;

transfer commands of the storage transport protocol and portions of the storage data as cells from the initiator to the target via a switch fabric in based on the protocol stack and in accordance with flow control information associated with the reliable connection protocol;

access the storage data in respective memories associated with each of the processors; and

access the respective memories in accordance with respective direct memory access protocols that are in accordance with the transferring.

11. The computer-readable memory device of claim 10, further comprising:

reading a portion of the storage data;

supplying a portion of results of the reading to the switch fabric as cells addressed to a storage; and

writing at least some of the portion of results from the switch fabric into the storage.

12. The computer-readable memory device of claim 10, further comprising:

reading a portion of the storage data from the memory;

supplying a portion of results of the reading to the switch fabric as cells addressed to an application; and

via the application, writing at least some of the portion of results from the switch fabric into memory.