SCSI transport for fabric-backplane enterprise servers

• US 7,633,955 B1
• Filed: 01/19/2006
• Issued: 12/15/2009
• Est. Priority Date: 02/13/2004
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Habanero Holdings Incorporated
• Inventors: Saraiya, Nakul Pratap, Sundaresan, Ganesh, Fisher, William E.
• Primary Examiner(s): Shah, Chirag G
• Assistant Examiner(s): Rivas, Salvador E
• Application Number:
  US11/337,172
• Time in Patent Office: 1,426 Days
• Field of Search: 370/401, 709/223
• US Class Current: 370/401
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: H04L 49/00 Packet switching elements
• CPC Class Codes: H04L 49/254 Centralised controller, i.e. arbitration or scheduling
• CPC Class Codes: H04L 49/3045 Virtual queuing
• CPC Class Codes: H04L 49/3072 Packet splitting
• CPC Class Codes: H04L 49/356 for storage area networks
• CPC Class Codes: H04L 49/357 Fibre channel switches
• CPC Class Codes: H04L 49/503 Policing
• CPC Class Codes: H04L 49/552 by ensuring the integrity of packets received through redundant connections
• CPC Class Codes: H04L 49/70 Virtual switches
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/326 in the transport layer [OSI layer 4] H04L69/16 takes precedence

Abstract

A Small Computer System Interface (SCSI) transport for fabric backplane enterprise servers provides for local and remote communication of storage system information between storage sub-system elements of an ES system and other elements of an ES system via a storage interface. The transport includes encapsulation of information for communication via a reliable transport implemented in part across a cellifying switch fabric. The transport may optionally include communication via Ethernet frames over any of a local network or the Internet. Remote Direct Memory Access (RDMA) and Direct Data Placement (DDP) protocols are used to communicate the information (commands, responses, and data) between SCSI initiator and target end-points. A Fibre Channel Module (FCM) may be operated as a SCSI target providing a storage interface to any of a Processor Memory Module (PMM), a System Control Module (SCM), and an OffLoad Module (OLM) operated as a SCSI initiator.

Claims

1. A system comprising:

a switch fabric enabled to transfer data via cells;

a network module coupled to the switch fabric;

a fibre channel module coupled to the switch fabric and comprising

a plurality of queues to communicate data with the switch fabric,

a plurality of fibre channel ports, and

control logic coupled to the queues; and

wherein the control logic is enabled

to operate each of the queues to manage information transfer for a respective one of a plurality of reliable communication channels, the reliable communication channels being compatible with a Remote Direct Memory Access (RDMA) protocol,

to operate each of the reliable communication channels to provide retransmission of data not received and retransmission of data received out of sequence, the retransmissions being from a sender to a receiver and in response to the receiver conveying to the sender that a sequence number has been received out of sequence,

to determine a selected one of the fibre channel ports in response to an identification transferred via the network module and the switch fabric, and

to manage communication of Small Computer System Interface (SCSI)-compatible data between a plurality of the queues and the selected one of the fibre channel ports.

2. The system of claim 1, wherein the queues comprise ingress queues and the data communicated with the switch fabric comprises data received from the switch fabric.

3. The system of claim 1, further comprising a mass storage device coupled to one of the fibre channel ports.

4. The system of claim 3, wherein the mass storage device is a magnetic disk drive.

5. The system of claim 1, wherein each of the queues comprises a respective send sub-queue and a respective receive sub-queue.

6. The system of claim 5, wherein at least two of the queues collectively comprise a single shared status sub-queue.

18. The system of claim 1, wherein the data received out of sequence is with respect to a particular one of the reliable communication channels, and the control logic is further enabled to discard the data received out of sequence and to discard further data received with respect to the particular reliable communication channel until reception of the retransmission of the data not received.

7. A method comprising:

transferring storage traffic as cells via a switch fabric between a network module and a fibre channel module;

operating each of a plurality of queues implemented in the fibre channel module to manage information transfer with the switch fabric for a respective one of a plurality of communication channels;

providing respective communication channel handshake information for each of the communication channels to enable detection of data received out of sequence and to infer that data was lost, each of the respective communication channel handshake information comprising a respective series of sequence numbers and the detection is at least in part via determining that a received one of the sequence numbers is out of sequence;

communicating results of the detection at least in part via the switch fabric, and in response retransmitting the data that was lost and the data received out of sequence;

mapping transactions included in the storage traffic to fibre channel interfaces provided by the fibre channel module according to transaction identifiers included in the transactions, the mapping comprising mapping at least two of the transactions communicated via respective distinct ones of the communication channels to respective distinct ones of the fibre channel interfaces; and

wherein the transactions comprise Small Computer System Interface (SCSI)-compatible commands and the communication channels are enabled to convey Remote Direct Memory Access (RDMA) traffic.

8. The method of claim 7, wherein each of the queues comprises a respective send sub-queue and a respective receive sub-queue.

9. The method of claim 8, wherein at least two of the queues collectively comprise a single shared status sub-queue.

10. The method of claim 7, further comprising communicating storage data between some of the queues and a plurality of fibre channel compatible storage elements.

11. The method of claim 10, wherein the fibre channel compatible storage elements comprise at least one of a disk drive, a tape drive, a tape library, a storage array, a storage switch, a storage director, a storage router, a storage network, and a Storage Area Network (SAN).

12. The method of claim 7, wherein each of the communication channels has a respective endpoint external to the fibre channel module.

19. The method of claim 7, wherein the data received out of sequence is associated with a particular one of the communication channels, and the method further comprises discarding the data received out of sequence and discarding further data received and associated with the particular communication channel until reception of the retransmission of the data that was lost.

13. A method comprising:

receiving network packets each having a respective network destination address;

forwarding some of the network packets as cells each having a respective cell destination address determined at least in part by the respective network destination address;

switching and receiving the cells according to the cell destination addresses;

according to a plurality of queues, reassembling some of the switched and received cells as storage information that comprises a target address;

according to a plurality of reliable communication channels, providing handshake information to enable detecting out of sequence data and to infer that data was lost, the handshake information comprising sequence numbers, the detecting being at least in part via determining that a received one of the sequence numbers is out of sequence;

communicating results of the detecting at least in part via the switch fabric, and in response retransmitting the data that was lost and the out of sequence data;

selecting one of a plurality of fibre channel storage interfaces based at least in part on the target address;

communicating some of the storage information to the selected fibre channel storage interface as storage frames;

wherein a portion of the storage information is Small Computer System Interface (SCSI)-compatible storage system requests;

wherein each of the reliable communication channels is compatible with at least one of a Remote Direct Memory Access (RDMA) protocol and a Direct Data Placement (DDP) protocol, is associated with one of the queues, and is identified by a respective identifier; and

wherein the target address comprises one of the identifiers.

14. The method of claim 13, wherein the target address further comprises at least one of an Internet protocol compatible address and an Ethernet compatible address.

15. The method of claim 13, wherein at least two of the reliable communication channels communicate data with a single one of the fibre channel storage interfaces.

16. The method of claim 13, wherein the receiving network packets is via a plurality of network modules.

17. The method of claim 13, wherein the communicating some of the storage information is via a plurality of fibre channel modules.

20. The method of claim 13, wherein the out of sequence data corresponds to a particular one of the reliable communication channels, and the method further comprises discarding the out of sequence data and discarding further data received that corresponds to the particular reliable communication channel until reception of the retransmission of the data that was lost.