IP storage processor and engine therefor using RDMA
First Claim
1. A hardware processor providing Transmission Control Protocol (TCP) or Stream Control Transmission Protocol (SCTP) or User Datagram Protocol (UDP) operations, or a combination of any of the foregoing, over an Internet Protocol (IP), said operations including remote direct memory access (RDMA) capability for data transfer over a network from or to an initiator and to or from a target, said operations requested by a host processor having a Small Computer Systems Interface (SCSI) command layer and an iSCSI driver or an IP Storage driver, said hardware processor comprising:
- a. an RDMA mechanism;
b. a command scheduler for scheduling commands from the command layer of said host processor for operation in said hardware processor;
c. first command queues for queuing commands from said host processor for existing sessions;
d. second command queues for queuing commands from said host processor for sessions that do not currently exist;
e. a database for recording a state of the session on which said command is transported, said database also for recording progress of RDMA for those of said commands that use RDMA;
f. a communication path between said processor and said SCSI layer for communicating status of command execution to said SCSI layer for processing; and
g. at least one transmit/receive engine and at least one command engine coupled to work together to interpret commands and perform appropriate operations for performing RDMA for storing/retrieving data to/from or transmitting/receiving data to/from said target or said initiator.
5 Assignments
0 Petitions
Accused Products
Abstract
An IP Storage processor and processing engines for use in the IP storage processor is disclosed. The IP Storage processor uses an architecture that may provide capabilities to transport and process Internet Protocol (IP) packets from Layer 2 through transport protocol layer and may also perform packet inspection through Layer 7. The engines may perform pass-through packet classification, policy processing and/or security processing enabling packet streaming through the architecture at nearly the full line rate. A scheduler schedules packets to packet processors for processing. An internal memory or local session database cache may store a transport protocol session information database and/or store a storage information session database, for a certain number of active sessions. The session information that is not in the internal memory is stored and retrieved to/from an additional memory. An application running on an initiator or target can in certain instantiations register a region of memory, which is made available to its peer(s) for access directly without substantial host intervention through RDMA data transfer.
-
Citations
55 Claims
-
1. A hardware processor providing Transmission Control Protocol (TCP) or Stream Control Transmission Protocol (SCTP) or User Datagram Protocol (UDP) operations, or a combination of any of the foregoing, over an Internet Protocol (IP), said operations including remote direct memory access (RDMA) capability for data transfer over a network from or to an initiator and to or from a target, said operations requested by a host processor having a Small Computer Systems Interface (SCSI) command layer and an iSCSI driver or an IP Storage driver, said hardware processor comprising:
-
a. an RDMA mechanism; b. a command scheduler for scheduling commands from the command layer of said host processor for operation in said hardware processor; c. first command queues for queuing commands from said host processor for existing sessions; d. second command queues for queuing commands from said host processor for sessions that do not currently exist; e. a database for recording a state of the session on which said command is transported, said database also for recording progress of RDMA for those of said commands that use RDMA; f. a communication path between said processor and said SCSI layer for communicating status of command execution to said SCSI layer for processing; and g. at least one transmit/receive engine and at least one command engine coupled to work together to interpret commands and perform appropriate operations for performing RDMA for storing/retrieving data to/from or transmitting/receiving data to/from said target or said initiator. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17)
-
-
18. A system for a programmable Internet Protocol (IP) Storage processor engine, said processor having Remote Direct Memory Access (RDMA) capability, and used for processing IP Storage Protocol Data Units (PDUs) transported over an IP network using Internet Protocol packets having fields including a plurality of PDUs with fields, said system comprising:
-
a cyclic redundancy check (CRC) component for IP Storage PDU CRC verification or for new CRC generation or a combination of any of the foregoing; a data memory for storing said packets and/or said protocol data units; a packet look-up interface for providing access to said packets or PDUs or packet fields or PDU fields or a combination of any of the foregoing; an execution resource; an instruction decoder and an IP Storage PDU classifier to direct IP Storage processor engine operation; a sequence manager providing specific sequence operations for use in IP Storage data sequencing calculations; and further comprising; an IP Storage session manager including a hash engine to perform hash operations against predetermined fields of the packet to perform a hash table walk to determine the correct session entry for said packet; a register file for extracting predetermined header fields from said packets and PDUs for IP Storage processing; pointer registers for indicating data source and destination; and context register sets for holding multiple contexts for packet execution; said multiple contexts allowing, in response to a given packet execution stalling, another context to be invoked to enable said IP Storage processor engine to continue the execution of another packet stream; said IP Storage processor engine having a cache for holding recently or frequently used session entries, including connection IDs, for local use and further having an interface for informing a packet scheduler of the connection identifiers that are cached for each IP Storage processor engine resource. - View Dependent Claims (19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39)
-
-
40. A hardware implemented Internet Protocol (IP) storage controller providing a remote direct memory access (RDMA) useable in one of a Transmission Control Protocol (TCP), a Stream Control Transmission Protocol (SCTP), and a User Datagram Protocol (UDP) over IP, said IP Storage controller for transporting a plurality of Internet Small Computer Systems Interface (iSCSI) commands and a plurality of Protocol Data Units (PDUs), said IP Storage controller having access to a database for keeping track of data processing operations, said database located in one of a memory on said IP Storage controller and a memory partly on said IP Storage controller and partly in a computing apparatus other than said IP Storage controller, said IP Storage controller coupled to a host processor having an SCSI command layer and an IP storage driver, said IP Storage controller having a transmit and a receive path for data flow, said IP Storage controller comprising:
-
a. a command scheduler configured to schedule processing of the iSCSI commands, said scheduler coupled to said SCSI command layer and to said IP storage driver; b. a receive path configured to facilitate a data flow of received data and a transmit path configured to facilitate a data flow of transmitted data, wherein the received data and the transmitted data are related to the iSCSI commands; c. at least one transmit engine configured to transmit the PDUs related to the transmitted data; d. at least one transmit command engine configured to interpret said PDUs and configured to perform a plurality of operations comprising retrieving information from said host processor to service the iSCSI commands and configured to keep command flow data related to the iSCSI commands in said database updated as said retrieving progresses; e. at least one receive command engine for receiving said iSCSI commands; and f. at least one receive command engine for interpreting said commands and configured to perform a plurality of operations comprising; storing the information to the host processor to service one of the iSCSI commands; and keeping the command flow data in said database updated as the one of the storing and the retrieving progresses. - View Dependent Claims (41, 42, 43, 44)
-
-
45. An Internet Small Computer Systems Interface (iSCSI) processor implemented on a chip having remote direct memory access (RDMA) for enabling low latency iSCSI protocol transfers of data over an Internet Protocol (IP) network, said iSCSI processor including an iSCSI stack providing iSCSI protocol termination and origination, said iSCSI processor transporting the data in a plurality of active sessions over the IP network by transporting a plurality of Protocol Data Units (PDUs) specified by the iSCSI standard, said iSCSI processor comprising:
-
a. an RDMA mechanism configured to perform an RDMA data transfer of the data; b. at least one iSCSI processor engine configured to process a plurality of iSCSI packets including the data; c. an iSCSI session database memory configured to store the state of the session on which said data is transported, said session database memory further configured to record a progress of RDMA for those of said data that use RDMA; d. at least one memory controller configured to control a plurality of memory accesses related to the data; e. at least one media interface coupled to the IP network; and f. a host interface coupled to at least one host processor that provides a command related to the data; said iSCSI processor configured to concurrently perform; i. receiving a first one of the iSCSI packets; ii. providing security processing to a second one of the iSCSI packets; iii. classifying a third one of the iSCSI packets; iv. scheduling a fourth one of the iSCSI packets for processing; v. providing direct memory access for transferring a fifth one of the iSCSI packets between said iSCSI processor and a memory of a system external to said iSCSI processor; vi. executing a plurality of protocol processing operations on the data or a plurality of commands forming a sixth one of the iSCSI packets; and vii. transmitting a seventh one of the iSCSI packets onto a network.
-
-
46. A programmable Internet Protocol (IP) Storage processor for providing remote direct memory access (RDMA) and for processing IP Storage Protocol Data Units (PDUs) transported over an IP network using Internet Protocol packets having fields including a plurality of PDUs, said IP Storage processor comprising:
-
a cyclic redundancy check (CRC) component configured to perform a CRC verification on the PDUs; a data memory configured to store at least one of said packets and said PDUs; a packet look-up interface configured to provide access to at least one of said packets, the PDUs, a packet field of one of the packets, and a PDU field of one of the PDUs; a command decoder configured to decode a command related to data within the packets and an IP Storage PDU classifier configured to direct IP Storage processor operation on the data; an IP Storage session manager including a hash engine configured to perform hash operations against predetermined fields of one of the packets to perform a hash table walk to determine a first session entry of a session for said one of the packets; a register file configured to extract a plurality of predetermined header fields from said packets and the PDUs for IP Storage processing; a plurality of pointer registers configured to indicate a source of the data and a destination of the data; and a plurality of context register sets configured to hold multiple contexts in which one of the packets is executed, said multiple contexts allowing, in response to stalling of execution of the one of the packets within one of the multiple contexts, another one of the multiple contexts to be invoked to enable said IP Storage processor to continue an execution of another one of the packets; a cache configured to hold a plurality of recently or frequently used session entries including a plurality of connection identifiers related to the packets; and an interface configured to inform a packet scheduler of the connection identifiers.
-
-
47. A system comprising:
-
a hardware processor configured to transfer data by executing at least one of a Transmission Control protocol (TCP), a Stream Control Transmission Protocol (SCTP), and a User Datagram Protocol (UDP) over an Internet Protocol (IP), said hardware processor configured to execute a remote direct memory access (RDMA) to perform the transfer of the data over a network between an initiator and target; and a host processor having a Small Computer Systems Interface (SCSI) layer and an IP Storage driver, said host processor configured to request the transfer of the data, said SCSI layer further comprising an SCSI command layer, said hardware processor comprising; a. a command scheduler configured to schedule a plurality of commands received from the SCSI command layer of said host processor to operate in said hardware processor; b. a first set of command queues configured to queue a first subset of the commands received from said host processor, the first subset of the commands relating to a plurality of existing sessions; c. a second set of command queues configured to queue a second subset of the commands received from said host processor, the second subset of the commands relating to a plurality of nonexisting sessions that do not exist; d. a database configured to record a state of one of the existing sessions on which one of the commands is transported, said database also further configured to record a progress of RDMA for one of the commands that use RDMA; e. a communication path between said hardware processor and said SCSI layer, said communication path configured to facilitate a communication of status of execution of the commands to said SCSI layer; and f. at least one transmit/receive engine and at least one command engine coupled to each other, said at least one transmit/receive engine and at least one command engine configured to work together to interpret the commands and perform operations to execute RDMA to access the data from at least one of said target and said initiator.
-
-
48. A programmable Internet Small Computer System Interface (iSCSI) processor implemented on a chip having RDMA capability, said iSCSI processor chip for processing Internet Protocol (IP) packets during TCP/IP sessions in accordance with TCP/IP session information, said IP packets comprising at least one of:
- a) IP Storage Protocol Data Units, b) TCP/IP headers and fields and c) RDMA headers and fields;
said iSCSI processor chip having a memory for storing or retrieving said Internet Protocol Packets or said TCP/IP session information, and said iSCSI processor chip capable of being coupled to a host system, said host system comprising one or more host processors and host buffers, said iSCSI processor chip comprising;an iSCSI processing engine for offloading iSCSI processing from said one or more host processors to perform iSCSI data transfers; a memory controller for controlling access to said memory for storing or retrieving said TCP/IP session information or said Internet Protocol packets; an execution resource for generating or processing said TCP/IP header and field information and for performing TCP/IP operations on said Internet Protocol packets; a network interface to send or receive said Internet Protocol packets from or to said iSCSI processor chip; a host interface controller to couple said iSCSI processor chip to said one or more host processors; an RDMA execution resource for generating or processing said RDMA header and field information and for performing RDMA operations on said Internet Protocol packets that require RDMA operations; and a transmit engine or a receive engine, or a combination thereof, coupled to one of said one or more host processors for retrieving data from said host buffers or performing direct data placement or zero copy in said host buffers without substantial intervention by said host processor. - View Dependent Claims (49, 50)
- a) IP Storage Protocol Data Units, b) TCP/IP headers and fields and c) RDMA headers and fields;
-
51. An Internet Small Computer Systems Interface (iSCSI) processor implemented on a chip having a remote direct memory access (RDMA) capability, said iSCSI processor configured to process Internet Protocol (IP) packets comprising at least one of:
- a) IP Storage Protocol Data Units, b) TCP/IP headers and fields and c) RDMA headers and fields, said iSCSI processor comprising;
a. an RDMA mechanism configured to perform RDMA on a set of data; b. Cyclic Redundancy Check (CRC) hardware for performing a plurality of CRC functions on the data; c. a data memory configured to store the data used in the iSCSI processor and store or retrieve said IP packets or said TCP/IP session information; d. an instruction memory configured to store a plurality of instructions used in the iSCSI processor, the instructions related to the data; e. an instruction fetch mechanism configured to fetch said instructions; f. an instruction decoder configured to decode said instructions; g. an instruction sequencer configured to sequentially organize said instructions; h. an iSCSI session database memory configured to store the state of the session on which said data is transported, said session database memory also for recording progress of RDMA for those of said data that use RDMA; i. an iSCSI session database memory controller configured to control said iSCSI session database memory; j. an interface coupled to a host processor, said interface configured to receive the instructions; said iSCSI processor coupled to a host system, said host system comprising one or more host processors and host buffers, said iSCSI processor chip further comprising; an iSCSI processing engine for offloading iSCSI processing from said one or more host processors to perform iSCSI data transfers; an execution resource for generating or processing said TCP/IP header and field information and for performing TCP/IP operations on said IP packets; an RDMA execution resource for generating or processing said RDMA header and field information and for performing RDMA operations on said IP packets that require RDMA operations; and a transmit engine or a receive engine, or a combination thereof, coupled to one of said one or more host processors for retrieving said data from said host buffers or performing direct data placement or zero copy in said host buffers without substantial intervention by said host processor. - View Dependent Claims (52)
- a) IP Storage Protocol Data Units, b) TCP/IP headers and fields and c) RDMA headers and fields, said iSCSI processor comprising;
-
53. A motherboard comprising a programmable Internet Small Computer System Interface (iSCSI) processor implemented on a chip having RDMA capability, said iSCSI processor chip for processing Internet Protocol (IP) packets during TCP/IP sessions in accordance with TCP/IP session information, said IP packets comprising at least one of:
- a) IP Storage Protocol Data Units, b) TCP/IP headers and fields and c) RDMA headers and fields, said iSCSI processor chip having a memory for storing or retrieving said Internet Protocol Packets or said TCP/IP session information, and said iSCSI processor chip capable of being coupled to one or more host processors and host buffers, said iSCSI processor chip comprising;
an iSCSI processing engine for offloading iSCSI processing from said one or more host processors to perform iSCSI data transfers; a memory controller for controlling access to said memory for storing or retrieving said TCP/IP session information or said Internet Protocol packets; an execution resource for generating or processing said TCP/IP header and field information and for performing TCP/IP operations on said Internet Protocol packets; a network interface to send or receive said Internet Protocol packets from or to said iSCSI processor chip; a host interface controller to couple said iSCSI processor chip to said one or more host processors; an RDMA execution resource for generating or processing said RDMA header and field information and for performing RDMA operations on said Internet Protocol packets that require RDMA operations; and a transmit engine or a receive engine, or a combination thereof, coupled to one of said one or more host processors for retrieving data from said host buffers or performing direct data placement or zero copy in said host buffers without substantial intervention by said host processor.
- a) IP Storage Protocol Data Units, b) TCP/IP headers and fields and c) RDMA headers and fields, said iSCSI processor chip having a memory for storing or retrieving said Internet Protocol Packets or said TCP/IP session information, and said iSCSI processor chip capable of being coupled to one or more host processors and host buffers, said iSCSI processor chip comprising;
-
54. A server comprising a server chipset, one or more host processors and host buffers, and a programmable Internet Small Computer System Interface (iSCSI) processor implemented on a chip having RDMA capability, said iSCSI processor chip for processing Internet Protocol (IP) packets during TCP/IP sessions in accordance with TCP/IP session information, said IP packets comprising at least one of:
- a) IP Storage Protocol Data Units, b) TCP/IP headers and fields and c) RDMA headers and fields, said iSCSI processor chip having a memory for storing or retrieving said Internet Protocol Packets or said TCP/IP session information, said iSCSI processor chip comprising;
an iSCSI processing engine for offloading iSCSI processing from said one or more host processors to perform iSCSI data transfers; a memory controller for controlling access to said memory for storing or retrieving said TCP/IP session information or said Internet Protocol packets; an execution resource for generating or processing said TCP/IP header and field information and for performing TCP/IP operations on said Internet Protocol packets; a network interface to send or receive said Internet Protocol packets from or to said iSCSI processor chip; a host interface controller to couple said iSCSI processor chip to said one or more host processors; an RDMA execution resource for generating or processing said RDMA header and field information and for performing RDMA operations on said Internet Protocol packets that require RDMA operations; and a transmit engine or a receive engine, or a combination thereof, coupled to one of said one or more host processors for retrieving data from said host buffers or performing direct data placement or zero copy in said host buffers without substantial intervention by said host processor.
- a) IP Storage Protocol Data Units, b) TCP/IP headers and fields and c) RDMA headers and fields, said iSCSI processor chip having a memory for storing or retrieving said Internet Protocol Packets or said TCP/IP session information, said iSCSI processor chip comprising;
-
55. A programmable Internet Small Computer System Interface (iSCSI) processor embodied inside a processor chipset, said chipset:
-
(i) having RDMA capability, (ii) capable of processing Internet Protocol (IP) packets during TCP/IP sessions in accordance with TCP/IP session information, said IP packets comprising at least one of;
a) IP Storage Protocol Data Units, b) TCP/IP headers and fields and c) RDMA headers and fields,(iii) having a memory for storing or retrieving said Internet Protocol Packets or said TCP/IP session information, (iv) capable of being coupled to a host system, said host system comprising one or more host processors and host buffers, and (v) comprising; (a) an iSCSI processing engine for offloading iSCSI processing from said one or more host processors to perform iSCSI data transfers; (b) a memory controller for controlling access to said memory for storing or retrieving said TCP/IP session information or said Internet Protocol packets; (c) an execution resource for generating or processing said TCP/IP header and field information and for performing TCP/IP operations on said Internet Protocol packets; (d) a network interface to send or receive said Internet Protocol packets from or to said iSCSI processor chip; (e) a host interface controller to couple said iSCSI processor chip to said one or more host processors; (f) an RDMA execution resource for generating or processing said RDMA header and field information and for performing RDMA operations on said Internet Protocol packets that require RDMA operations; and (g) a transmit engine or a receive engine, or a combination thereof, coupled to one of said one or more host processors for retrieving data from said host buffers or performing direct data placement or zero copy in said host buffers without substantial intervention by said host processor.
-
Specification