Gigabit Ethernet adapter supporting the iSCSI and IPSEC protocols

• US 20040062267A1
• Filed: 06/05/2003
• Published: 04/01/2004
• Est. Priority Date: 03/06/2002
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: Johnson, Michael Ward, Uyeshiro, Robin Yasu, Minami, John Shigeto, Smith, Michael John Sebastian, Chen, Addison Kwuanming, Doctor, Mihir Shaileshbhai, Greenfield, Daniel Leo, Su, Steve
• Granted Patent: US 7,535,913 B2
• Time in Patent Office: Days
• US Class Current: 370/463
• CPC Class Codes: H04L 45/00 Routing or path finding of packets in data switching networks routing or path finding in wireless networks H04W40/00
• CPC Class Codes: H04L 45/60 Router architectures
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/12 Avoiding congestion; Recovering from congestion
• CPC Class Codes: H04L 47/2408 for supporting different services, e.g. a differentiated services [DiffServ] type of service
• CPC Class Codes: H04L 47/2441 relying on flow classification, e.g. using integrated services [IntServ]
• CPC Class Codes: H04L 47/37 Slow start
• CPC Class Codes: H04L 47/50 Queue scheduling
• CPC Class Codes: H04L 47/6215 Individual queue per QOS, rate or priority
• CPC Class Codes: H04L 47/6235 Variable service order
• CPC Class Codes: H04L 47/6255 queue load conditions, e.g. longest queue first
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability
• CPC Class Codes: H04L 69/12 Protocol engines
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/161 Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
• CPC Class Codes: H04L 69/162 involving adaptations of sockets based mechanisms secure socket layer H04L63/168
• CPC Class Codes: H04L 69/163 In-band adaptation of TCP data exchange; In-band control procedures
• CPC Class Codes: H04L 69/165 Combined use of TCP and UDP protocols; selection criteria therefor
• CPC Class Codes: H04L 69/166 IP fragmentation; TCP segmentation
• CPC Class Codes: H04L 69/18 Multiprotocol handlers, e.g. single devices capable of handling multiple protocols
• CPC Class Codes: H04L 9/40 : Network security protocols

Abstract

The invention is embodied in a gigabit Ethernet adapter. A system according to the invention provides a compact hardware solution to handling high network communication speeds. In addition, the invention adapts to multiple communication protocols via a modular construction and design.

Claims

1. An integrated network adapter for decoding and encoding network protocols and processing data, comprising:

a hardwired data path for processing streaming data;

a hardwired data path for receiving and transmitting packets and for encoding and decoding packets;

a plurality of parallel, hardwired protocol state machines;

wherein each protocol state machine is optimized for a specific network protocol; and

wherein said protocol state machines execute in parallel; and

means for scheduling shared resources based on traffic.

2. An integrated network adapter embodied in a single integrated circuit, said network adapter comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE;

a physical layer module (PHY);

a media-access layer module (MAC);

an IPsec processing engine integrated with said TOE; and

an upper-level protocol (ULP) for offload processing, said ULP integrated with said TOE.

3. The network adapter of claim 2, wherein said ULP implements an iSCSI protocol.

61. The network adapter of claim 3, wherein said ULP offloads the calculation of the iSCSI CRC for transmit and receive.

62. The network adapter of claim 3, wherein said ULP performs iSCSI framing using Fixed Interval Markers (FIM) for transmit.

63. The network adapter of claim 3, wherein said TOE accepts iSCSI header segments and iSCSI data segments from the host iSCSI driver and prepares iSCSI PDUs for transmission.

64. The network adapter of claim 3, further comprising:

an iSCSI driver resident on a host computer; and

wherein said host iSCSI driver communicates with said TOE.

65. The network adapter of claim 64, wherein said TOE receives iSCSI Protocol Data Units (PDU), calculates iSCSI CRCs, and passes the iSCSI CRCs to said host iSCSI driver.

66. The network adapter of claim 64, wherein said host iSCSI driver assembles a complete iSCSI Protocol Data Unit (PDU) header in host memory, creates an iSCSI Instruction Block (IB), and sends the iSCSI IB to said TOE.

67. The network adapter of claim 66, wherein an iSCSI IB contains a set of address and length pairs, known as transfer blocks, which correspond to a linked-list of buffers in host computer memory.

68. The network adapter of claim 67, wherein said host iSCSI driver adjusts buffer size of a final transfer block when receiving iSCSI data to account for CRC bytes and still allows correct separation of iSCSI header and data segments.

69. The network adapter of claim 64, wherein an iSCSI Protocol Data Unit (PDU), including its corresponding Basic Header Segment (BHS), any Additional Header Segment (AHS), and any data segment, is transferred between said host iSCSI driver and said TOE using an iSCSI Instruction Block (IB).

70. The network adapter of claim 65, wherein said host iSCSI driver seeds a calculated iSCSI CRC value using an iSCSI CRC seed field in an iSCSI Instruction Block (IB).

71. The network adapter of claim 64, wherein said host iSCSI driver splits iSCSI Protocol Data Unit (PDU) header and data segments on receive by posting receive buffers of the correct size for the iSCSI PDU header and, if there are data segments, posting receive buffers of the correct size for the iSCSI PDU data segment.

72. The network adapter of claim 71, wherein said host iSCSI driver posts correctly sized buffers for any Additional Header Segments (AHS) received by using instruction blocks.

73. The network adapter of claim 64, wherein said TOE and said host iSCSI driver interface at the iSCSI PDU level.

74. The network adapter of claim 64, wherein said TOE separates header and data segments of iSCSI Protocol Data Units (PDU) without requiring additional memory copies in the host computer'"'"'s memory by DMAing PDU headers to either said integrated processor or the host computer and DMAing PDU data sections to the host computer.

75. The network adapter of claim 3, wherein said TOE performs IPsec anti-replay support on a per-SA basis.

76. The network adapter of claim 3, wherein said TOE implements IPSec null, DES, 3DES algorithms, and AES 128-bit algorithm in cipher-block chaining (CBC) mode.

77. The network adapter of claim 3, wherein said TOE implements IPsec null, SHA-1 and MD-5 authentication algorithms.

78. The network adapter of claim 3, wherein said TOE implements IPsec variable-length encryption keys.

79. The network adapter of claim 3, wherein said TOE implements IPsec variable-length authentication keys.

80. The network adapter of claim 3, wherein said TOE implements IPsec jumbo frame support.

81. The network adapter of claim 3, wherein said TOE implements IPsec automatic handling of Security Association (SA) expiration on the basis of both time and total data transferred.

82. The network adapter of claim 3, wherein said TOE implements IPsec Policy Enforcement.

83. The network adapter of claim 3, wherein said TOE implements IPsec exception handling, including exception-packet generation and status reporting.

4. An integrated network adapter, comprising:

a hardwired data path for receiving and transmitting packets and for encoding and decoding packets;

at least one hardwired protocol state machine; and

at least one communication channel between said network adapter and a host computer.

5. The network adapter of claim 4, wherein said at least one communication channel employs instruction blocks (IBs) and status messages (SMs) to transfer data and control information.

6. The network adapter of claim 4, further comprising:

at least one threshold timer for controlling communication via said at least one communication channel;

wherein data are transferred at selected threshold interval.

7. The network adapter of claim 4, further comprising:

a module for establishing at least one data threshold for controlling communication via said at least one communication channel;

wherein data are transferred when data levels reach a selected threshold.

8. The network adapter of claim 6, wherein said timer threshold comprises an interrupt aggregation mechanism for reducing a number of interrupts between said network adapter and said host computer and for increasing data throughput.

9. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

an interrupt aggregation mechanism for optimizing data throughput of said processor and TOE.

10. The network adapter of claim 4, further comprising:

a module that provides optimized hardware support for TCP Selective Acknowledgement (SACK);

wherein TCP acknowledges missing data packets and retransmits said missing data packets, but only said missing data packets.

11. The network adapter of claim 4, further comprising:

a module that provides optimized hardware support for TCP slow start.

12. The network adapter of claim 11, wherein slow start slowly ramps up a number of data segments in flight at one time by:

initially only allowing two data segments that correspond to a current window, cwnd, of twice a maximum segment size (MSS) fly before expecting an acknowledgement (ACK); and

increasing cwnd by one MSS for each successful ACK received, to allow one more segment in flight, until cwnd is equivalent to a receiver'"'"'s advertised window.

13. The network adapter of claim 11, wherein slow start is always started on a new data connection;

and wherein slow start may be activated in the middle of a connection when a data traffic congestion event occurs.

14. The network adapter of claim 4, further comprising:

a module that provides optimized hardware support for TCP fast retransmit.

15. The network adapter of claim 14, wherein fast retransmit immediately generates an ACK when an out of order segment is received to allow a sender to fill a hole quickly, instead of waiting for a standard time out.

16. The network adapter of claim 14:

wherein fast retransmit is invoked when a receiver receives three duplicate ACKS;

wherein a sender tries to fill a hole when fast retransmit is invoked; and

wherein a duplicate ACK is considered duplicate when ACK and window advertisement values in a segment match one another.

17. The network adapter of claim 4, further comprising:

a module that provides optimized hardware support for TCP window scaling.

18. The network adapter of claim 17, wherein a window scaling operation is based on three variables, which comprise:

a least one bit for enabling window scale;

at least one bit for setting a scaling factor; and

a parameter for determining a scaling value.

19. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for iSCSI header and data CRC generation and checking.

20. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for iSCSI fixed-interval marker (FIM) generation.

21. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for TCP dump mode,

wherein TCP dump mode supports diagnostic programs and packet monitoring programs.

22. The network adapter of claim 21, wherein when TCP dump mode is enabled all received packets are sent to said host as exceptions and all outgoing TCP/UDP packets coming from a hardware stack are looped back as exception packets.

23. The network adapter of claim 22, further comprising:

a driver copying said exception packets for a network monitor, and for re-injecting rx packets and sending TX packets as raw Ethernet frames.

24. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for host ACK mode;

wherein a TCP ACK is only sent when said host has received data from a TCP segment to provide data integrity where data may be corrupted as they are passed between said host computer and said network adapter.

25. The network adapter of claim 24, wherein host ACK mode waits for a DMA of an MTX buffer that contains a data segment to complete before sending an ACK.

26. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for TCP timestamps to allow TCP to calculate a Round Trip Time Measurement (RTTM) better, and to support Protect Against Wrapped Sequences (PAWS).

27. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for TCP PAWS to protect against old duplicate segments corrupting TCP connections.

28. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for TCP host retransmit mode to allow retransmission of data directly out of a host'"'"'s memory buffers, instead of out of buffers located in said network adapter.

29. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for random initial sequence numbers.

30. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE;

a module that provides optimized hardware support for dual stack mode; and

a hardware TCP/IP stack integrated into said network adapter that works in cooperation and in conjunction with a software TCP/IP stack in said host;

wherein said network adapter supports co-existence of said software TCP/IP stack running in parallel using a same IP address as said network adapter.

31. The network adapter of claim 30, further comprising:

a module for supporting SYN status message mode;

wherein any received SYN generates a status message back to said host;

wherein SYN/ACK is not generated by said network adapter until said host sends a proper instruction block back to said network adapter; and

wherein if said SYN status message mode is not enabled on said network adapter, then SYN/ACKs are generated automatically by said network adapter, and SYN received status messages are not generated.

32. The network adapter of claim 30, further comprising:

a module for supporting suppression of RST messages from said network adapter when a TCP packet is received that does not match a network adapter control block database;

wherein, instead of automatically generating a RST, said network adapter hardware sends a packet to said host as an exception packet to allow said software TCP/IP stack in said host to handle said packet as an exception packet.

33. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for IP ID splitting to allow said host and said network adapter to share an IP address without overlapping IP ID'"'"'s.

34. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for filtering of data packets to restrict, accept, or take special action on certain types of packets.

35. The network adapter of claim 34, wherein said filtering can take any of the following attributes:

accept a programmed uni-cast address;

accept broadcast packets;

accept multicast packets;

accept addresses within a range specified by a netmask; and

allow a promiscuous mode that accepts all packets.

36. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for virtual local area network (VLAN).

37. The network adapter of claim 36, wherein said VLAN module comprises any of:

an element for stripping incoming packets of their VLAN headers;

an element for generating VLAN tagged outbound packets;

an element for generating VLAN parameters from incoming SYN frames; and

an element for passing VLAN tag information for exception packets and UDP packets.

38. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for jumbo frames.

39. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for Simple Network Management Protocol (SNMP).

40. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for management information base (MIB).

41. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for flexible and programmable memory error checking and correction (ECC).

42. The network adapter of claim 41:

wherein said ECC module uses at least one extra bit to store an encrypted ECC code with data in a packet;

wherein when said data are written to memory, said ECC code is also stored;

wherein when said data are read back, said stored ECC code is compared to an ECC code which would have been generated when said data were written;

wherein if said ECC codes do not match, a determination is made as to which bit in said data is in error;

wherein said bit in error is flipped and a memory controller releases said corrected data;

wherein errors are corrected on-the-fly, and corrected data are not placed back in said memory; and

wherein if same corrupt data are read again, operation of said ECC module is repeated.

43. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for network adapter operation in legacy modes;

wherein all network traffic are sent to said host regardless of traffic type; and

wherein said network adapter operate as if a hardware TCP/IP stack were not present therein.

44. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support that allows IP fragmentation to be handled in either of hardware and software;

wherein IP fragmented packets that are passed up as exception packets and reassembled in a software driver are re-injected via an IP injection mode back into said network adapter.

45. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for IP injection that allows IP packets to be injected into a TCP/IP stack in said network adapter.

46. The network adapter of claim 45, said IP injection module further comprising:

one or more injection control registers for injecting an IP packet into said network adapter TCP/IP stack;

wherein said one or more injection control registers allow said host to inject an IP packet into said network adapter TCP/IP stack.

47. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for any of Network Address Translation (NAT), IP masquerading, and port forwarding via port range registers that forward all packets of a specified type UDP or TCP that fall in a programmable range of ports to an exception path;

wherein said port registers enable certain ranges of ports to be used for network control operations and port forwarding.

48. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for multiple IP addresses.

49. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for a debug mode;

wherein When a test and control bit is enabled in said network adapter, all IP packets are sent as exceptions to said host.

50. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for TCP time wait state.

51. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for a variable number of connections.

52. The network adapter of claim 51, wherein when said network adapter accepts a connection that equals a network adapter maximum capacity, a next SYN is passed up to said host as an exception packet to allow said host to handle said connection.

53. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for User Datagram Protocol (UDP).

54. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for TTL (time to live) to limit an IP packet life on to a selected number of hops.

55. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for TCP keepalive to allow an idle TCP connection to stay connected and not time out by periodically sending a keep alive packet across a link.

56. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for TCP type of service (TOS) for use by routers to prioritize an IP packet.

57. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for TCP quality of service (QoS).

58. The network adapter of claim 57, wherein TCP transmit data flow starts with a Socket Query module, which goes through a transmit data available Bit table looking for entries that have Transmit Data Available bits set, and wherein when said Socket Query module finds such an entry, said Socket Query module puts that entry into one of a plurality of queues according to a socket'"'"'s User Priority level.

59. The network adapter of claim 4, further comprising:

a hardwired transport offload engine (TOE);

a processor integrated with said TOE; and

a module that provides optimized hardware support for failover.

60. The network adapter of claim 59, said failover module comprising:

a NO_SYN mode that allows a socket to be created without trying to initiate a connection;

wherein a socket and all its related data structures in said network adapter are created without creating a connection; and

wherein NO_SYN mode supports failover from another card or connection migration from a software TCP/IP stack to said network adapter.