Token ID mechanism for network data transfer

• US 7,804,862 B1
• Filed: 05/25/2005
• Issued: 09/28/2010
• Est. Priority Date: 05/25/2004
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: QLogic Corporation (Marvell Technology Group Limited)
• Inventors: Rubin, Jeffrey B., Olson, Dave, Lindahl, Gregory B.
• Primary Examiner(s): Harper; Kevin C
• Assistant Examiner(s): WONG, XAVIER S
• Application Number:
  US11/137,920
• Time in Patent Office: 1,952 Days
• Field of Search: 370/395.1, 370/395.7, 370/395.2, 370/395.3, 370/450, 370/236, 370/546, 709/203, 711/202
• US Class Current: 370/546
• CPC Class Codes: G06F 13/126 and has means for transferring I/O instructions and statuses between control unit and main processor

Abstract

A node in a network that receives data from a data source, such as another node. The receiving node sends a token identifier to the data source and receives data from the data source, along with the token identifier. A token identifier identifies a location in memory on the receiving node, but is not the same as an address in the memory. Thus, a token identifier is preferably neither a physical memory address nor a virtual address.

Claims

1. A method performed by a receiving node in a network, comprising:

the receiving node receiving an indication from the network that a sending node wants to send data to the receiving node, the indication from the sending node including a field for identifying a request for sending data using token identifiers;

the receiving node sending to the sending node a token identifier identifying a memory location on the receiving node, an offset value that is updated when a transfer is split into multiple packets, and a data size for the memory location, the token identifier being a value that acts as an index into a token array, which identifies the memory location, but is not a physical or virtual address, so that the sending node is unaware of the memory location;

the receiving node receiving from the sending node the token identifier and at least a first portion of the data;

writing the data to the receiving node at the memory location;

using a first token identifier array for processing the data, if the data is unexpected, as indicated by a field of the token identifier; and

using a second token identifier array for processing the data, if the data is expected.

2. The method of claim 1, wherein the indication that a sending node wants to send data to the receiving node is in the form of a Message Passing Interface (MPI) message containing a length of the data.

3. The method of claim 1, wherein the receiving node comprises an interface and at least one host processor.

4. The method of claim 3, wherein the interface and the host processor communicate via a Hypertransport protocol.

5. The method of claim 3, wherein the interface and the host processor communicate via a Peripheral Component Interconnect (PCI) Express protocol.

6. The method of claim 1, wherein receiving, by the receiving node from the sending node, the token identifier and at least a first portion of the data further includes receiving an InfiniBand packet format modified to include at least the token identifier.

7. The method of claim 1, wherein receiving, by the receiving node from the sending node, the token identifier and at least a first portion of the data further includes receiving an offset value into a memory buffer on the host.

8. The method of claim 7, further comprising receiving, by the receiving node from the sending node, the token identifier and at least a second portion of the data, along with a second offset into the memory buffer on the host where the second portion of data is to be stored.

9. The method of claim 1, wherein receiving, by the receiving node from the sending node, the token identifier and at least a first portion of the data further includes receiving memory header data that includes the token identifier.

10. The method of claim 1, wherein receiving, by the receiving node from the sending node, the token identifier and at least a first portion of the data further includes receiving memory header data that includes the token identifier and an offset into a memory buffer on a host.

11. The method of claim 1, wherein receiving, by the receiving node from the sending node, the token identifier and at least a first portion of the data further includes receiving memory header data including the token identifier and a port on a host.

12. The method of claim 1, wherein receiving, by the receiving node from the sending node, the token identifier and at least a first portion of the data further includes receiving memory header data including the token identifier and a version of the packet format.

13. The method of claim 1, wherein receiving, by the receiving node from the sending node, the token identifier and at least a first portion of the data further includes receiving a cut-through checksum based at least on data in the memory header but not the received data.

14. The method of claim 1 wherein the method is performed by an I/O interface communicating with a host processor.

15. The method of claim 14, wherein the I/O interface is a semiconductor chip.

16. The method of claim 14, wherein the I/O interface is a Field Programmable Gate Array (FPGA).

17. The method of claim 14, wherein the host processor determines the token identifier and which memory location on the host is associated with the token identifier.

18. The method of claim 1, further comprising validating the received packet before the data is written to the memory.

19. The method of claim 1, further comprising checking that the token identifier identifies a valid address in memory before the data is written to the memory.

20. The method of claim 1, further comprising checking that the data will not overrun an associated memory buffer before the data is written to the memory.

21. The method of claim 1, further comprising beginning to Direct Memory Access (DMA) the received data to the memory before all of the data is received.

22. A method performed by an interface of a receiving node in a network between a host, which has a memory, and a network, the method comprising:

the interface of the receiving node receiving an indication from a sending node in the network that the sending node wants to send data to the receiving node, wherein the indication includes a field for identifying a request for sending data using token identifiers;

the receiving node sending to the sending node a token identifier identifying a memory location on the receiving node, an offset value that is updated when a transfer is split into multiple packets, and a data size for the memory location, wherein the token identifier is a value that acts as an index into a token array, which identifies the memory location, but is not a physical or virtual address, so that the sending node is unaware of the memory location;

the interface of the receiving node receiving a packet having at least a first portion of data and a token identifier;

extracting the token identifier from the received packet;

determining a memory address in the memory of the host in accordance with the token identifier;

writing the data to the memory of the host at the determined memory address;

using a first token identifier array for processing the data, if the data is unexpected, as indicated by a field of the token identifier; and

using a second token identifier array for processing the data, if the data is expected.

23. The method of claim 22, wherein writing the data comprises writing the data to the memory of the host using Direct Memory Access (DMA).

24. The method of claim 22, wherein the interface is an interface chip communicating with the host.

25. An interface to a host in a network, which has a memory, the interface comprising:

a token sending unit for sending a token identifier identifying a memory location on the host to a packet sending node in the network, an offset value that is updated when a transfer is split into multiple packets, and a data size for the memory location, the token identifier being a value that acts as an index into a token array, which identifies the memory location, but is not a physical or virtual address, so that the sending node is unaware of the memory location, and before the interface sends the token identifier, the interface receives an indication from the sending node that the sending node wants to send data to the host, and the indication includes a field for identifying a request for sending data using token identifiers;

a staging buffer for holding a packet received by the interface from the sending node, the packet received from the sending node containing at least received data and the token identifier;

a first token identifier array indexed by the token identifier to yield an address of the memory location, if the data is expected data;

a second token identifier array indexed by the token identifier to yield the address of the memory location, if the data is unexpected data, as indicated by a field of the token identifier; and

an engine to send the data to the extracted memory location of the host.

26. The interface of claim 25, where the engine is a Direct Memory Access (DMA) engine.

27. The interface of claim 25, further comprising a validation engine to validate the token identifier before it is used as an index.

28. The interface of claim 25, further comprising a validation engine to validate the memory location before data is written to the memory location.

29. The interface of claim 25, further comprising a validation engine to validate the length of the data before it is written to the memory location.

30. The interface of claim 25, further comprising an engine to communicate with the host to obtain the token identifier before the token identifier is sent.

31. The interface of claim 25, further comprising a validation engine to validate checksum in accordance with the token identifier but not in accordance with the received data.

32. The interface of claim 25, wherein a packet header is sent to the host to be placed in a Receive Header Queue of the host.

33. The interface of claim 32, wherein the interface updates a tail pointer to the Receive Header Queue after the data has been written to the memory of the host.

34. A system, comprising:

a host system; and

an interface to the host system connected to the host system and to a network, the interface including;

a module that receives an indication that a sending node in the network wants to send data to the host system, the indication including a field for identifying a request for sending data using token identifiers;

a module that sends to the sending node a token identifier identifying a memory location on the host system, an offset value that is updated when a transfer is split into multiple packets, and a data size for the memory location, the token identifier being a value that acts as an index into a token array, which identifies the memory location, but is not a physical or virtual address, so that the sending node is unaware of the memory location;

a module that receives, from the sending node, the token identifier and at least a first portion of the data, and

a module that writes the data at the memory location;

a first token identifier array for processing the data, if the data is unexpected, as indicated by a field of the token identifier; and

a second token identifier array for processing the data, if the data is expected.