Hardware packet pacing using a DMA in a parallel computer

• US 8,509,255 B2
• Filed: 06/26/2007
• Issued: 08/13/2013
• Est. Priority Date: 06/26/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chen, Dong, Heidelberger, Phillip, Vranas, Pavlos
• Primary Examiner(s): Patel, Jay P
• Application Number:
  US11/768,682
• Publication Number:
  US 20090003203A1
• Time in Patent Office: 2,240 Days
• Field of Search: None
• US Class Current: 370/413
• CPC Class Codes: G06F 13/128 for dedicated transfers to a network for protocol converters G06F13/387

Abstract

Method and system for hardware packet pacing using a direct memory access controller in a parallel computer which, in one aspect, keeps track of a total number of bytes put on the network as a result of a remote get operation, using a hardware token counter.

Claims

1. A method of hardware packet pacing using a direct memory access controller in a multi-node parallel computer, comprising:

establishing in a first node a token counter on a direct memory access controller initially set to a first predetermined value;

establishing in the first node maximum pacing submessage size, the maximum pacing submessage size being a value less than or equal to the first predetermined value;

establishing in the first node a remaining bytes count, the remaining bytes count initially set to a message length field value in an original remote get packet;

setting in the first node a submessage size to the maximum pacing submessage size or the remaining bytes count, whichever is less;

waiting in the first node for the token counter to be greater than or equal to the submessage size;

injecting a remote get packet of the submessage size to a network of at least one or more other nodes when the token counter is greater than or equal to the submessage size and decrementing the token counter and the remaining bytes count by the submessage size; and

repeating the steps of setting, waiting and injecting until the remaining bytes count is zero.

2. The method of claim 1, further including:

detecting a put packet received on the direct memory access controller; and

incrementing the token byte counter by a number of payload bytes specified in the put packet.

3. The method of claim 1, wherein the step of injecting a remote get packet of the submessage size further includes assembling the remote get packet to include a put descriptor having a message size field set to the submessage size.

4. The method of claim 3, wherein the step of injecting a remote get packet of the submessage size further includes updating offsets of the remote get packet to offset plus the submessage size.

5. The method of claim 1, further including:

the direct memory access controller detecting a message descriptor associated with the original remote get in an injection fifo.

6. The method of claim 5, further including:

reading a pacing bit in the message descriptor associated with the original remote get message descriptor; and

performing the steps of claim 1 if the pacing bit is set.

7. The method of claim 1, further including:

the direct memory access controller detecting a plurality of original remote get message descriptors from a plurality of injection fifos; and

performing the steps of claim 1 for the plurality of original remote get packets.

8. The method of claim 1, wherein the maximum pacing submessage size is specified in the message descriptor of the original remote get message descriptor.

9. The method of claim 1, further including:

establishing a plurality of token counters; and

assigning each of the plurality of token counters to a different injection fifo in memory, wherein the direct memory access controller uses a token counter assigned to an injection fifo in which the original remote get packet is injected.

10. The method of claim 1, further including:

establishing a plurality of token counters; and

using a selected token counter.

11. The method of claim 10, wherein the selected token counter is specified in the message descriptor of the original remote get.

12. The method of claim 10, wherein further including:

establishing a plurality of maximum pacing submessage sizes; and

specifying in a message descriptor of the original remote get which one of the plurality of maximum pacing submessage sizes to use.

13. The method of claim 10, wherein a register in the direct memory access controller specifies the selected token counter to use for a message descriptor associated with the original remote get based on an injection fifo in which the message descriptor associated with the original remote get is injected.

14. The method of claim 10, further including:

detecting a put packet received on the direct memory access controller;

determining from the put packet a token counter to use from the plurality of token counters; and

incrementing the determined token counter by a message length specified in the put packet.

15. A method of hardware packet pacing using a direct memory access controller in a multi-node parallel computer, comprising:

detecting in a first node a remote get message descriptor in an injection fifo associated with a direct memory controller;

retrieving in the first node put descriptor information from information specified in the remote get message descriptor, the put description information including at least a message size, an injection offset, a reception offset, an injection counter identifier and a reception counter identifier;

setting in the first node a pacing submessage size;

setting in the first node a token counter to a predetermined value greater than the pacing size;

assembling in the first node a new remote get packet using the put description information, the new remote get packet specifying a message size that is at most the pacing size;

before sending the new remote get packet, waiting in the first node until the token counter is greater than or equal to the message size specified in the new remote get packet;

sending the new remote get packet to at least one or more other nodes;

decrementing in the first node the token counter by the message size specified in the new remote get packet; and

incrementing in the first node the injection offset and the reception offset by the message size specified in the new remote get packet; and

repeating the steps of assembling, waiting, sending, decrementing and incrementing until the message size of the remote get has been processed by all the new remote get packets.

16. The method of claim 15, further including:

detecting a put packet; and

incrementing the token counter by a number of payload bytes specified in the put packet.