System and method for ordering of data transferred over multiple channels

• US 9,432,299 B2
• Filed: 03/02/2015
• Issued: 08/30/2016
• Est. Priority Date: 07/21/2000
• Status: Expired due to Term

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Silicon Graphics International Corporation (Hewlett-Packard Enterprise Company)
• Inventors: Martin, Randal G., Miller, Steven C., Stadler, Mark D., Kruckemyer, David A.
• Primary Examiner(s): Haile, Feben M
• Application Number:
  US14/635,708
• Publication Number:
  US 20150188832A1
• Time in Patent Office: 547 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04L 45/24 Multipath
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/26 using explicit feedback to the source, e.g. choke packets
• CPC Class Codes: H04L 47/266 Stopping or restarting the source, e.g. X-on or X-off
• CPC Class Codes: H04L 47/34 ensuring sequence integrity, e.g. using sequence numbers
• CPC Class Codes: H04L 47/39 Credit based
• CPC Class Codes: H04L 47/41 by acting on aggregated flows or links

Abstract

A multiple channel data transfer system (10) includes a source (12) that generates data packets with sequence numbers for transfer over multiple request channels (14). Data packets are transferred over the multiple request channels (14) through a network (16) to a destination (18). The destination (18) re-orders the data packets received over the multiple request channels (14) into a proper sequence in response to the sequence numbers to facilitate data processing. The destination (18) provides appropriate reply packets to the source (12) over multiple response channels (20) to control the flow of data packets from the source (12).

Claims

1. A method for transferring data packets, the method comprising a packet source:

receiving packet data from a data source;

initializing a credit counter with a number of credits by a reset signal from the data source;

transmitting a first packet of a plurality of packets over a first network request interface of a plurality of network request interfaces at the packet source according to capacity and flow control instructions, wherein the credit counter is adjusted in a first direction after the first packet of the plurality of packets is transmitted over the first network interface of the plurality of network request interfaces;

transmitting a second packet of the plurality of packets over a second network request interface of the plurality of network request interfaces at the packet source according to the capacity and flow control instructions, wherein the credit counter is adjusted in the first direction after the second packet of the plurality of packets is transmitted over the second network interface of the plurality of network request interfaces, wherein the plurality of packets are received and re-ordered at a destination after at least the first and the second packet are received at the destination;

sending an indication to the data source, the indication identifying that the credit counter has crossed below a threshold value, wherein the indication causes the data source to stop sending the packet data;

receiving one or more responses from the destination over a plurality of network response interfaces at the packet source including credits indicating that the destination has processed data packets and has capability to receive more data packets, each response of the one or more responses corresponding to a specific packet of the one or more packets transmitted over the plurality of network request interfaces, wherein the credit counter is adjusted in a second direction after receiving a response of the one or more responses over the plurality of network response interfaces; and

sending an indication to the data source, the indication identifying that the credit counter has crossed above the threshold value, wherein the indication causes the data source to send additional packet data.

2. The method of claim 1, further comprising:

receiving an interrupt at the packet source from the data source after the data source has stopped sending the packet data; and

receiving additional packet data from the data source;

initializing the credit counter;

transmitting one or more additional packets over one or more network request interfaces at the packet source, wherein the credit counter is adjusted in the first direction after each of the one or more additional packets are transmitted over the plurality of network request interfaces; and

receiving one or more additional responses over the one or more network response interfaces at the packet source, wherein the credit counter is adjusted in the second direction after receiving a response of the one or more additional responses, each response of the one or more additional responses corresponding to a specific packet of the one or more additional packets transmitted.

3. The method of claim 1, further comprising:

that the receiving the additional packet data from the data source; and

transmitting one or more additional packets over the at least one of the plurality of network request interfaces at the packet source, wherein the credit counter is adjusted in the first direction after each of the one or more additional packets are transmitted over the plurality of network request interfaces.

4. The method of claim 1, wherein the first direction corresponds to decrementing the credit counter and the second direction corresponds to incrementing the credit counter.

5. The method of claim 1, wherein the first direction corresponds to incrementing the credit counter and the second direction corresponds to decrementing the credit counter.

6. The method of claim 1, wherein crossing the threshold value is detected by comparing the credit counter to a maximum sequence number.

7. A non-transitory computer readable medium having embodied thereon a program executed by a process to perform a method for transferring data packets, the method comprising a packet source:

receiving packet data from a data source;

initializing a credit counter with a number of credits by a reset signal from the data source;

transmitting a first packet of a plurality of packets over a first network request interface of a plurality of network request interfaces at the packet source according to capacity and flow control instructions, wherein the credit counter is adjusted in a first direction after the first packet of the plurality of packets is transmitted over the first network interface of the plurality of network request interfaces;

transmitting a second packet of the plurality of packets over a second network request interface of the plurality of network request interfaces at the packet source according to the capacity and flow control instructions, wherein the credit counter is adjusted in the first direction after the second packet of the plurality of packets is transmitted over the second network interface of the plurality of network request interfaces, wherein the plurality of packets are received and re-ordered at a destination after at least the first and the second packet are received at the destination;

sending an indication to the data source, the indication identifying that the credit counter has crossed below a threshold value, wherein the indication causes the data source to stop sending the packet data;

receiving one or more responses from the destination over a plurality of network response interfaces at the packet source including credits indicating that the destination has processed data packets and has capability to receive more data packets, each response of the one or more responses corresponding to a specific packet of the one or more packets transmitted over the plurality of network request interfaces, wherein the credit counter is adjusted in a second direction after receiving a response of the one or more responses over the plurality of network response interfaces; and

sending an indication to the data source, the indication identifying that the credit counter has crossed above the threshold value, wherein the indication causes the data source to send additional packet data.

8. The non-transitory computer readable storage medium of claim 7, the program being further executable to:

receive an interrupt at the packet source from the data source after the data source has stopped sending the packet data;

receive additional packet data from the data source;

initialize the credit counter;

transmit one or more additional packets over the plurality of network request interfaces at the packet source, wherein the credit counter is adjusted in the first direction after each of the one or more additional packets are transmitted over the one or more network request interfaces; and

receive one or more additional responses over the one or more network response interfaces at the packet source, wherein the credit counter is adjusted in the second direction after receiving a response of the one or more additional responses, each response of the one or more responses corresponding to a specific packet of the one or more additional packets transmitted.

9. The non-transitory computer readable storage medium of claim 7, the program further executable to:

receive the additional packet data from the data source; and

transmit one or more additional packets over the at least one of the plurality of network request interfaces at the packet source, wherein the credit counter is adjusted in the first direction after each of the one or more additional packets are transmitted over the plurality of network request interfaces.

10. The non-transitory computer readable storage medium of claim 7, wherein the first direction corresponds to decrementing the credit counter and the second direction corresponds to incrementing the credit counter.

11. The method of claim 7, wherein the first direction corresponds to incrementing the credit counter and the second direction corresponds to decrementing the credit counter.

12. The method of claim 7, wherein crossing the threshold value is detected by comparing the credit counter to a maximum sequence number.

13. A system for transferring data packets from a packet source, the system comprising:

a memory;

a processor; and

a plurality of network communication interfaces, wherein

packet data is received from a data source;

a credit counter is initialized counter with a number of credits by a reset signal from the data source;

a first packet of a plurality of packets is transmitted over a first network request interface of the plurality of network request interfaces at the packet source according to capacity and flow control instructions, wherein the credit counter is adjusted in a first direction after the first packet of the plurality of packets is transmitted over the plurality of network request interfaces;

a second packet of a plurality of packets is transmitted over a second network request interface of the plurality of network request interfaces at the packet source according to the capacity and flow control instructions, wherein the credit counter is adjusted in the first direction after the second packet of the plurality of packets is transmitted over the second network request interface of the plurality of network request interfaces, wherein the plurality of packets are received and re-ordered at a destination after at least the first and the second packet are received at the destination;

an indication is sent to the data source after determining that the credit counter has crossed below a threshold value wherein the indication causes the data source to stop sending the packet data;

one or more responses is received from the destination over a plurality of network response interfaces at the packet source including credits indicating that the destination has processed data packets and has capability to receive more data packets, each response of the one or more responses corresponding to a specific packet of the one or more packets transmitted over the plurality of network request interfaces, wherein the credit counter is adjusted in a second direction after receiving a response of the one or more responses over the plurality of network response interfaces; and

an indication is sent to the data source after determining that the credit counter has crossed above the threshold value, wherein the indication causes the data source to send additional packet data.

14. The system of claim 13, wherein:

an interrupt is received at the packet source from the data source after the data source has stopped sending the packet data;

additional packet data is received from the data source;

the credit counter is initialized;

one or more additional packets are transmitted over the one or more network request interfaces at the packet source, wherein the credit counter is adjusted in the first direction after each of the one or more additional packets are transmitted over the one or more network request interfaces; and

one or more additional responses are received over one or more network response interfaces at the packet source, wherein the credit counter is adjusted in the second direction after receiving a response of the one or more additional responses, each response of the one or more additional responses corresponding to a specific packet of the one or more packets transmitted.

15. The system of claim 13, the wherein the packet source:

receives the additional packet data from the data source; and

transmits one or more additional packets over the at least one of the plurality of network request interfaces at the packet source, wherein the credit counter is adjusted in the first direction after each of the one or more additional packets are transmitted over the plurality of network request interfaces.

16. The system of claim 13, wherein the first direction corresponds to decrementing the credit counter and the second direction corresponds to incrementing the credit counter.

17. The system of claim 13, wherein the first direction corresponds to incrementing the credit counter and the second direction corresponds to decrementing the credit counter.

18. The system of claim 13, wherein crossing the threshold value is detected by comparing the credit counter to a maximum sequence number.