Credit-based flow control in lossless ethernet networks

• US 9,237,111 B2
• Filed: 03/14/2013
• Issued: 01/12/2016
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Banavalikar, Bhalachandra G., DeCusatis, Casimer M., Gusat, Mircea, Kamble, Keshav G., Recio, Renato J.
• Primary Examiner(s): Renner, Brandon
• Assistant Examiner(s): Preval, Voster
• Application Number:
  US13/830,858
• Publication Number:
  US 20140269274A1
• Time in Patent Office: 1,034 Days
• Field of Search: 709/235
• US Class Current: 1/1
• CPC Class Codes: H04L 12/4641 Virtual LANs, VLANs, e.g. virtual private networks [VPN] LAN interconnection over a bridge based backbone H04L12/462; encapsulation techniques H04L12/4633; routing of packets H04L45/00; packet switches H04L49/00; virtual private networks for security H04L63/0272
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 47/39 Credit based
• CPC Class Codes: H04L 49/10 characterised by the switching fabric construction

Abstract

In one embodiment, a system includes a hardware processor and logic integrated with and/or executable by the processor or media access control (MAC) functionality of a network port, the logic being adapted to initialize a link between a receiving endpoint and a sending endpoint, the receiving and sending endpoints being connected in a network fabric, wherein at least one virtual link is created within the link, receive an amount of available flow credits from the receiving endpoint, wherein the amount of available flow credits are used to determine a capacity to process packets at the receiving endpoint, and transmit one or more packets to the receiving endpoint until all packets are sent or the amount of available flow credits is insufficient to process additional packets, wherein exchange of flow credits is performed on a per virtual link basis.

Claims

1. A system, comprising:

a hardware processor and logic integrated with and/or executable by the processor or media access control (MAC) functionality of a network port, the logic being configured to;

initialize a link between a receiving endpoint and a sending endpoint, the receiving and sending endpoints being connected in a network fabric, wherein at least one virtual link is created within the link;

negotiate a credit aging duration during initialization of the link, wherein the credit aging duration is determined at least partially based on a speed of the link between the receiving endpoint and the sending endpoint;

receive an amount of available flow credits from the receiving endpoint, wherein the amount of available flow credits are used to determine a capacity to process packets at the receiving endpoint;

transmit one or more packets to the receiving endpoint until all packets are sent or the amount of available flow credits is insufficient to process additional packets; and

expire unused flow credits from the amount of available flow credits as an age of the unused flow credits reaches the credit aging duration,

wherein the receiving endpoint and the sending endpoint each track an age of flow credits using a local clock or a synchronized clock, and

wherein exchange of flow credits is performed on a per virtual link basis.

2. The system as recited in claim 1, wherein the logic is further configured to expire unused flow credits from the amount of available flow credits as their age reaches the credit aging duration.

3. The system as recited in claim 1, wherein each flow credit allows for one or more quanta of data to be transmitted, wherein data successfully received is measured in quanta, and wherein successfully received data includes data which is received with an amount of error below a predetermined threshold including data which is received without error.

4. The system as recited in claim 1, wherein the logic is further configured to:

decrease the amount of available flow credits by an amount corresponding to data in the one or more packets that is successfully received by the receiving endpoint, wherein successfully received data includes data which is received with an amount of error below a predetermined threshold including data which is received without error; and

provide replacement flow credits to the receiving endpoint equal to the data in the one or more packets successfully forwarded by the receiving endpoint, wherein successfully forwarded data includes data which is consumed by a network interface card (NIC) driver of the receiving endpoint, the NIC driver being used to forward packets away from the receiving endpoint.

5. A computer program product for providing credit-based flow control, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code readable/executable by a processor to:

initialize, using the processor, a link between a receiving endpoint and a sending endpoint, the receiving and sending endpoints being connected in a network fabric, the link comprises at least one virtual link;

negotiate, using the processor, a credit aging duration during initialization of the link, wherein the credit aging duration is used to expire flow credits after the credit aging duration is reached rendering the flow credits unusable;

send, using the processor, an amount of available flow credits to the sending endpoint, wherein the amount of available flow credits are used to determine a capacity to process packets;

receive, using the processor, one or more packets from the sending endpoint;

decrease, using the processor, the amount of available flow credits by an amount corresponding to data in the one or more packets that is successfully received; and

receive, using the processor, replacement flow credits equal to the data in the one or more packets successfully forwarded by the receiving endpoint,

wherein exchange of flow credits is performed on a per virtual link basis.

6. The computer program product as recited in claim 5, wherein the credit aging duration is determined based on at least one characteristic of the link.

7. The computer program product as recited in claim 6, wherein the credit aging duration is at least partially determined based on a speed of the link between the receiving endpoint and the sending endpoint and a bit size of the flow credits, and wherein the receiving endpoint and the sending endpoint each track an age of flow credits using a local clock or a synchronized clock.

8. The computer program product as recited in claim 6, wherein the program code readable/executable by the processor is further configured to expire unused flow credits from the amount of available flow credits as their age reaches the credit aging duration.

9. The computer program product as recited in claim 5, wherein each flow credit allows for one or more quanta of data to be transmitted, wherein data successfully received is measured in quanta, wherein successfully received data includes data which is received with an amount of error below a predetermined threshold including data which is received without error, and wherein successfully forwarded data includes data which is consumed by a network interface card (NIC) driver of the receiving endpoint, the NIC driver being used to forward packets away from the receiving endpoint.

10. A method for providing credit-based flow control, the method comprising:

initializing a link between a receiving endpoint and a sending endpoint, the receiving and sending endpoints being connected in a network fabric, wherein the link comprises at least one virtual link;

negotiating a credit aging duration during initialization of the link, wherein the credit aging duration is used to expire flow credits after the credit aging duration is reached, wherein the credit aging duration is determined at least partially based on a speed of the link between the receiving endpoint and the sending endpoint; and

in an iterative process until the sending endpoint has no further packets to transmit;

sending an amount of available flow credits from the receiving endpoint to the sending endpoint, wherein the amount of available flow credits are used to determine a capacity to process packets at the receiving endpoint;

receiving, at the receiving endpoint, one or more packets transmitted from the sending endpoint;

decreasing the amount of available flow credits by an amount corresponding to data in the one or more packets that is successfully received by the receiving endpoint;

expiring unused flow credits from the amount of available flow credits as an a of the unused flow credits reaches the credit aging durations and

receiving, at the receiving endpoint, an amount of flow credits transmitted from the sending endpoint in an amount equivalent to the data in the one or more packets successfully forwarded by the receiving endpoint,

wherein exchange of flow credits is performed on a per virtual link basis.

11. The method as recited in claim 10, wherein the credit aging duration is determined based on a round trip time (RTT) of the link.

12. The method as recited in claim 10, wherein each flow credit allows for one or more quanta of data to be transmitted, wherein data successfully received is measured in quanta, and wherein successfully received data includes data which is received with an amount of error below a predetermined threshold including data which is received without error.

13. The method as recited in claim 10, wherein initially, the amount of available flow credits is equal to a total capacity of flow credits at the receiving endpoint.

14. The method as recited in claim 10, wherein the amount of available flow credits are replenished for every quanta successfully consumed by a network interface card (NIC) driver of the receiving endpoint which is used to forward the one or more received packets away from the receiving endpoint.

15. The method as recited in claim 10, further comprising tracking an age of flow credits using a local clock or a synchronized clock using the receiving endpoint.

16. The method as recited in claim 15, further comprising determining the credit aging duration based on a round trip time (RTT) of the link.

17. A system, comprising a hardware processor and logic integrated with and/or executable by the processor, the logic being configured to:

generate a plurality of flow credits;

determine a credit aging duration based at least partially on speeds of links between interface and/or ports, wherein the credit aging duration is used to expire flow credits after the credit aging duration is reached;

distribute one or more flow credits to every interface and/or port to be exchanged on a per virtual link basis; and

expire unused flow credits from an amount of available flow credits at one or more interface and/or port as an age of the unused flow credits reaches the credit aging duration,

wherein the flow credits are exchanged with other devices in order to provide for processing of packets sent thereto, and

wherein an amount of flow credits distributed to an interface and/or port is proportionate to at least one of;

a bandwidth of the interface and/or port and according to a predetermined policy.

18. The system as recited in claim 17, wherein every interface and/or port are local to the system, wherein the other devices are external to the system, and wherein the policy dictates bandwidth control for each interface and/or port in the system.

19. The system as recited in claim 17, wherein the logic is further configured to facilitate implementation of various bandwidth management policies on the system, on interfaces, and/or on ports.

20. The system as recited in claim 19, wherein the flow credit generation and distribution is configured to be controlled by a switch controller in communication with the system, wherein the switch controller is configured to govern the various bandwidth management policies.