Transfer ready frame reordering

• US 20030056000A1
• Filed: 07/24/2002
• Published: 03/20/2003
• Est. Priority Date: 07/26/2001
• Status: Abandoned Application

• Current Assignee: Brocade Communications Systems, Inc. (Broadcom, Inc.)
• Original Assignee: Nishan Systems, Inc. (Broadcom, Inc.)
• Inventors: Mehta, Anil, Oberman, Stuart F., Mullendore, Rodney N., Schakel, Keith, Malik, Kamran
• Application Number:
  US10/201,809
• Publication Number:
  US 20030056000A1
• Time in Patent Office: Days
• US Class Current: 709/236
• CPC Class Codes: H04L 47/245 using preemption
• CPC Class Codes: H04L 47/50 Queue scheduling
• CPC Class Codes: H04L 47/56 implementing delay-aware scheduling
• CPC Class Codes: H04L 47/6215 Individual queue per QOS, rate or priority
• CPC Class Codes: H04L 47/623 Weighted service order
• CPC Class Codes: H04L 47/624 Altering the ordering of packets in an individual queue
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/163 In-band adaptation of TCP data exchange; In-band control procedures
• CPC Class Codes: H04L 69/169 Special adaptations of TCP, UDP or IP for interworking of IP based networks with other networks protocols for interworking, protocol conversion H04L69/08
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A system and method for reordering received frames to ensure that transfer ready (XFER_RDY) frames among the received frames are handled at higher priority, and thus with lower latency, than other frames. In one embodiment, an output that is connected to one or more devices may be allocated an additional queue specifically for XFER_RDY frames. Frames on this queue are given a higher priority than frames on the normal queue. XFER_RDY frames are added to the high priority queue, and other frames to the lower priority queue. XFER_RDY frames on the higher priority queue are forwarded before frames on the lower priority queue. In another embodiment, a single queue may be used to implement XFER_RDY reordering. In this embodiment, XFER_RDY frames to be inserted in front of other types of frames in the queue.

Claims

1. A method comprising:

receiving a plurality of information units each comprising one or more frames destined for a first device, wherein the plurality of information units includes one or more transfer ready information units and one or more non-transfer-ready information units, and wherein at least one of the one or more non-transfer-ready information units is received before the one or more transfer ready information units;

adding the plurality of information units to at least one buffer, wherein the at least one buffer is configured to queue information units destined for the first device; and

conveying the queued plurality of information units from the at least one buffer to the first device;

wherein, in said conveying, the one or more transfer ready information units are conveyed to the first device prior to the one or more non-transfer-ready information units.

2. The method as recited in claim 1, wherein each of the one or more transfer ready information units is sent by a second device to the first device in response to a data write command previously sent from the first device to the second device, and wherein each of the one or more transfer ready information units indicates that the second device is ready to receive write data from the first device.

3. The method as recited in claim 1, wherein the plurality of information units are sent by a plurality of devices.

4. The method as recited in claim 1, wherein said receiving further comprises receiving one or more frames destined for a device other than the first device.

5. The method as recited in claim 1, wherein each of the one or more non-transfer-ready information units comprises a portion of read data requested in a data read command previously sent from the first device to a second device.

6. The method as recited in claim 1, wherein the at least one buffer comprises a first buffer and a second buffer, and wherein said adding the plurality of information units to the at least one buffer comprises:

adding the one or more transfer ready information units to the first buffer; and

adding the one or more non-transfer-ready information units to the second buffer.

7. The method as recited in claim 6, wherein said conveying the queued information units from the at least one buffer to the first device comprises:

conveying the one or more transfer ready information units from the first buffer to the first device; and

conveying the one or more non-transfer-ready information units from the second buffer to the first device after said conveying the one or more transfer ready information units from the first buffer to the first device.

8. The method as recited in claim 1, wherein the at least one buffer comprises a first buffer, and wherein said adding the plurality of information units to the at least one buffer comprises adding the plurality of information units to the first buffer, wherein the one or more transfer ready information units are added in front of the non-transfer-ready information units in the first buffer.

9. The method as recited in claim 8, wherein said conveying the queued information units from the at least one buffer to the first device comprises:

conveying the one or more transfer ready information units from the first buffer to the first device; and

conveying the one or more non-transfer-ready information units from the first buffer to the first device after said conveying the one or more transfer ready information units from the first buffer to the first device.

10. The method as recited in claim 1, wherein the information units are Fibre Channel Protocol for SCSI (FCP) frame sequences.

11. The method as recited in claim 1, wherein the information units are iSCSI (SCSI protocol over TCP/IP) frame sequences.

12. The method as recited in claim 1, wherein the information units are FCIP (Fibre Channel over Internet Protocol) frame sequences.

13. The method as recited in claim 1, wherein the information units are iFCP (Internet Fibre Channel Protocol) frame sequences.

14. The method as recited in claim 1, wherein the first device is a Fibre Channel host bus adapter.

15. The method as recited in claim 1, wherein the first device is an iSCSI (SCSI protocol over TCP/IP) host bus adapter.

16. The method as recited in claim 1, wherein the second device is a storage device.

17. The method as recited in claim 1, wherein said receiving, said adding, and said conveying are performed by a network switch.

18. The method as recited in claim 1, wherein said receiving, said adding, and said conveying are performed in a pipelined fashion.

19. The method as recited in claim 1, wherein the one or more transfer ready information units are received in a sequence, and wherein, in said conveying, the one or more transfer ready information units are conveyed to the first device in the sequence in which they were received.

20. A method comprising:

a first device sending a data read command to a second device;

the first device sending a data write command to the second device;

the second device sending one or more read data information units to the first device in response to the data read command, wherein the one or more read data information units each comprise a portion of read data requested in the data read command;

the second device sending a transfer ready information unit to the first device in response to the data write command, wherein the transfer ready information unit indicates that the second device is ready to receive write data from the first device;

receiving the one or more read data information units and the transfer ready information unit destined for the first device, wherein at least one of the one or more read data information units is received before the transfer ready information unit;

adding the one or more read data information units and the transfer ready information unit to at least one buffer, wherein the at least one buffer is configured to queue information units destined for the first device; and

conveying the one or more read data information units and the transfer ready information unit from the at least one buffer to the first device;

wherein, in said conveying, the transfer ready information unit is conveyed to the first device prior to the one or more read data information units.

21. The method as recited in claim 20, wherein the at least one buffer comprises a first buffer and a second buffer;

wherein said adding the one or more read data information units and the transfer ready information unit to the at least one buffer comprises;

adding the transfer ready information unit to the first buffer; and

adding the one or more read data information units to the second buffer;

wherein said conveying the one or more read data information units and the transfer ready information unit from the at least one buffer to the first device comprises;

conveying the transfer ready information unit from the first buffer to the first device; and

conveying the one or more read data information units from the second buffer to the first device after said conveying the one or more transfer ready information units from the first buffer to the first device.

22. The method as recited in claim 20, wherein the at least one buffer comprises a first buffer;

wherein said adding the one or more read data information units and the transfer ready information unit to the at least one buffer comprises adding the one or more read data information units and the transfer ready information unit to the first buffer, wherein the transfer ready information unit is added in front of the one or more read data information units in the first buffer;

wherein said conveying the one or more read data information units and the transfer ready information unit from the at least one buffer to the first device comprises;

conveying the transfer ready information unit from the first buffer to the first device; and

conveying the one or more read data information units from the first buffer to the first device after said conveying the transfer ready information unit from the first buffer to the first device.

23. The method as recited in claim 20, wherein the one or more read data information units and the transfer ready information unit are Fibre Channel Protocol for SCSI (FCP) frame sequences.

24. The method as recited in claim 20, wherein the first device is a Fibre Channel host bus adapter and the second device is a storage device.

25. The method as recited in claim 20, wherein said receiving, said adding, and said conveying are performed by a network switch.

26. The method as recited in claim 20, wherein said receiving, said adding, and said conveying are performed in a pipelined fashion.

27. A device comprising:

a first port operable to couple to a first device;

a second port operable to couple to a second device and configured to receive a plurality of information units each comprising one or more frames sent from the second device and destined for the first device, wherein the plurality of information units includes one or more transfer ready information units and one or more non-transfer-ready information units, and wherein at least one of the one or more non-transfer-ready information units is received by the device before the one or more transfer ready information units;

a memory comprising at least one buffer operable to queue information units in transit between the first and second ports of the device;

distribution logic coupled between the first port and the memory and configured to add the plurality of information units to the at least one buffer; and

scheduler logic coupled between the memory and the second port and configured to convey the queued plurality of information units from the at least one buffer to the first device;

wherein, in said conveying, the scheduler logic is further configured to convey the one or more transfer ready information units to the first device prior to conveying the one or more non-transfer-ready information units.

28. The device as recited in claim 27, wherein each of the one or more transfer ready information units is sent by the second device to the first device in response to a data write command previously sent from the first device to the second device, and wherein each of the one or more transfer ready information units indicates that the second device is ready to receive write data from the first device.

29. The device as recited in claim 27, wherein at least one of the one or more non-transfer ready information units comprises a portion of read data requested in a data read command previously sent from the first device to the second device.

30. The device as recited in claim 27, wherein the plurality of information units are sent by a plurality of devices.

31. The device as recited in claim 27, wherein the second port is configured to receive one or more frames destined for one or more devices other than the first device.

32. The device as recited in claim 27, wherein the at least one buffer comprises a first buffer and a second buffer;

wherein, in said adding the plurality of information units to the at least one buffer, the distribution logic is further configured to;

add the one or more transfer ready information units to the first buffer; and

add the one or more non-transfer-ready information units to the second buffer;

wherein, in said conveying the queued information units from the at least one buffer to the first device, the scheduler logic is further configured to;

convey the one or more transfer ready information units from the first buffer to the first device; and

convey the one or more non-transfer-ready information units from the second buffer to the first device after said conveying the one or more transfer ready information units from the first buffer to the first device.

33. The device as recited in claim 27, wherein the at least one buffer comprises a first buffer, wherein, in said adding the plurality of information units to the at least one buffer, the distribution logic is further configured to add the plurality of information units to the first buffer, wherein the one or more transfer ready information units are added in front of the non-transfer-ready information units in the first buffer;

wherein, in said conveying the queued information units from the at least one buffer to the first device, the scheduler logic is further configured to;

convey the one or more transfer ready information units from the first buffer to the first device; and

convey the one or more non-transfer-ready information units from the first buffer to the first device after said conveying the one or more transfer ready information units from the first buffer to the first device.

34. The device as recited in claim 27, wherein the information units are Fibre Channel Protocol for SCSI (FCP) frame sequences.

35. The device as recited in claim 27, wherein the information units are iSCSI (SCSI protocol over TCP/IP) frame sequences.

36. The device as recited in claim 27, wherein the information units are FCIP (Fibre Channel over Internet Protocol) frame sequences.

37. The device as recited in claim 27, wherein the information units are iFCP (Internet Fibre Channel Protocol) frame sequences.

38. The device as recited in claim 27, wherein the first device is a Fibre Channel host bus adapter.

39. The device as recited in claim 27, wherein the first device is an iSCSI (SCSI protocol over TCP/IP) host bus adapter.

40. The device as recited in claim 27, wherein the second device is a storage device.

41. The device as recited in claim 27, wherein the device is a network switch.

42. A network switch comprising:

a plurality of ports, wherein the plurality of ports comprises;

a first port operable to couple to a first device; and

a second port operable to couple to a second device and configured to receive a plurality of information units each comprising one or more frames sent from the second device and destined for the first device, wherein the plurality of information units includes one or more transfer ready information units and one or more non-transfer-ready information units, and wherein at least one of the one or more non-transfer-ready information units is received by the device before the one or more transfer ready information units;

a memory comprising at least one buffer operable to queue information units in transit between the first and second ports of the device;

logic means coupled between the first port and the second port and configured to;

add the plurality of information units to the at least one buffer; and

convey the queued plurality of information units from the at least one buffer through the first port to the first device, wherein the one or more transfer ready information units are conveyed to the first device prior to conveying the one or more non-transfer-ready information units.

43. The network switch as recited in claim 42, wherein the at least one buffer comprises a first buffer and a second buffer;

wherein, in said adding the plurality of information units to the at least one buffer, the logic means is further configured to;

add the one or more transfer ready information units to the first buffer; and

add the one or more non-transfer-ready information units to the second buffer.

44. The network switch as recited in claim 42, wherein the at least one buffer comprises a first buffer, wherein, in said adding the plurality of information units to the at least one buffer, the logic means is further configured to add the plurality of information units to the first buffer, wherein the one or more transfer ready information units are added in front of the non-transfer-ready information units in the first buffer.

45. The network switch as recited in claim 42, wherein the information units are Fibre Channel Protocol for SCSI (FCP) frame sequences, wherein the first device is a Fibre Channel host bus adapter, and wherein the second device is a storage device.

46. A system comprising:

a first device;

a second device; and

a third device operable to couple to the first device and to the second device, wherein information units in transit between the first and second devices are conveyed through the third device, and wherein the third device comprises at least one buffer operable to queue information units in transit between the first and second devices;

wherein the first device is configured to;

send one or more data read commands to the second device; and

send one or more data write commands to the second device;

wherein the second device is configured to;

send one or more read data information units to the first device in response to the one or more data read commands, wherein the one or more read data information units each comprise a portion of read data requested in the one or more data read commands; and

send one or more transfer ready information units to the first device in response to the one or more data write commands, wherein each transfer ready information unit indicates that the second device is ready to receive write data from the first device;

wherein the third device is configured to;

receive the one or more read data information units and the one or more transfer ready information units destined for the first device, wherein at least one of the one or more read data information units is received before the one or more transfer ready information units;

add the one or more read data information units and the one or more transfer ready information units to the at least one buffer; and

convey the one or more read data information units and the one or more transfer ready information units from the at least one buffer to the first device;

wherein, in said conveying, the one or more transfer ready information units are conveyed to the first device prior to the one or more read data information units.

47. The system as recited in claim 46, wherein the at least one buffer comprises a first buffer and a second buffer;

wherein, in said adding the one or more read data information units and the one or more transfer ready information units to the at least one buffer, the third device is further configured to;

add the one or more transfer ready information units to the first buffer; and

add the one or more read data information units to the second buffer;

wherein, in said conveying the one or more read data information units and the one or more transfer ready information units from the at least one buffer to the first device, the third device is further configured to;

convey the one or more transfer ready information units from the first buffer to the first device; and

convey the one or more read data information units from the second buffer to the first device after said conveying the one or more transfer ready information units from the first buffer to the first device.

48. The system as recited in claim 46, wherein the at least one buffer comprises a first buffer;

wherein, in said adding the one or more read data information units and the one or more transfer ready information units to the at least one buffer, the third device is further configured to add the one or more read data information units and the one or more transfer ready information units to the first buffer, wherein the one or more transfer ready information units are added in front of the one or more read data information units in the first buffer;

wherein, in said conveying the one or more read data information units and the one or more transfer ready information units from the at least one buffer to the first device, the third device is further configured to;

convey the one or more transfer ready information units from the first buffer to the first device; and

convey the one or more read data information units from the first buffer to the first device after said conveying the one or more transfer ready information units from the first buffer to the first device.

49. The system as recited in claim 46, wherein the one or more read data information units and the one or more transfer ready information units are Fibre Channel Protocol for SCSI (FCP) frame sequences.

50. The system as recited in claim 46, wherein the information units are iSCSI (SCSI protocol over TCP/IP) frame sequences.

51. The system as recited in claim 46, wherein the information units are FCIP (Fibre Channel over Internet Protocol) frame sequences.

52. The system as recited in claim 46, wherein the information units are iFCP (Internet Fibre Channel Protocol) frame sequences.

53. The system as recited in claim 46, wherein the first device is a Fibre Channel host bus adapter and the second device is a storage device.

54. The system as recited in claim 46, wherein the third device is a network switch.

55. The system as recited in claim 46, wherein the first device is an iSCSI (SCSI protocol over TCP/IP) host bus adapter.

56. The system as recited in claim 46, wherein the third device is further configured to perform said receiving, said adding, and said conveying in a pipelined fashion.

57. A storage area network comprising:

a device comprising at least one buffer operable to queue information units in transit on the storage area network;

a plurality of storage devices coupled to the device; and

a host bus adapter coupled to the device;

wherein the device is configured to;

receive from the plurality of storage devices a plurality of information units destined for the host bus adapter, wherein the plurality of information units includes one or more transfer ready information units and one or more non-transfer-ready information units, and wherein at least one of the one or more non-transfer-ready information units is received by the device before the one or more transfer ready information units;

add the plurality of information units to the at least one buffer; and

convey the queued plurality of information units from the at least one buffer to the host bus adapter;

wherein, in said conveying, the device is further configured to convey the one or more transfer ready information units to the host bus adapter prior to conveying the one or more non-transfer-ready information units to the host bus adapter.

58. The storage area network as recited in claim 57,

wherein each of the one or more transfer ready information units is sent by one of the plurality of storage devices to the host bus adapter in response to a data write command previously sent from the host bus adapter to the particular storage device, and wherein each of the one or more transfer ready information units indicates that the particular storage device is ready to receive write data from the host bus adapter;

and

wherein at least one of the one or more non-transfer-ready information units comprises different portions of read data requested by one or more data read commands previously sent from the host bus adapter to one or more of the plurality of storage devices.

59. The storage area network as recited in claim 57, wherein the at least one buffer comprises a first buffer and a second buffer;

wherein, in said adding the plurality of information units to the at least one buffer, the device is further configured to;

add the one or more transfer ready information units to the first buffer; and

add the one or more non-transfer-ready information units to the second buffer;

wherein, in said conveying the queued information units from the at least one buffer to the host bus adapter, the device is further configured to;

convey the one or more transfer ready information units from the first buffer to the host bus adapter; and

convey the one or more non-transfer-ready information units from the second buffer to the host bus adapter after said conveying the one or more transfer ready information units from the first buffer to the host bus adapter.

60. The storage area network as recited in claim 57, wherein the at least one buffer comprises a first buffer;

wherein, in said adding the plurality of information units to the at least one buffer, the device is further configured to add the plurality of information units to a first buffer, wherein the one or more transfer ready information units are added in front of the non-transfer-ready information units in the first buffer;

wherein, in said conveying the queued information units from the at least one buffer to the host bus adapter, the device is further configured to;

convey the one or more transfer ready information units from the first buffer to the host bus adapter; and

convey the one or more non-transfer-ready information units from the first buffer to the host bus adapter after said conveying the one or more transfer ready information units from the first buffer to the host bus adapter.

61. The storage area network as recited in claim 57, wherein the information units are Fibre Channel Protocol for SCSI (FCP) frame sequences.

62. The storage area network as recited in claim 57, wherein the information units are iSCSI (SCSI protocol over TCP/IP) frame sequences.

63. The storage area network as recited in claim 57, wherein the information units are FCIP (Fibre Channel over Internet Protocol) frame sequences.

64. The storage area network as recited in claim 57, wherein the information units are iFCP (Internet Fibre Channel Protocol) frame sequences.

65. The storage area network as recited in claim 57, wherein the host bus adapter is a Fibre Channel host bus adapter.

66. The storage area network as recited in claim 57, wherein the device is a network switch.

67. A carrier medium comprising program instructions executable within a network switch, wherein the program instructions are executable to implement:

receiving a plurality of information units each comprising one or more frames destined for a first device coupled to the network switch, wherein the plurality of information units includes one or more transfer ready information units and one or more non-transfer-ready information units, and wherein at least one of the one or more non-transfer-ready information units is received before the one or more transfer ready information units;

adding the plurality of information units to at least one buffer on the network switch, wherein the at least one buffer is configured to queue information units destined for the first device; and

conveying the queued plurality of information units from the at least one buffer to the first device;

wherein, in said conveying, the one or more transfer ready information units are conveyed to the first device prior to the one or more non-transfer-ready information units.

68. The carrier medium as recited in claim 67, wherein the at least one buffer comprises a first buffer and a second buffer, and wherein, in said adding the plurality of information units to the at least one buffer, the program instructions are further executable to implement:

adding the one or more transfer ready information units to the first buffer; and

adding the one or more non-transfer-ready information units to the second buffer.

69. The carrier medium as recited in claim 67, wherein the at least one buffer comprises a first buffer, and wherein, in said adding the plurality of information units to the at least one buffer, the program instructions are further executable to implement adding the plurality of information units to the first buffer, wherein the one or more transfer ready information units are added in front of the non-transfer-ready information units in the first buffer.

70. The carrier medium as recited in claim 67, wherein the first device is a Fibre Channel host bus adapter, wherein the second device is a storage device, and wherein the information units are Fibre Channel Protocol for SCSI (FCP) frame sequences.