Disaggregated hybrid optical transport network, internet protocol, and Ethernet switching system

• US 10,461,873 B1
• Filed: 09/17/2018
• Issued: 10/29/2019
• Est. Priority Date: 09/17/2018
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Cai, Biaodong, Dunsmore, Richard
• Primary Examiner(s): Alagheband, Abbas H
• Application Number:
  US16/133,088
• Time in Patent Office: 407 Days
• Field of Search: 398 45
• CPC Class Codes: H04B 10/00 Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
• CPC Class Codes: H04J 14/0212 using optical switches or wavelength selective switches [WSS]
• CPC Class Codes: H04J 2203/0012 Switching modules and their interconnections
• CPC Class Codes: H04J 2203/0025 Peripheral units
• CPC Class Codes: H04J 2203/0083 Support of the IP protocol
• CPC Class Codes: H04J 2203/0085 Support of Ethernet
• CPC Class Codes: H04J 3/1658 carrying packets or ATM cells; H04J3/1664 takes precedence for payloads with different packet types
• CPC Class Codes: H04L 12/64 Hybrid switching systems
• CPC Class Codes: H04L 45/24 Multipath
• CPC Class Codes: H04L 45/745 Address table lookup; Address filtering
• CPC Class Codes: H04L 47/12 Avoiding congestion; Recovering from congestion
• CPC Class Codes: H04L 47/6225 Fixed service order, e.g. Round Robin
• CPC Class Codes: H04L 47/623 Weighted service order
• CPC Class Codes: H04L 47/6275 based on priority
• CPC Class Codes: H04L 49/1523 Parallel switch fabric planes
• CPC Class Codes: H04L 49/1546 using pipelined operation
• CPC Class Codes: H04L 49/25 Routing or path finding in a switch fabric
• CPC Class Codes: H04L 49/253 using establishment or release of connections between ports
• CPC Class Codes: H04L 49/3045 Virtual queuing
• CPC Class Codes: H04L 49/351 for local area network [LAN], e.g. Ethernet switches
• CPC Class Codes: H04Q 2011/0016 : using wavelength multiplexing or demultiplexing
• CPC Class Codes: H04Q 2011/0086 : Network resource allocation, dimensioning or optimisation

Abstract

Systems and Methods for switching optical data units (ODUs) and Internet Protocol (IP) packets as Ethernet packets in an optical transport network (OTN), IP, and Ethernet switching system. The OTN, IP, and Ethernet switching system may include an Ethernet fabric having a set of M Ethernet switches each including a set of N switch ports, and a set of N input/output (IO) devices each including a set of W IO ports, a set of M Ethernet ports, an IO side packet processor (IOSP), and a fabric side packet processor (FSP). Each Ethernet switch may establish switch queues. Each IO device may establish a set of M hierarchical virtual output queues each including a set of N ingress-IOSP queues and ingress-virtual output queues, a set of W egress-IOSP queues, a set of M ingress-FSP queues, and a set of N hierarchical virtual input queues each including a set of N egress-FSP queues and egress-virtual input queues.

Claims

1. An optical transport network (OTN), Internet Protocol (IP), and Ethernet switching system comprising:

an Ethernet fabric including a set of M Ethernet switches each comprising a set of N switch ports, wherein a variable i having a value ranging from 1 to M to denote the ith Ethernet switch of the set of M Ethernet switches and a variable j having a value ranging from 1 to N to denote the jth switch port of the set of N switch ports;

a set of O input/output (IO) devices each comprising;

a set of M Ethernet ports, wherein a variable u having a value ranging from 1 to O to denote the uth IO device of the set of O IO devices, and wherein the jth Ethernet port of the uth IO device is connected to the uth switch port of the ith Ethernet switch;

an IO side packet processor (IOSP) configured to;

establish a set of M hierarchical virtual output queues (H-VOQs) each comprising a set of N ingress-IOSP queues (I-IOSPQs) and I-VOQs;

create M virtual lanes (v-lanes) including a first v-lane and a second v-lane, each of the M v-lanes corresponds to a respective H-VOQ of the set of M H-VOQs;

create A equal cost multi-path (ECMP) pipes including B ECMP pipes and C ECMP pipes, each of the A ECMP pipes connects to one of the M v-lanes, each of the B ECMP pipes connects to the first v-lane, and each of the C ECMP pipes connects to the second v-lane;

generate micro-flows by 5-Tuple look-up based on packet header information of a received IP packet and an I-IOSP forwarding information base (FIB);

distribute the micro-flows into the A ECMP pipes; and

queue the IP packet including first metadata to an I-IOSPQ of an H-VOQ corresponding to an egress IO device and a switch number of a corresponding Ethernet switch based on the micro-flows and an identified micro-flow for an ECMP hash key in a ECMP pipe hash of the IOSP.

2. The OTN, IP, and Ethernet switching system of claim 1, wherein the number O of the set of O IO devices is equal to the number N of the set of N Ethernet ports, wherein the number B of the B ECMP pipes is greater than the number C of the C ECMP pipes and packet traffic bandwidth of the first v-lane is greater than the packet traffic bandwidth of the second v-lane.

3. The OTN, IP, and Ethernet switching system of claim 2, wherein the M v-lanes further includes a third v-lane, wherein the A ECMP pipes further includes D ECMP pipes, each of the D ECMP pipes corresponds to the third v-lane, and wherein the number C of the C ECMP pipes is greater than the number D of the D ECMP pipes, and the packet traffic bandwidth of the second v-lane is greater than the packet traffic bandwidth of the third v-lane.

4. The OTN, IP, and Ethernet switching system of claim 1, wherein the number O of the set of O IO devices is equal to the number N of the set of N Ethernet ports, wherein the M v-lanes further includes a third v-lane, wherein the A ECMP pipes further includes D ECMP pipes, each of the D ECMP pipes corresponds to the third v-lane, and wherein the number B of the B ECMP pipes is equal to the number C of the C ECMP pipes, the number C of the C ECMP pipes is equal to the number D of the C ECMP pipes, packet traffic bandwidth of the first v-lane is equal to the packet traffic bandwidth of the second v-lane, and packet traffic bandwidth of the second v-lane is equal to the packet traffic bandwidth of the third v-lane.

5. The OTN, IP, and Ethernet switching system of claim 1, further comprising:

a set of Q optical transport network leaf (O-leaf) plug-in universal (PIU) modules each comprising a set of L Ethernet ports, wherein a variable v having a value ranging from 1 to Q to denote the vth O-leaf PIU module, a variable z having a value ranging from 1 to L to denote the zth Ethernet port, and a variable g having a value ranging from 1+H to M to denote the gth Ethernet switch, and wherein the zth Ethernet port of the set of L Ethernet ports of the vth O-leaf PIU module is connected to the O+vth switch port of the set of N switch ports of the gth Ethernet switch.

6. The OTN, IP, and Ethernet switching system of claim 1, wherein the set of Q O-leaf PIU modules includes a first O-leaf PIU module and a second O-leaf PIU module, wherein the first O-leaf PIU module configured to:

establish a first optical data unit (ODU) switched connection from the first O-leaf PIU module to the second O-leaf PIU module via the subset of L Ethernet switches of the M Ethernet switches;

select a first sequential order of the subset of L Ethernet switches;

receive a first ODU at the first O-leaf PIU module and generate a second Ethernet packet corresponding to the first ODU, wherein the first ODU is for transmission via the first ODU switched connection; and

transmit the second Ethernet packet from a first Ethernet port of the set of L Ethernet ports of the first O-leaf PIU module, wherein the first Ethernet port is selected based on the first sequential order.

7. The OTN, IP, and Ethernet switching system of claim 1, wherein each Ethernet switch of the set of M Ethernet switches to establish switch queues;

wherein the first Ethernet switch of the set of M Ethernet switches configured to;

identify an egress port of the set of N switch ports of the first Ethernet switch based on packet header information of the received first Ethernet packet including an egress port number of first metadata, the first metadata, and a first MAC header from an ingress IO device of the set of O IO devices;

generate a second MAC header based on the egress port number and an egress IO device of the set of O IO devices of the first metadata;

generate second metadata from the first metadata by removing the egress IO device;

queue the packet data, the second metadata, and the second MAC header to switch queues of the first Ethernet switch;

de-queue the packet data, the second metadata, and the second MAC header from the switch queues using a scheduling algorithm; and

transmit a second Ethernet packet including the de-queued packet data, the second metadata, and the second MAC header to the egress IO device via the egress port of the first Ethernet switch.

8. The OTN, IP, and Ethernet switching system of claim 1, each IO device further comprising:

a set of W IO ports, wherein a variable x having a value ranging from 1 to W to denote the xth IO port of the set of W IO ports, wherein

the IOSP further configured to;

establish a set of W egress-IOSP queues (E-IOSPQs), wherein the xth E-IOSPQ corresponds to an xth IO port of the set of W IO ports of the IO device;

de-queue the IP packet including the first metadata from the I-VOQs of the H-VOQ using a scheduling algorithm; and

transmit the de-queued IP packet including the first metadata to the FSP of the ingress IO device, wherein

the FSP further configured to;

establish a set of M ingress-FSP queues (I-FSPQs), wherein the ith I-FSPQ corresponds to the ith Ethernet switch;

generate a first Ethernet packet including the packet data of the IP packet, second metadata based on the first metadata, and a first media access control (MAC) header;

queue the first Ethernet packet to an I-FSPQ corresponding to the switch number of the first metadata;

de-queue the first Ethernet packet including the packet data of the IP packet, the second metadata, and the first MAC header from the I-FSPQ using the scheduling algorithm; and

transmit the de-queued first Ethernet packet to the egress IO device via an Ethernet switch corresponding to the switch number.

9. The OTN, IP, and Ethernet switching system of claim 1, wherein an FSP of an egress IO device of the set of O IO devices configured to:

establish a set of O hierarchical virtual input queues (H-VIQs) each comprising a set of O egress-FSP queues (E-FSPQs) and E-VIQs, wherein the uth H-VIQ corresponds to the uth IO device, and wherein the uth E-FSPQ of the uth H-VIQ corresponds to the uth IO device;

receive a first Ethernet packet including the packet data of the IP packet, second metadata, and a first MAC header at an Ethernet port of the egress IO device;

determine an ingress IO device of the set of O IO devices based on an E-FSP FIB of the FSP and an internal flow ID of the second metadata of the first Ethernet packet;

queue the packet data and the second metadata to an E-FSPQ of an H-VIQ corresponding respectively to the ingress IO device and the IO port of the egress IO device;

de-queue the packet data and the second metadata from the E-VIQs of the H-VIQ using a scheduling algorithm; and

transmit the de-queued packet data and the second metadata to an IOSP of the egress IO device, wherein

the IOSP of the egress IO device configured to;

generate the IP packet including the received packet data and packet header information of the second metadata from the received packet data and the second metadata;

queue the IP packet to an E-IOSPQ of the set of W E-IOSPQs corresponding to the egress port of the egress IO device of the second metadata; and

de-queue the IP packet data from the E-IOSPQ using the scheduling algorithm; and

transmit the IP packet via the egress port.

10. The OTN, IP, and Ethernet switching system of claim 9, wherein

each I-IOSPQ of the set of O I-IOSPQs of each H-VOQ of the set M H-VOQs of an IOSP of each IO device of the set of O IO devices comprising a set of P priority I-IOSPQs, wherein packet data in each I-IOSPQ is de-queued using the scheduling algorithm, wherein

the I-VOQs of each H-VOQ of the set M H-VOQs of the IOSP of each IO device of the set of O IO devices comprising a set of P priority I-VOQs, wherein packet data in the I-VOQs is de-queued using the scheduling algorithm, wherein

each I-FSPQ of the set of M I-FSPQs of the FSP of each IO device of the set of O IO devices comprising a set of P priority I-FSPQs, wherein packet data in each I-FSPQ is de-queued using the scheduling algorithm, wherein

each E-FSPQ of the set of O E-FSPQs of each H-VIQs of the set O H-VIQs of an FSP of each IO device of the set of O IO devices comprising a set of P Priority E-FSPQs, wherein packet data in each E-FSPQ is de-queued using the scheduling algorithm, wherein

the E-VIQs of each H-VIQ of the set N H-VIQs of the FSP of each IO device of the set of O IO devices comprising a set of P priority E-VIQs, wherein packet data in the E-VIQs is de-queued using the scheduling algorithm, and wherein the scheduling algorithm comprises a strict priority algorithm, a weighted fair queuing algorithm, a weighted round robin algorithm, a strict priority and weighted fair queuing algorithm, or a strict priority weighted round robin algorithm.

11. The OTN, IP, and Ethernet switching system of claim 1, wherein an IOSP of each IO device of the set of O IO devices configured to:

establish quantize congestion notification (QCN) between each I-IOSPQ of each H-VOQ of the set of M H-VOQs of the IOSP of each IO device of the set of O IO devices and each corresponding E-FSPQ of each H-VIQ of a set of O H-VIQs of an FSP of each IO device of the set of O IO devices, and wherein

packet data in each I-IOSPQ of the set of O I-IOSPQs of each H-VOQ of the set M H-VOQs of the IOSP of each IO device of the set of O IO devices is de-queued using a scheduling algorithm based on the established QCN.

12. The OTN, IP, and Ethernet switching system of claim 1, wherein an IOSP of each IO device of the set of O IO devices configured to:

establish priority-based flow control (PFC) between each I-FSPQ of a subset of R I-FSPQs of the M I-FSPQs of the FSP of each IO device of the set of O IO devices and switch queues of a corresponding subset of R Ethernet switches of the set of M Ethernet switches, wherein a variable y having a value ranging from 2 to N−

1 to denote the yth I-FSPQ of the subset of R I-FSPQs, and wherein

packet data in each I-FSPQ of the subset of R I-FSPQs of the M I-FSPQs of the FSP of each IO device of the set of O IO devices is de-queued using a scheduling algorithm based on the established PFC.

13. The OTN, IP, and Ethernet switching system of claim 1, further comprising:

a virtual switch fabric including a set of N virtual line card slots each comprises a logical aggregation of the jth switch port of the set of N switch ports of each of the set of M Ethernet switches, wherein the uth IO device of the set of O IO devices associated with only the uth virtual line card slot of the N virtual line card slots, and wherein the with O-leaf PIU module of the set of Q O-leaf PIU module associated with only the O+vth virtual line card slot of the N virtual line card slots.

14. The OTN, IP, and Ethernet switching system of claim 1, an FSP of an egress IO device of the set of O IO devices configured to:

establish a set of O E-VIQs, wherein the uth E-VIQ corresponds to the uth IO device;

receive packet data and second metadata at an Ethernet port of the egress IO device;

determine an ingress IO device of the set of O IO devices based on an E-FSP FIB of the FSP and an internal flow ID of the second metadata;

queue the packet data and the second metadata to an E-VIQs corresponding respectively to the IO port of the egress IO device;

de-queue the packet data and the second metadata from the E-VIQs using a scheduling algorithm; and

transmit the de-queued packet data and the second metadata to an IOSP of the egress IO device, wherein

the IOSP of the egress device configured to;

queue the packet data and packet header information of the second metadata to an E-IOSPQ of a set of W E-IOSPQs corresponding to the egress port of the egress IO device of the second metadata; and

de-queue the packet data and the packet header information from the E-IOSPQ using the scheduling algorithm; and

transmit the packet data and the packet header information via the egress port.

15. The OTN, IP, and Ethernet switching system of claim 1, wherein an IOSP of each IO device of the set of O IO devices configured to:

establish quantize congestion notification (QCN) between each I-IOSPQ of each H-VOQ of the set of M H-VOQs of the IOSP of each IO device of the set of O IO devices and each E-VIQ of a set of O E-VIQs of an FSP of each IO device of the set of O IO devices, and wherein

packet data in each I-IOSPQ of the set of O I-IOSPQs of each H-VOQ of the set M H-VOQs of the IOSP of each IO device of the set of O IO devices is de-queued using a scheduling algorithm based on the established QCN.

16. The OTN, IP, and Ethernet switching system of claim 1, wherein an IOSP of each IO device of the set of O IO devices configured to:

establish priority-based flow control (PFC) between each I-FSPQ of a subset of R I-FSPQs of the M I-FSPQs of the FSP of each IO device of the set of O IO devices and switch queues of a corresponding subset of R Ethernet switches of the set of M Ethernet switches, wherein a variable y having a value ranging from 2 to O−

1 to denote the yth I-FSPQ of the subset of R I-FSPQs, and wherein

packet data in each I-FSPQ of the subset of R I-FSPQs of the M I-FSPQs of the FSP of each IO device of the set of O IO devices is de-queued using a scheduling algorithm based on the established PFC.

17. The OTN, IP, and Ethernet switching system of claim 1, an FSP of an egress IO device of the set of O IO devices configured to:

establish a set of W E-VIQs, wherein a variable x having a value ranging from 1 to W to denote the xth E-VIQs of the set of W E-VIQ;

receive packet data and second metadata at an Ethernet port of the egress IO device;

determine an ingress IO device of the set of O IO devices based on an E-FSP FIB of the FSP and an internal flow ID of the second metadata;

queue the packet data and the second metadata to an E-VIQs corresponding respectively to an IO port of the egress IO device;

de-queue the packet data and the second metadata from the E-VIQs using a scheduling algorithm; and

transmit the de-queued packet data and the second metadata to an IOSP of the egress IO device, wherein

the IOSP configured to;

queue the packet data and packet header information of the second metadata to an E-IOSPQ of a set of W E-IOSPQs corresponding to the egress port of the egress IO device of the second metadata; and

de-queue the packet data and the packet header information from the E-IOSPQ using the scheduling algorithm; and

transmit the packet data and the packet header information via the egress port.

18. The OTN, IP, and Ethernet switching system of claim 1, wherein an IOSP of each IO device of the set of O IO devices configured to:

establish priority-based flow control (PFC) between each I-FSPQ of M I-FSPQs of the FSP of each IO device of the set of O IO devices and switch queues of a corresponding set of M Ethernet switches, wherein

packet data in each I-FSPQ of the M I-FSPQs of the FSP of each IO device of the set of O IO devices is de-queued using a scheduling algorithm based on the established PFC.

20. The method of claim 15, wherein the method further comprising:

in the OTN, IP, and Ethernet switching system further comprising;

a set of Q optical transport network leaf (O-leaf) plug-in universal (PIU) modules including a first O-leaf PIU module and a second O-leaf PIU module, each O-leaf PIU module of the set of Q O-leaf PIU modules comprising a set of L Ethernet ports, wherein a variable v having a value ranging from 1 to Q to denote the vth O-leaf PIU module, a variable z having a value ranging from 1 to L to denote the zth Ethernet port, and a variable g having a value ranging from 1+H to M to denote the gth Ethernet switch, and wherein the zth Ethernet port of the set of L Ethernet ports of the vth O-leaf PIU module is connected to the O+vth switch port of the set of N switch ports of the gth Ethernet switch,

establishing, by the first O-leaf PIU module, a first optical data unit (ODU) switched connection from the first O-leaf PIU module to the second O-leaf PIU module via the subset of L Ethernet switches of the M Ethernet switches;

selecting, by the first O-leaf PIU module, a first sequential order of the subset of L Ethernet switches;

receiving, by the first O-leaf PIU module, a first ODU at the first O-leaf PIU module and generate a second Ethernet packet corresponding to the first ODU, wherein the first ODU is for transmission via the first ODU switched connection; and

transmitting, by the first O-leaf PIU module, the second Ethernet packet from a first Ethernet port of the set of L Ethernet ports of the first O-leaf PIU module, wherein the first Ethernet port is selected based on the first sequential order.

19. A method, the method comprising:

in an optical transport network (OTN), Internet Protocol (IP), and Ethernet switching system comprising;

an Ethernet fabric including a set of M Ethernet switches each comprising a set of N switch ports, wherein a variable i having a value ranging from 1 to M to denote the ith Ethernet switch of the set of M Ethernet switches and a variable j having a value ranging from 1 to N to denote the jth switch port of the set of N switch ports;

a set of O input/output (IO) devices each comprising;

a set of M Ethernet ports, wherein a variable u having a value ranging from 1 to O to denote the uth IO device of the set of O IO devices, and wherein the jth Ethernet port of the uth IO device is connected to the uth switch port of the ith Ethernet switch; and

an IO side packet processor (IOSP),

establishing, by the IOSP, a set of M hierarchical virtual output queues (H-VOQs) each comprising a set of N ingress-IOSP queues (I-IOSPQs) and I-VOQs;

creating, by the IOSP, M virtual lanes (v-lanes) corresponding to a respective H-VOQ of the set of M H-VOQs, the M v-lanes including a first v-lane and a second v-lane;

creating, by the IOSP, A equal cost multi-path (ECMP) pipes including B ECMP pipes and C ECMP pipes, each of the A ECMP pipes connects to one of the M v-lanes, each of the B ECMP pipes connects to the first v-lane, and each of the C ECMP pipes connects to the second v-lane;

generating, by the IOSP, micro-flows by 5-Tuple look-up based on packet header information of a received IP packet and an I-IOSP forwarding information base (FIB);

distributing, by the IOSP, the micro-flows into the A ECMP pipes; and

queueing, by the IOSP, the IP packet including first metadata to an I-IOSPQ of an H-VOQ corresponding to an egress IO device and a switch number of a corresponding Ethernet switch based on the micro-flows and an identified micro-flow for an ECMP hash key in a ECMP pipe hash of the IOSP.