Multi-stage switch fabric fault detection and handling

• US 9,866,427 B2
• Filed: 02/16/2015
• Issued: 01/09/2018
• Est. Priority Date: 02/16/2015
• Status: Active Grant

• Current Assignee: Juniper Networks Incorporated
• Original Assignee: Juniper Networks Incorporated
• Inventors: Yadav, Alam, Golash, Mukul
• Primary Examiner(s): Acolatse, Kodzovi
• Application Number:
  US14/623,083
• Publication Number:
  US 20160241430A1
• Time in Patent Office: 1,058 Days
• Field of Search: None
• CPC Class Codes: H04L 41/0654 using network fault recovery ring fault isolation or reconfiguration in loop networks without recovery actions by a network management system H04L12/437
• CPC Class Codes: H04L 41/08 Configuration management of networks or network elements address allocation H04L61/50
• CPC Class Codes: H04L 43/0811 by checking connectivity
• CPC Class Codes: H04L 45/28 using route fault recovery
• CPC Class Codes: H04L 49/00 Packet switching elements
• CPC Class Codes: H04L 49/101 using crossbar or matrix
• CPC Class Codes: H04L 49/1523 Parallel switch fabric planes
• CPC Class Codes: H04L 49/40 Constructional details, e.g. power supply, mechanical construction or backplane
• CPC Class Codes: H04L 49/555 Error detection
• CPC Class Codes: H04L 49/557 Error correction, e.g. fault recovery or fault tolerance
• CPC Class Codes: H04Q 11/04 for time-division multiplexing H04Q11/0001 takes precedence

Abstract

In some examples, a switching system includes a plurality of fabric endpoints and a multi-stage switching fabric having a plurality of fabric planes each having a plurality of stages to switch data units between any of the plurality of fabric endpoints. A fabric endpoint of the fabric endpoints is configured to send, to a switch of a first one of the stages and within a first fabric plane of the plurality of fabric planes, a self-ping message destined for the fabric endpoint. The fabric endpoint is configured to send, in response to determining the fabric endpoint has not received the self-ping message after a predetermined time, an indication of a connectivity fault for the first fabric plane.

Claims

1. A method of verifying fabric connectivity with a multi-stage switch fabric having a plurality of fabric planes each having a plurality of stages to switch data units between any of a plurality of fabric endpoints, the method comprising:

sending, by a fabric endpoint of the plurality of fabric endpoints to a first-stage switch of a first fabric plane of the plurality of fabric planes, a self-ping message destined for the fabric endpoint sending the self-ping message; and

sending, by the fabric endpoint to a first-stage switch of a second fabric plane of the plurality of fabric planes and in response to determining the fabric endpoint has not received the self-ping message after a predetermined time, a fault data unit that includes an indication of a connectivity fault for the first fabric plane, the first fabric plane different than the second fabric plane.

2. The method of claim 1, further comprising:

switching, by an intermediate-stage switch of the multi-stage switch fabric, the self-ping message to a final-stage switch of the first fabric plane, the final-stage switch coupled by an output port to the fabric endpoint;

switching, by the final-stage switch to the fabric endpoint, the self-ping message; and

determining, by the fabric endpoint in response to receiving the self-ping message within the predetermined time, the first fabric plane does not have a connectivity fault.

3. The method of claim 1, wherein the plurality of fabric endpoints comprises a plurality of forwarding components interconnected within a network router by the multi-stage switch fabric.

4. The method of claim 1,

wherein the plurality of fabric endpoints comprises a plurality of edge devices interconnected within a data center by the multi-stage switch fabric.

5. The method of claim 1, wherein the fabric endpoint comprises a first fabric endpoint, and wherein the fault data unit comprises a first fault data unit, the method further comprising:

receiving, by the first-stage switch of the second fabric plane of the multi-stage switch fabric and from the first fabric endpoint, the first fault data unit that includes the indication of the connectivity fault;

receiving, by the first-stage switch of the second fabric plane and from a second fabric endpoint of the plurality of fabric endpoints, a second fault data unit that includes the indication of the connectivity fault; and

sending, by the first-stage switch of the second fabric plane to a final-stage switch of the second fabric plane and after consolidating the first fault data unit and the second fault data unit, a fault notification data unit that includes the indication of the connectivity fault.

6. The method of claim 1, wherein the indication of the connectivity fault identifies the first fabric plane.

7. The method of claim 1, wherein the fault data unit is destined for a broadcast address that specifies each of the plurality of fabric endpoints as an intended recipient of the fault data unit.

8. A method of signaling a fabric fault for a multi-stage switch fabric having a plurality of fabric planes each having a plurality of stages to switch data units between any of a plurality of fabric endpoints, the method comprising:

receiving, by a first-stage switch of a first fabric plane of the plurality of fabric planes, an indication of a fabric fault for a second fabric plane of the plurality of fabric planes, the first fabric plane different than the second fabric plane;

sending, by the first-stage switch to a first final-stage switch of the first fabric plane, a first fault notification data unit that includes the indication of the fabric fault; and

sending, by the first-stage switch to a second final-stage switch of the first fabric plane, a second fault notification data unit that includes the indication of the fabric fault.

9. The method of claim 8,

wherein the indication comprises a fault data unit sent by a fabric endpoint of the plurality of fabric endpoints and the fabric fault is a connectivity fault for the second fabric plane,

wherein sending the first fault notification data unit comprises sending, by the first-stage switch in response to determining the fault data unit includes a broadcast address that specifies each of the plurality of fabric endpoints as an intended recipient of the fault data unit, the first fault notification data unit, and

wherein sending the second fault notification data unit comprises sending, by the first-stage switch in response to determining the fault data unit includes the broadcast address, the second fault notification data unit.

10. The method of claim 8, further comprising:

sending, by the first final-stage switch to at least one fabric endpoint of the plurality of fabric endpoints that is coupled to the first final-stage switch and in response to receiving the first fault notification data unit, a third fault notification data unit that includes the indication of the fabric fault.

11. The method of claim 10, further comprising:

modifying, by the at least one fabric endpoint in response to receiving the third fault notification data unit, a configuration of the at least one fabric endpoint to cause the at least one fabric endpoint to at least temporarily cease sending data units via the second fabric plane having the fabric fault.

12. The method of claim 8,

wherein the first fault notification data unit includes a first destination identifier that identifies the first final-stage switch, and

wherein the second fault notification data unit includes a second destination identifier that identifies the second final-stage switch.

13. The method of claim 8, further comprising:

receiving, by an intermediate-stage switch of the first fabric plane, the first fault notification data unit; and

sending, by the intermediate-stage switch in response to determining the first fault notification data unit includes a first destination identifier that identifies the first final-stage switch, the first fault notification data unit to the first final-stage switch.

14. The method of claim 8, wherein the fabric fault comprises a fabric link fault for fabric link connecting one of a first-stage switch with a second stage switch, a second stage switch with a third stage switch, and a third stage switch with a fabric endpoint of the plurality of fabric endpoints.

15. A switching system comprising:

a plurality of fabric endpoints;

a multi-stage switching fabric having a plurality of fabric planes each having a plurality of stages to switch data units between any of the plurality of fabric endpoints,

wherein a fabric endpoint of the fabric endpoints is configured to send, to a first-stage switch of a first fabric plane of the plurality of fabric planes, a self-ping message destined for the fabric endpoint,

wherein the fabric endpoint is configured to send, to a first-stage switch of a second fabric plane of the plurality of fabric planes, in response to determining the fabric endpoint has not received the self-ping message after a predetermined time, a fault data unit that includes an indication of a connectivity fault for the first fabric plane, the first fabric plane different than the second fabric plane.

16. The switching system of claim 15, further comprising:

switching, by an intermediate-stage switch of the multi-stage switch fabric, the self-ping message to a final-stage switch of the first fabric plane, the final-stage switch coupled by an output port to the fabric endpoint;

switching, by the final-stage switch to the fabric endpoint, the self-ping message; and

determining, by the fabric endpoint in response to receiving the self-ping message within the predetermined time, the first fabric plane does not have a connectivity fault.

17. The switching system of claim 15, wherein the plurality of fabric endpoints comprises one of a plurality of forwarding components interconnected within a network router by the multi-stage switch fabric.

18. The switching system of claim 15,

wherein the plurality of fabric endpoints comprises a plurality of edge devices interconnected within a data center by the multi-stage switch fabric.

19. The switching system of claim 15,

wherein the fabric endpoint comprises a first fabric endpoint,

wherein the fault data unit comprises a first fault data unit,

wherein a first-stage switch of a second fabric plane of the plurality of fabric planes is configured to receive a first fault data unit that includes the indication of the connectivity fault,

wherein the first-stage switch of the second fabric plane is configured to receive, from a second fabric endpoint of the plurality of fabric endpoints, a second fault data unit that includes the indication of the connectivity fault, and

wherein the first-stage switch of the second fabric plane is configured to send, to a final-stage switch of the second fabric plane and after consolidating the first fault data unit and the second fault data unit, a fault notification data unit that includes the indication of the connectivity fault.

20. The switching system of claim 15, wherein the indication of the connectivity fault identifies the first fabric plane.

21. The switching system of claim 15, wherein the fault data unit is destined for a broadcast address that specifies each of the plurality of fabric endpoints as an intended recipient of the fault data unit.

22. A switching system comprising:

a plurality of fabric endpoints;

a multi-stage switching fabric having a plurality of fabric planes each having a plurality of stages to switch data units between any of a plurality of fabric endpoints,

wherein a first-stage switch of a first fabric plane of the plurality of fabric planes is configured to receive an indication of a fabric fault for a second fabric plane of the plurality of fabric planes, the first fabric plane different than the second fabric plane;

wherein the first-stage switch is configured to send, to a first final-stage switch of the first fabric plane, a first fault notification data unit that includes the indication of the fabric fault, and

wherein the first-stage switch is configured to send, to a second final-stage switch of the first fabric plane, a second fault notification data unit that includes the indication of the fabric fault.

23. The switching system of claim 22,

wherein the indication comprises a fault data unit sent by a fabric endpoint of the plurality of fabric endpoints and the fabric fault is a connectivity fault for the second fabric plane,

wherein to sending the first fault notification data unit the first-stage switch is configured to send, in response to determining the fault data unit includes a broadcast address that specifies each of the plurality of fabric endpoints as an intended recipient of the fault data unit, the first fault notification data unit, and

wherein sending the second fault notification data unit comprises sending, by the first-stage switch in response to determining the fault data unit includes the broadcast address, the second fault notification data unit.

24. The switching system of claim 22,

wherein the first final-stage switch is configured to send, to at least one fabric endpoint of the plurality of fabric endpoints that is coupled to the first final-stage switch and in response to receiving the first fault notification data unit, a third fault notification data unit that includes the indication of the fabric fault.

25. The switching system of claim 24,

wherein the at least one fabric endpoint is configured to modify, in response to receiving the third fault notification data unit, a configuration of the at least one fabric endpoint to cause the at least one fabric endpoint to at least temporarily cease sending data units via the second fabric plane having the fabric fault.

26. The switching system of claim 22,

wherein the first fault notification data unit includes a first destination identifier that identifies the first final-stage switch, and

wherein the second fault notification data unit includes a second destination identifier that identifies the second final-stage switch.

27. The switching system of claim 22,

wherein an intermediate-stage switch of the first fabric plane is configured to receive the first fault notification data unit,

wherein the intermediate-stage switch is configured to send, in response to determining the first fault notification data unit includes a first destination identifier that identifies the first final-stage switch, the first fault notification data unit to the first final-stage switch.

28. The switching system of claim 22, wherein the fabric fault comprises a fabric link fault for fabric link connecting one of a first-stage switch with a second stage switch, a second stage switch with a third stage switch, and a third stage switch with a fabric endpoint of the plurality of fabric endpoints.

29. A computer-readable storage medium, the computer-readable storage medium comprising instructions for causing at least one programmable processor of a multi-stage switch fabric, having a plurality of fabric planes each having a plurality of stages to switch data units between any of a plurality of fabric endpoints, to:

send, by a fabric endpoint of the plurality of fabric endpoints to a first-stage switch of a first fabric plane of the plurality of fabric planes, a self-ping message destined for the fabric endpoint sending the self-ping message; and

send, by the fabric endpoint to a first-stage switch of a second fabric plane of the plurality of fabric planes and in response to determining the fabric endpoint has not received the self-ping message after a predetermined time, a fault data unit that includes an indication of a connectivity fault for the first fabric plane, the first fabric plane different than the second fabric plane.

30. A computer-readable storage medium, the computer-readable storage medium comprising instructions for causing at least one programmable processor of a multi-stage switch fabric, having a plurality of fabric planes each having a plurality of stages to switch data units between any of a plurality of fabric endpoints, to:

receive, by a first-stage switch of a first fabric plane of the plurality of fabric planes, an indication of a fabric fault for a second fabric plane of the plurality of fabric planes, the first fabric plane different than the second fabric plane;

send, by the first-stage switch to a first final-stage switch of the first fabric plane, a first fault notification data unit that includes the indication of the fabric fault; and

send, by the first-stage switch to a second final-stage switch of the first fabric plane, a second fault notification data unit that includes the indication of the fabric fault.