System and method for router arbiter protection switching

• US 6,999,411 B1
• Filed: 01/12/2001
• Issued: 02/14/2006
• Est. Priority Date: 01/12/2001
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Chiaro Networks, Ltd.
• Inventors: Brewer, Tony M., Palmer, Gregory S., Shaw, Keith W.
• Primary Examiner(s): Nguyen, Brian
• Assistant Examiner(s): Phan, Tri H.
• Application Number:
  US09/760,027
• Time in Patent Office: 1,859 Days
• Field of Search: 370216-217, 370219-220, 370/244, 455/8, 398/1, 398/2, 398/5, 714/2
• US Class Current: 370/220
• CPC Class Codes: H04L 45/28 using route fault recovery
• CPC Class Codes: H04L 45/583 Stackable routers
• CPC Class Codes: H04Q 11/0005 Switch and router aspects
• CPC Class Codes: H04Q 11/0066 Provisions for optical burst or packet networks
• CPC Class Codes: H04Q 2011/0039 Electrical control
• CPC Class Codes: H04Q 2011/0043 Fault tolerance

Abstract

In a router with redundant central arbiters, a set of control processors (CPs) determines which arbiter is active, which is standby, and when to switch between them. In normal operation ingress ASICs issue requests to the active central arbiter ASIC and keep-alive requests cyclically once per chunk period to the passive arbiter ASIC, which then returns keep-alive grants through the same links to the ingress ASICs and sends standby configuration information to the optical switch ASICs. The arbiter ASICs pass a switch-over decision simultaneously to the optical switch ASICs and ingress ASICs, which empty all queues of outstanding requests, and then resend all of those requests to the new active central arbiter after all queues are empty, such that no router traffic is lost. Mechanisms ensure that during the transition the ASICs properly recognize which data links are healthy and which arbiter is active.

Claims

1. A method of protection switching of redundant central arbiters in a router system, comprising:

selecting an active central arbiter;

selecting a standby central arbiter different from said active central arbiter;

communicating the active status of said active central arbiter;

communicating the standby status of said standby central arbiter;

receiving at said active central arbiter requests to pass chunks of data through an optical switching fabric;

at said active central arbiter in response to said requests concurrently issuing grants to pass said chunks and issuing optical switch configuration information corresponding to said grants, such that each said chunk passes during a single chunk period;

at said standby central arbiter periodically receiving keep-alive requests;

at said standby central arbiter concurrently issuing keep-alive grants and standby configuration information in response to said keep-alive requests; and

interchanging said active and standby status of said respective active and standby central arbiters, such that said standby central arbiter becomes a new active central arbiter and said active central arbiter becomes a new standby central arbiter;

wherein decisions of said selecting and said interchanging are initiated by a control processor selected from the group consisting of a master control processor (MCP) and shelf control processors, wherein said communicating of said active status occurs simultaneously from said active central arbiter to an optical switch ASIC and to a plurality of ingress ASICs; and

said communicating of said standby status occurs simultaneously from said standby central arbiter to an optical switch ASIC and to a plurality of ingress ASICs, wherein said issuing grants and said issuing corresponding switch configuration information both occur within the same chunk period, wherein said issuing keep-alive grants and said issuing standby configuration information both occur within the same chunk period, wherein said requests received by said active central arbiter are issued from a plurality of ingress ASICs through first multiple links, wherein said grants issued by said active central arbiter are received by said plurality of ingress ASICs through said first multiple links, and wherein, if a grant is not received by said ingress ASIC within a predetermined timeout period after the corresponding request was issued, then said ingress ASIC resets an internal outstanding request queue as well as request queues in said active central arbiter, until the respective queues of both said ingress ASIC and said active central arbiter are emptied of all outstanding requests, such that said ingress ASIC and said active central arbiter are synchronized in an empty state.

2. The method of claim 1 wherein said selecting and said interchanging are performed by writing a CSR to said respective active and standby central arbiters.

3. The method of claim 1 wherein no traffic in said router system is dropped.

4. The method of claim 3 wherein traffic in said router system is delayed no longer than six chunk periods.

5. A method of protection switching of redundant central arbiters in a router system, comprising:

selecting an active central arbiter;

selecting a standby central arbiter different from said active central arbiter;

communicating the active status of said active central arbiter;

communicating the standby status of said standby central arbiter;

receiving at said active central arbiter requests to pass chunks of data through an optical switching fabric;

at said active central arbiter in response to said requests concurrently issuing grants to pass said chunks and issuing optical switch configuration information corresponding to said grants, such that each said chunk passes during a single chunk period;

at said standby central arbiter periodically receiving keep-alive requests;

at said standby central arbiter concurrently issuing keep-alive grants and standby configuration information in response to said keep-alive requests;

interchanging said active and standby status of said respective active and standby central arbiters, such that said standby central arbiter becomes a new active central arbiter and said active central arbiter becomes a new standby central arbiter;

wherein decisions of said selecting and said interchanging are initiated by a control processor selected from the group consisting of a master control processor (MCP) and shelf control processors, wherein;

said communicating of said active status occurs simultaneously from said active central arbiter to an optical switch ASIC and to a plurality of ingress ASICs; and

said communicating of said standby status occurs simultaneously from said standby central arbiter to an optical switch ASIC and to a plurality of ingress ASICs, wherein said issuing grants and said issuing corresponding switch configuration information both occur within the same chunk period, wherein said issuing keep-alive grants and said issuing standby configuration information both occur within the same chunk period, wherein said requests received by said active central arbiter are issued from a plurality of ingress ASICs through first multiple links, wherein said grants issued by said active central arbiter are received by said plurality of ingress ASICs through said first multiple links, wherein said optical switch configuration information is issued by said active central arbiter to said optical switching fabric through second multiple links differing from said first multiple links, wherein said optical switch configuration information is issued by an active central arbiter ASIC in said active central arbiter to an optical switch ASIC in said optical switching fabric, wherein said keep-alive requests received by said standby central arbiter are issued from a plurality of ingress ASICs through third multiple links differing from said first and second multiple links, wherein said keep-alive grants issued by said standby central arbiter are received by said plurality of ingress ASICs through said third multiple links differing from said first and second multiple links, wherein said standby configuration information is issued by said standby central arbiter to said optical switching fabric through fourth multiple links differing from said first, second, and third multiple links, wherein said standby configuration information is issued by a standby central arbiter ASIC in said standby central arbiter to an optical switch ASIC in said optical switching fabric, wherein the issuing of said keep-alive requests and the receiving of said keep-alive grants by said ingress ASICs is performed cyclically;

encoding grants, requests, keep-alive grants, keep-alive requests, optical switch configuration information, standby configuration information, and active or standby status communication messages with error detection codes;

sending said grants, requests, keep-alive grants, keep-alive requests, optical switch configuration information, standby configuration information, and arbiter active or standby status bit messages with error detection codes to destinations through multiple links selected from the group consisting of said first, second, third, and fourth multiple links;

within each chunk period at said destinations receiving said messages and decoding said error detection codes to determine a healthy or weak state of said multiple links; and

within each chunk period intercomparing said determinations among a plurality of said destinations to decide whether to interchange the active or standby status of said central arbiters.

6. The method of claim 5 wherein said error detection codes comprise a cyclical redundancy check (CRC).

7. The method of claim 5 wherein said destinations comprise said ingress ASICs, said active arbiter ASIC, said standby arbiter ASIC, and said optical switch ASICs.

8. The method of claim 7 wherein:

if said destination decodes as error-free a message received through a said multiple link, then said multiple link is determined to be healthy, and said destination decodes an

arbiter active or standby status bit received through said healthy link;

otherwise

said destination determines said multiple link to be weak, and said destination does not decode an arbiter active or standby status bit received through said weak link.

9. The method of claim 8 further comprising:

if at said ingress ASIC, said first and said third multiple links are both determined to have the same healthy link state and report the same active or standby arbiter status; and

at said optical switch ASIC, said second and said fourth multiple links are both determined to have the same healthy link state and report the same active or standby arbiter status;

then

said ingress ASIC and said optical switch ASIC intercompare said determinations and decide not to interchange said active or standby arbiter status;

otherwise

if at said ingress ASIC only one of said first and said third multiple links is determined to have both a healthy link state and active reported arbiter status, and at said optical switch ASIC only one of said second and fourth multiple links is determined to have both a healthy link state and active reported arbiter status, then said ingress ASIC and said optical switch ASIC intercompare said determinations and recognize as active the link reporting active arbiter status;

otherwise

if at said ingress ASIC only one of said first and said third multiple links is determined to have both a healthy link state and standby reported arbiter status, and at said optical switch ASIC only one of said second and fourth multiple links is determined to have both a healthy link state and standby reported arbiter status, then said ingress ASIC and said optical switch ASIC intercompare said determinations, recognize as active the link not reporting standby arbiter status, and request said control processor (CP) to initiate an interchange of active or standby arbiter status.

10. The method of claim 9 wherein said interchanging said active and standby status of said respective active and standby central arbiters comprises:

setting by said CP the active or standby status bit of said active central arbiter from active to standby status;

setting by said CP the active or standby status bit of said standby central arbiter from standby to active status;

communicating said setting of each of said active or standby bits by each of said respective central arbiters simultaneously to said ingress ASICs and said optical switch ASICs;

in response to said communicating, resetting by said ingress ASICs of internal outstanding request queues of said ingress ASICs and of request queues in said active and standby central arbiters, until the respective queues of both said ingress ASIC and said active and standby central arbiters are emptied of all outstanding requests, such that said ingress ASICs and said arbiters are synchronized in an empty state; and

proceeding with normal operation, such that no traffic in said router system is dropped.

11. The method of claim 10, wherein said interchanging is initiated by said CP following an occurrence selected from the group consisting of failure to receive a grant within a specified period after issuing a corresponding request, failure at a destination to receive an error-free message through any one of said first, second, third, and fourth multiple links, and scheduling of maintenance involving one of said central arbiters.

12. A communication network router incorporating a system for central arbitration protection, said system comprising:

redundant interchangeable central arbiters including an active central arbiter and a standby central arbiter;

a plurality of control processors including a master control processor and distributed control processors, said control processors being interconnected with one another and with said redundant central arbiters through a control network;

a plurality of ingress ASICs interconnected through first multiple links with said active central arbiter and through third multiple links with said standby central arbiter;

an optical switching fabric containing an optical switch ASIC interconnected through second multiple links with said active central arbiter and through fourth multiple links with said standby central arbiter;

said standby central arbiter configured to receive keep-alive requests and send keep-alive grants cyclically through said third multiple links, and to send standby configuration information through said fourth multiple links in response to said keep-alive requests;

said redundant central arbiters being configured to interchange in response to a switch over decision from said plurality of control processors, such that said standby central arbiter becomes a new active central arbiter and said active central arbiter becomes a new standby central arbiter, such that said ingress ASICs reset request queues within said ingress ASICs and within said redundant central arbiters, and such that the active and standby status of respective redundant central arbiters is communicated through said first through said fourth multiple links; and

said ingress ASICs and said optical switch ASICs being configured to cooperatively determine a healthy or weak state of each of respective first through fourth multiple links and said active and standby status of said respective redundant central arbiters.