Implementing network manager quarantine mode

• US 8,914,538 B2
• Filed: 05/13/2010
• Issued: 12/16/2014
• Est. Priority Date: 05/13/2010
• Status: Active Grant

• Current Assignee: Lenovo International Limited (Lenovo Group Ltd.)
• Original Assignee: Lenovo Enterprise Solutions (Singapore) Pte., Ltd. (Lenovo Group Ltd.)
• Inventors: Byers, Marcy E., Valk, Kenneth M.
• Primary Examiner(s): Dharia, Rupal
• Assistant Examiner(s): MCKENZIE, MARCUS A
• Application Number:
  US12/779,076
• Publication Number:
  US 20110283028A1
• Time in Patent Office: 1,678 Days
• Field of Search: 709/237
• US Class Current: 709/237
• CPC Class Codes: G06F 13/4022 using switching circuits, e.g. switching matrix, connection or expansion network G06F13/4009 takes precedence

Abstract

A method and circuit for implementing a network manager quarantine mode in an interconnect system, and a design structure on which the subject circuit resides are provided. A respective network manager on a source interconnect chip and a destination interconnect chip sends end-to-end (ETE) heartbeats on each path between the source and destination interconnect chips. Each network manager maintains a heartbeat table with counters to track each path to each destination interconnect chip. When a first network manager of a first interconnect chip detects a change from at least one valid path to no working paths for a second interconnect chip of the interconnect chips, the quarantine mode is established for a programmable quarantine time interval and all paths are prevented from advertising good heartbeats during the quarantine time interval.

Claims

1. A method for implementing a network manager quarantine mode in an interconnect system, said method comprising:

providing a plurality of paths defined by links connected between each source interconnect chip and each destination interconnect chip;

providing each said interconnect chip with a network manager, said network manager sending end-to-end (ETE) heartbeats on said plurality of paths through the interconnect system to each destination interconnect chip;

each said network manager maintaining a heartbeat table with a respective counter to track path status of each path through the interconnect system to each destination interconnect chip;

each path being identified by a chip-identification (ID), exit port;

said heartbeat table including a respective up/down counter for each chip-ID, exit port, each said up/down counter for each chip-ID, exit port being incremented with a heartbeat being received for said chip-ID, and each said up/down counter for each chip-ID, exit port being decremented on a set interval;

first predefined counter values of each said up/down counter for each chip-ID, exit port represent a bad heartbeat and second first predefined counter values of each said up/down counter for each chip-ID, exit port represent a good heart beat;

said heartbeat table including an heartbeat return bit from a last heartbeat received, a hop count indicating a hop count of said last heartbeat received; and

an enable bit to control sending or not sending heartbeats;

said network manager of a first interconnect chip detecting a path status change from at least one valid path to no working paths for a second interconnect chip; and

said network manager of the first interconnect chip, responsive to said detected path status change to no working paths for said second interconnect chip, establishing a quarantine mode for a programmable quarantine time interval to prevent advertising good heartbeats for each of said paths for said second interconnect chip during said programmable quarantine time interval;

said network manager of the first interconnect chip, responsive to said programmable quarantine time interval expiring, advertising good heartbeats for each of said paths for said second interconnect chip, and sending path update responsive to said paths for said second interconnect chip being indicated as good; and

wherein each said network manager maintaining said heartbeat table with said respective counter to track status of each path through the interconnect system to each destination interconnect chip includes providing said respective up/down counter with a predefined multiple-bit counter, where a first count range represents said bad heartbeat and a second count range represents said good heartbeat, said second count range being greater than said first count range.

2. The method as recited in claim 1 includes said network manager on said second interconnect chip declaring all paths lost to said first interconnect chip during the quarantine time interval responsive to the quarantine mode of the first network manager.

3. The method as recited in claim 1 wherein providing said respective up/down counter with said predefined multiple-bit counter includes providing a 7-bit counter, where counts 0-63 represent said bad heartbeat, and counts 64-127 represent said good heartbeat.

4. The method as recited in claim 3 includes responsive to said programmable quarantine time interval expiring, and sending a path update responsive to said paths for said second interconnect chip being indicated as good includes allowing each of said paths to be declared working responsive to being validated by both said network manager of the first interconnect chip and said network manager on said second interconnect chip chips.

5. The method as recited in claim 3 includes transmitting a heartbeat at a higher rate than said set decrement rate of said respective up/down counter for each said path.

6. The method as recited in claim 3 includes identifying a path status change, and broadcasting a path status update to each transport layer of said interconnect chip of said network manager.

8. The method as recited in claim 3 includes identifying a bad path responsive to a predefined count value of said respective up/down counter for said path.

9. The method as recited in claim 3 includes providing predefined fields with said respective up/down counter for each said path, said predefined fields including said heartbeat return bit from said last heartbeat received.

10. The method as recited in claim 3 includes providing predefined fields with said respective up/down counter for each said path, said predefined fields including said hop count indicating a hop count of said last heartbeat received.

11. The method as recited in claim 3 includes providing predefined fields with said respective up/down counter for each said path, said predefined fields including said enable bit to control sending or not sending heartbeats.

7. The method as recited in 3 includes identifying a good path responsive to a predefined count value of said respective up/down counter for said path.

12. A circuit for implementing a network manager quarantine mode in an interconnect system, said circuit comprising:

an interconnect chip including a network manager, said network manager sending end-to-end (ETE) heartbeats on a plurality of paths through the interconnect system to each destination interconnect chip;

said plurality of paths defined by links connected between said interconnect chip and each said destination interconnect chip;

each said network manager maintaining a heartbeat table with a respective counter to track path status of each said path through the interconnect system to each destination interconnect chip;

each path being identified by a chip-identification (ID), exit port;

said heartbeat table including a respective up/down counter for each chip-ID, exit port, each said up/down counter for each chip-ID, exit port being incremented with a heartbeat being received for said chip-ID, and each said up/down counter for each chip-ID, exit port being decremented on a set interval;

first predefined counter values of each said up/down counter for each chip-ID, exit port represent a bad heartbeat and second first predefined counter values of each said up/down counter for each chip-ID, exit port represent a good heart beat;

said heartbeat table including an heartbeat return bit from a last heartbeat received, a hop count indicating a hop count of said last heartbeat received; and

an enable bit to control sending or not sending heartbeats;

said network manager of a first interconnect chip detecting a path status change from at least one valid path to no working paths to a second interconnect chip; and

said network manager of the first interconnect chip, responsive to said detected path status change to no working paths to said second interconnect chip, establishing a quarantine mode for a programmable quarantine time interval to prevent advertising good heartbeats for each of said paths for said second interconnect chip during said programmable quarantine time interval;

said network manager of the first interconnect chip, responsive to said programmable quarantine time interval expiring, advertising good heartbeats for each of said paths for said second interconnect chip, and sending a path update responsive to said paths for said second interconnect chip being indicated as good; and

wherein each said network manager maintaining said heartbeat table with said respective counter to track status of each path through the interconnect system to each destination interconnect chip includes providing said respective up/down counter with a predefined multiple-bit counter, where a first count range represents said bad heartbeat and a second count range represents said good heartbeat, said second count range being greater than said first count range.

13. The circuit as recited in claim 12 includes said network manager on said second interconnect chip declares all paths lost to said first interconnect chip during the quarantine time interval responsive to the quarantine mode established by said network manager of said first interconnect chip.

14. The circuit as recited in claim 12 wherein said predefined multiple-bit counter includes a 7-bit counter, where counts 0-63 represent said bad heartbeat, and counts 64-127 represent said good heartbeat.

15. The circuit as recited in claim 12 wherein said network manager transmits a heartbeat at a higher rate than said set decrement rate of said respective up/down counter for each said path.

16. The circuit as recited in claim 12 wherein said heartbeat table includes predefined fields provided with said respective up/down counter for each said path, said predefined fields including said heartbeat return bit from said last heartbeat received.

17. The circuit as recited in claim 12 wherein said heartbeat table includes predefined fields provided with said respective up/down counter for each said path, said predefined fields including said hop count indicating a hop count of said last heartbeat received.

18. The circuit as recited in claim 12 wherein said network manager responsive to said programmable quarantine time interval expiring, and sending a path update responsive to said paths for said second interconnect chip being indicated as good includes allowing each of said paths to be declared working responsive to being validated by both said network manager of the first interconnect chip and said network manager on said second interconnect chip.

19. A multiple-path local rack interconnect system comprising:

a plurality of interconnect chips including a source interconnect chip and a destination interconnect chip;

a plurality of paths defined by serial links connected between each of said plurality of interconnect chips;

said interconnect chips including a network manager, said network manager sending end-to-end (ETE) heartbeats on a plurality of paths through the interconnect system to each destination interconnect chip;

said plurality of paths defined by links connected between said interconnect chip and each said destination interconnect chip;

each said network manager maintaining a heartbeat table with a respective counter to track path status of each said path through the interconnect system to each destination interconnect chip;

each path being identified by a chip-identification (ID), exit port;

said heartbeat table including a respective up/down counter for each chip-ID, exit port, each said up/down counter for each chip-ID, exit port being incremented with a heartbeat being received for said chip-ID, and each said up/down counter for each chip-ID, exit port being decremented on a set interval;

first predefined counter values of each said up/down counter for each chip-ID, exit port represent a bad heartbeat and second first predefined counter values of each said up/down counter for each chip-ID, exit port represent a good heart beat;

said heartbeat table including an heartbeat return bit from a last heartbeat received, a hop count indicating a hop count of said last heartbeat received; and

an enable bit to control sending or not sending heartbeats;

said network manager of a first interconnect chip detecting a path status change from at least one valid path to no working paths to a second interconnect chip; and

said network manager of the first interconnect chip, responsive to said detected path status change to no working paths to said second interconnect chip, establishing a quarantine mode for a programmable quarantine time interval to prevent advertising good heartbeats for each of said paths for said second interconnect chip during said programmable during said quarantine time interval;

said network manager of the first interconnect chip, responsive to said programmable quarantine time interval expiring, advertising good heartbeats for each of said paths for said second interconnect chip, and sending path update responsive to said paths for said second interconnect chip being indicated as good; and

wherein each said network manager maintaining said heartbeat table with said respective counter to track status of each path through the interconnect system to each destination interconnect chip includes providing said respective up/down counter with a predefined multiple-bit counter, where a first count range represents said bad heartbeat and a second count range represents said good heartbeat, said second count range being greater than said first count range.

20. The multiple-path local rack interconnect system as recited in claim 19 wherein said network manager on said second interconnect chip declares all paths lost to said first interconnect chip during the quarantine time interval responsive to the quarantine mode established by said network manager of said first interconnect chip.

21. A design structure embodied in a non-transitory machine readable medium used in a design process, the design structure comprising:

a circuit tangibly embodied in the non-transitory machine readable medium used in the design process, said circuit for implementing a network manager quarantine mode in an interconnect system, said circuit comprising;

an interconnect chip including a network manager, said network manager sending end-to-end (ETE) heartbeats on a plurality of paths through the interconnect system to each destination interconnect chip;

said plurality of paths defined by links connected between said interconnect chip and each said destination interconnect chip;

each said network manager maintaining a heartbeat table with a respective counter to track path status of each said path through the interconnect system to each destination interconnect chip;

each said path being identified by a chip-identification (ID), exit port;

said heartbeat table including a respective up/down counter for each chip-ID, exit port, each said up/down counter for each chip-ID, exit port being incremented with a heartbeat being received for said chip-ID, and each said up/down counter for each chip-ID, exit port being decremented on a set interval;

first predefined counter values of each said up/down counter for each chip-ID, exit port represent a bad heartbeat and second first predefined counter values of each said up/down counter for each chip-ID, exit port represent a good heart beat;

said heartbeat table including an heartbeat return bit from a last heartbeat received, a hop count indicating a hop count of said last heartbeat received; and

an enable bit to control sending or not sending heartbeats;

said network manager of a first interconnect chip detecting a path status change from at least one valid path to no working paths to a second interconnect chip; and

said network manager of the first interconnect chip, responsive to said detected path status change to no working paths to said second interconnect chip, establishing a quarantine mode for a programmable quarantine time interval to prevent advertising good heartbeats for each of said paths for said second interconnect chip during said programmable during said quarantine time interval;

said network manager of the first interconnect chip, responsive to said programmable quarantine time interval expiring, advertising good heartbeats for each of said paths for said second interconnect chip, and sending path update responsive to said paths for said second interconnect chip being indicated as good, wherein the design structure, when read and used in a manufacture of a semiconductor chip produces a chip comprising said circuit; and

wherein each said network manager maintaining said heartbeat table with said respective counter to track status of each path through the interconnect system to each destination interconnect chip includes providing said respective up/down counter with a predefined multiple-bit counter, where a first count range represents said bad heartbeat and a second count range represents said good heartbeat, said second count range being greater than said first count range.

22. The design structure of claim 21, wherein the design structure comprises a netlist, which describes said circuit.

23. The design structure of claim 21, wherein the design structure resides on storage medium as a data format used for the exchange of layout data of integrated circuits.

24. The design structure of claim 21, wherein the design structure includes at least one of test data files, characterization data, verification data, or design specifications.