MANAGING LATENCIES IN A MULTIPROCESSOR INTERCONNECT

• US 20110179423A1
• Filed: 10/02/2008
• Published: 07/21/2011
• Est. Priority Date: 10/02/2008
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Warner, Craig, Bockhaus, John W., Gostin, Gary, Lesartre, Gregg B.
• Granted Patent: US 8,732,331 B2
• Time in Patent Office: Days
• US Class Current: 718/104
• CPC Class Codes: G06F 15/173 using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake

Abstract

In a computing system having a plurality of transaction source nodes issuing transactions into a switching fabric, an underserviced node notifies source nodes in the system that it needs additional system bandwidth to timely complete an ongoing transaction. The notified nodes continue to process already started transactions to completion, but stop the introduction of new traffic into the fabric until such time as the underserviced node indicates that it has progressed to a preselected point.

Claims

1. In a computer system comprising a plurality of source nodes able to intercommunicate via a system switching fabric by introducing transactions onto the fabric, a method of managing the latency of internode communications, the method comprising:

initiating communication of a first transaction from an initiator source node to a target source node;

determining that the initiator node needs additional system bandwidth to the target node to timely complete the first transaction;

notifying a plurality of source nodes that the initiator node needs additional system bandwidth;

responsive to receiving the notification, stopping the introduction of new transactions onto the system fabric by at least some of the notified nodes;

continuing the communication of the first transaction from the initiator node to the target node;

determining that the first transaction has progressed to a preselected point;

indicating to the notified nodes that the initiator node no longer needs additional system bandwidth; and

responsive to receiving the indication, ceasing to stop the introduction of new transactions onto the system fabric by the notified nodes,

whereby the latency of the communication of the first transaction from the initiator node to the target node is managed.

2. The method of claim 1, further comprising, in conjunction with the initiating communication step, starting a transaction timer, wherein the step of determining that the initiator node needs additional system bandwidth to the target node comprises determining that the transaction timer has reached a select threshold.

3. The method of claim 1, wherein the notifying step comprises the initiator node sending a message (PWALLUP) to the plurality of source nodes that they should stop introducing new transactions onto the system fabric.

4. The method of claim 1, wherein the step of determining that the first transaction has progressed to a select point further comprises determining that all of the notified nodes'"'"' transactions have progressed to respective select points.

5. The method of claim 4, further comprising receiving by the initiator a message (PWALLCMP) from each of the notified nodes that their transactions have progressed to respective select points.

6. The method of claim 1, wherein the computer system is a partitionable computer system,

wherein the notifying step comprises the initiator node multicasting a message (PWALLUP) to all source nodes of the partition indicating that said nodes should stop introducing new transactions onto the system fabric, and

wherein the step of determining that the first transaction has progressed to a select point comprises receiving a message (PWALLCMP) from each of the notified nodes that their transactions have progressed to respective select points.

7. The method of claim 6, wherein the step of indicating that the initiator node no longer needs additional bandwidth comprises the initiator node multicasting a message (PWALLDN) to all of the said source nodes of the partition, indicating that the initiator node has received a PWALLCMP from each said source node.

8. The method of claim 7, further comprising the said source nodes returning to normal operation responsive to receiving their respective PWALLDNs.

9. The method of claim 8, further comprising:

a second initiator multicasting a second PWALLUP to all of the said source nodes in the partition while the first initiator is multicasting the first PWALLUP; and

selecting one of the initiators as a master and notifying the other, non-master initiator of the selection,

wherein the PWALLDN is sent by the master, and the non-master initiator does not issue any PWALLDNs.

10. The method of claim 9, further comprising the non-master initiator waiting to issue a PWALLCMP to the master until it has received all of its own PWALLCMPs.

11. A computer readable medium storing computer readable instructions which, when read by a computing system having a plurality of source nodes able to intercommunicate via a system switching fabric by introducing transactions onto the fabric, cause the computing system to perform a method to manage the latency of internode communications, the method comprising:

initiating communication of a first transaction from an initiator source node to a target source node;

determining that the initiator node needs additional system bandwidth to the target node to timely complete the first transaction;

notifying a plurality of source nodes including the initiator node, the target node, and at least one third source node, that the initiator node needs additional system bandwidth;

responsive to receiving the notification, stopping the introduction of new transactions onto the system fabric by at least some of the notified nodes;

continuing the communication of the first transaction from the initiator node to the target node;

determining that the first transaction has progressed to a select point;

indicating to the notified nodes that the initiator node no longer needs additional system bandwidth; and

responsive to receiving the indication, ceasing to stop the introduction of new transactions onto the system fabric by the notified nodes,

whereby the latency of the communication of the first transaction from the initiator node to the target node is managed.

12. The system of claim 11, wherein the instruction set is operable to cause the nodes to perform a method for the managing of the internode communications latency, the method comprising:

initiating communication of a first transaction from an initiator source node to a target source node and starting the transaction timer for the first transaction;

determining that the initiator node needs additional system bandwidth to the target node to timely complete the first transaction by determining that the transaction timer has reached a select threshold;

notifying a plurality of the source nodes that the initiator node needs additional system bandwidth, by sending a message (PWALLUP) to the plurality of source nodes indicating that they should stop introducing new transactions onto the system fabric;

responsive to receiving the notification, stopping introducing new transactions onto the system fabric by at least some of the notified nodes, and continuing to process by the notified nodes one or more transactions already being processed or received for processing at the time the notified nodes received the PWALLUP;

receiving by the initiator a message (PWALLCMP) from each of the notified nodes that their transactions have progressed to respective select points;

indicating to the notified nodes that the initiator node no longer needs additional system bandwidth; and

responsive to the indication, ceasing to stop the introduction of new transactions onto the system fabric by the notified nodes,

whereby the latency of the communication of the transaction from the initiator node to the target node is managed.

13. The system of claim 12, wherein the plurality of source nodes of the computing system are organized into a plurality of partitions, each partition operable to execute a separate instance of a computer operating system, wherein the instruction set provides instructions only to the source nodes of a partition comprising both the initiator node that needs additional system bandwidth and the target node to which the initiator node needs additional system bandwidth.

14. The system of claim 13, further comprising a system state machine having a timer (wall-down timer) that issues PWALLDNs to the notified source nodes in the event the operating system of the partition comprising the notified source nodes ceases to respond.

11-1. A system for managing communication latency in a computing system, comprising:

a plurality of source nodes, each source node comprising a microprocessor interface or an input/output (I/O) interface, each source node further comprising a transaction timer for tracking the time elapsed for each transaction issued to a system switching fabric by said node;

the system switching fabric for establishing a plurality of communication channels between the source nodes for internode communication of transactions and system messages; and

an instruction set operable to provide instructions to the source nodes for generating the system messages for managing internode communication latency.

15. A system for managing communication latency in a computing system, comprising:

a plurality of means for issuing transactions and system messages;

means for tracking the time elapsed for each transaction issued;

means for establishing a plurality of communication channels between the means for issuing transactions and system messages for communication of the transactions and system messages between the means for issuing; and

an instruction set operable to provide instructions to the means for issuing system messages, for managing latency of the communication of the transactions.