Multiprocessor system architecture with high availability

• US 4,958,273 A
• Filed: 08/26/1987
• Issued: 09/18/1990
• Est. Priority Date: 08/26/1987
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bunten, Roland J., Anderson, Patrick E., Mirabeau, Serge, Hruby, Ronda J., Higgins, William T.
• Primary Examiner(s): Shaw, Gareth D.
• Assistant Examiner(s): Kulik, Paul
• Application Number:
  US07/089,372
• Time in Patent Office: 1,119 Days
• Field of Search: 364/200 MS File, 364/900 MS File, 364/200, 364/900, 371/8, 371/10, 371/11
• US Class Current: 712/29
• CPC Class Codes: G06F 11/1666 where the redundant component is memory or memory area
• CPC Class Codes: G06F 11/1679 at clock signal level
• CPC Class Codes: G06F 11/20 using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
• CPC Class Codes: G06F 11/2043 where the redundant components share a common memory address space
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups

Abstract

High availability is achieved in a multiprocessor system by grouping the processors into two clusters operating on different clock and power boundaries. In each cluster is an array containing a substantially identical copy of the system information relating to the operation of the processors. Each processor has a local port for accessing the local array and a remote port for accessing the remote array. When an update is made on the system information, it is made on both the local and remote arrays with the remote update performed transparently.

Claims

1. A multiprocessor system of the kind in which each processor accesses common system information asynchronously, the information being updateable by more than one processor simultaneously and dynamically, the system comprising:

two processor clusters each comprising at least two processors, each cluster operating on a separate clock and power boundary,

an array in each cluster having definable, lockable regions therein for storing said system information,

a plurality of local ports and a plurality of remote ports in each cluster coupled to the array for enabling execution of array operations in that cluster;

each processor in each cluster being connected to a respective local port, each local port having at least one register for storing address, data, control and status information and each local port being coupled to a corresponding one of said remote ports in the other cluster for communication therewith;

each remote port having at least one register, each such register being connected to a corresponding register in the respective coupled local port of the other cluster, said at least one register storing said address, data, control and status information for causing array operations to be executed selectively in either cluster or in both of said clusters concurrently; and

an arbiter in each cluster connected to the local ports and remote ports therein, each arbiter being coupled to its corresponding array for controlling access thereto and operation thereof, each arbiter communicating with the arbiter of the other cluster for arbitrating global operations, including dual mode TEST AND SET and UNCONDITIONAL SET operations, to its own array and the other (remote) array, and each arbiter comprising means for executing read/write operations in unused array cycles between the TEST portion and the SET portion of a TEST AND SET operation.

2. The system as in claim 1, wherein each processor, via its associated local port and/or its corresponding remote port in the other cluster, issues an arbiter reset command to at least one of the arbiters for recover after an arbiter error, and wherein each arbiter further comprises means for inhibiting a processor from executing an arbiter reset command during recovery from an arbiter error when such a command has already been executed by another processor.

3. In a multiprocessor system wherein operation of the processors requires asynchronous access to common system information, the common system information being dynamically updateable by more than one processor concurrently, a method for improving the availability of the multiprocessor system and for communicating information between processors, comprising the steps of:

dividing the processors into two clusters;

providing an array in each cluster to store a copy of said system information, said array having a plurality of definable, lockable regions;

operating each cluster on a separate power and clock boundary;

connecting each processor in a particular cluster via a respective local port to the array and to an arbiter for that particular cluster and also to a corresponding remote port in the other cluster;

connecting each remote port in said other cluster to the array and to an arbiter for said other cluster;

at each update of said system information by a processor in its own cluster via its associated local port, also updating concurrently the copy of system information in the other cluster via the corresponding remote port therein if the array in the other cluster is accessible;

arbitrating array operations via said arbiters in both arrays among all requesting processors; and

while performing a TEST AND SET operation for one processor in one of the clusters, performing read/write operations for another processor in either of said clusters during unused array cycles between the TEST portion and the SET portion of the TEST AND SET operation.

4. A method as in claim 3, further comprising the step by each processor of performing an array read, write, TEST AND SET or UNCONDITIONAL SET operation in one of the following operating modes:

(1) local array only, (2) remote array only, or (3) both arrays, and wherein each processor can perform one of said types of array operation in one of said operation modes and another array operation in another of said operation modes.

5. A method as in claim 4, further comprising the step by each processor of directing other processors to operate under one of said operating modes.

6. A method as in claim 3, further comprising the step by one of said processors of performing recovery after an arbiter error, and the step by an arbiter of inhibiting more than one processor from executing an arbiter reset command during recovery from the same error.

7. A method as in claim 6, further comprising the step by each processor of checking whether the other cluster is operational, and restoring its own array using information from the other array if the remote cluster is operational.

8. A method as in claim 3, comprising the steps of:

initiating a mode change in response to one of the processors executing a TEST AND SET operation to effect a lock on a region in an at least one of the arrays; and

notifying each of the other processors of said mode change at the time it next accesses an array on which said lock was effected.

9. A method as in claim 3, comprising the steps of:

responsive to initiation of a mode change by a particular processor in one of the clusters, setting reconfiguration data in the local array of said one cluster and activating global interrupts in a local mode in said one cluster; and

transmitting the global interrupts from said particular processor to other processors in said one cluster via their local ports and to processors in the other cluster via their associated remote ports in said one cluster.

10. A method as in claim 3, comprising the following steps for reconfiguring the system for maximum availability without performance degradation following a failure in one of the arrays or arbiters or remote ports or under certain circumstances in a local port:

initiating a mode change in response to one of the processors executing a TEST AND SET operation on a lock byte in a mode change control block to effect a lock on a region in at least one of the arrays;

writing reconfiguration data in the control block; and

responsive to said one processor releasing the lock by an UNCONDITIONAL SET operation, setting global interrupt latches in the local and remote ports of the other processors in at least one of said clusters to indicate a mode change is pending.