Method for deadlock avoidance in a cluster environment

• US 6,738,871 B2
• Filed: 12/22/2000
• Issued: 05/18/2004
• Est. Priority Date: 12/22/2000
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Mak, Pak-Kin, Blake, Michael A., Van Huben, Gary A., Seigler, Adrian Eric
• Primary Examiner(s): Padmanabhan, Mano
• Assistant Examiner(s): Ross, John M
• Application Number:
  US09/745,830
• Publication Number:
  US 20020083149A1
• Time in Patent Office: 1,243 Days
• Field of Search: 711/124, 711/130, 711/148, 711/147, 711/150, 709/214
• US Class Current: 711/150
• CPC Class Codes: G06F 15/17381 Two dimensional, e.g. mesh, torus

Abstract

A remote resource management system for managing resources in a symmetrical multiprocessing environment having a plurality of clusters of symmetric multiprocessors each of which provides interfaces between cluster nodes of the symmetric multiprocessor system with a local interface and an interface controller. One or more remote storage controllers each has a local interface controller and a local-to-remote data bus. A remote fetch controller is responsible for processing data accesses in accordance with the methods described.

Claims

1. A method for deadlock avoidance in a clustered symmetric multiprocessor system having a plurality of clusters interfaced with one another, each cluster having:

(a) a local fetch interface controller, and

(b) a local store interface controller, and

(c) a remote fetch controller, and

(d) a remote storage controller, and

(e) a local-to-remote data bus and

(f) an interface controller,

(g) a plurality of processors,

(h) a shared cache memory,

(i) a plurality of I/O adapters, and

(j) a main memory accessible from the cluster, said method comprising the steps of;

detecting a plurality of storage requests between a local and a remote cluster,

responding to the detected storage requests to permit one request of a plurality of said storage requests to proceed while actively aborting all remaining storage requests of said plurality of storage requests and notifying the requesting local controllers of a successful completion of an aborted storage request,

causing a remote storage controller to store data in response to said one storage request which was permitted to proceed, and

then storing the data of said response that proceeded from said remote storage controller into said main memory.

2. A method for deadlock avoidance in a clustered symmetric multiprocessor system having a plurality of clusters interfaced with one another, each cluster

having:

(a) a local fetch interface controller, and

(b) a local store interface controller, and

(c) a remote fetch controller, and

(d) a remote storage controller, and

(e) a local-to-remote data bus and

(f) an interface controller,

(g) a plurality of processors,

(h) a shared cache memory,

(i) a plurality of I/O adapters, and

(j) a main memory accessible from the cluster,

said method comprising the steps of;

detecting a plurality of fetch requests between a local and a remote cluster,

responding to the detected fetch requests to permit at least one request of said plurality of fetch requests to proceed while actively delaying action among remaining requests of said plurality of requests,

causing a remote fetch controller to wait until said one request which was permitted to proceed has completed, and

then proceeding with a delayed request among said remaining requests of said plurality of requests,

whereby deadlock avoidance is achieved through the use of asynchronous cast outs which are permitted direct access to shared memory without the need to perform address compare interlocks against other simultaneous operations, and ensuring the cast out will complete without interruption from said simultaneous operations.

3. A method for deadlock avoidance in a clustered symmetric multiprocessor system having a plurality of clusters interfaced with one another, each cluster

having:

(a) a local fetch interface controller, and

(b) a local store interface controller, and

(c) a remote fetch controller, and

(d) a remote storage controller, and

(e) a local-to-remote data bus and

(f) an interface controller,

(g) a plurality of processors,

(h) a shared cache memory,

(i) a plurality of I/O adapters, and

(j) a main memory accessible from the cluster,

said method comprising the steps of;

detecting a plurality of fetch requests between a local and a remote cluster,

responding to the detected fetch requests to permit at least one request of said plurality of fetch requests to proceed while actively delaying action among remaining requests of said plurality of requests,

causing a remote fetch controller to wait until said one request which was permitted to proceed has completed, and

then proceeding with a delayed request among said remaining requests of said plurality of requests,

whereby memory operations utilize an abort mechanism which allows the move page operation to cease prior to performing any memory access and permit other operations to continue and wherein said memory operations include any combination of

a. said page move store operations involving movement of data from one memory location to

 

a target memory location and

b. cache coherency operations involving invalidation of remote shared caches and

c. I/O store operations involving storage of I/O data into main memory or a shared cache and

d. store pad operations involving replication of data patterns into main memory.

4. A method for deadlock avoidance in a clustered symmetric multiprocessor system having a plurality of clusters interfaced with one another, each cluster

having:

(a) a local fetch interface controller, and

(b) a local store interface controller, and

(c) a remote fetch controller, and

(d) a remote storage controller, and

(e) a local-to-remote data bus and

(f) an interface controller,

(g) a plurality of processors,

(h) a shared cache memory,

(i) a plurality of I/O adapters, and

(j) a main memory accessible from the cluster,

said method comprising the steps of;

detecting a plurality of fetch requests between a local and a remote cluster,

responding to the detected fetch requests to permit at least one request of said plurality of fetch requests to proceed while actively delaying action among remaining requests of said plurality of requests,

causing a remote fetch controller to wait until said one request which was permitted to proceed has completed, and

then proceeding with a delayed request among said remaining requests of said plurality of requests,

whereby a remote fetch controller processing a fetch request on behalf of a local memory controller utilizes a miss response in place of a reject response which permits the operation to complete without the need to recycle the operation back to the initiating processor.

5. A method for deadlock avoidance in a clustered symmetric multiprocessor system having a plurality of clusters interfaced with one another, each cluster

having:

(a) a local fetch interface controller, and

(b) a local store interface controller, and

(c) a remote fetch controller, and

(d) a remote storage controller, and

(e) a local-to-remote data bus and

(f) an interface controller,

(g) a plurality of processors,

(h) a shared cache memory,

(i) a plurality of I/O adapters, and

(j) a main memory accessible from the cluster,

said method comprising the steps of;

detecting a plurality of fetch requests between a local and a remote cluster,

responding to the detected fetch requests to permit at least one request of said plurality of fetch requests to proceed while actively delaying action among remaining requests of said plurality of requests,

causing a remote fetch controller to wait until said one request which was permitted to proceed has completed, and

then proceeding with a delayed request among said remaining requests of said plurality of requests,

and further comprising a fast hang quiesce mechanism embedded in the remote fetch and store controllers preventing deadlocks by detecting system hangs and causing the controllers to reject their current operations in an effort to permit the system operation to complete.

6. A method for deadlock avoidance in a clustered symmetric multiprocessor system having a plurality of clusters interfaced with one another, each cluster

having:

(a) a local fetch interface controller, and

(b) a local store interface controller, and

(c) a remote fetch controller, and

(d) a remote storage controller, and

(e) a local-to-remote data bus and

(f) an interface controller,

(g) a plurality of processors,

(h) a shared cache memory,

(i) a plurality of I/O adapters, and

(j) a main memory accessible from the cluster,

said method comprising the steps of;

detecting a plurality of fetch requests between a local and a remote cluster,

responding to the detected fetch requests to permit at least one request of said plurality of fetch requests to proceed while actively delaying action among remaining requests of said plurality of requests,

causing a remote fetch controller to wait until said one request which was permitted to proceed has completed, and

then proceeding with a delayed request among said remaining requests of said plurality of requests,

and further comprising a fast hang quiesce mechanism embedded in the remote fetch and store controllers preventing deadlocks caused by the controllers'"'"' own operations by detecting an internally generated hang period and using this hang period to signal the other controllers to quiesce their pending operations to permit the current remote fetch and/or store operation to complete.