Multi-node cluster computer system incorporating an external coherency unit at each node to insure integrity of information stored in a shared, distributed memory

• US 5,394,555 A
• Filed: 12/23/1992
• Issued: 02/28/1995
• Est. Priority Date: 12/23/1992
• Status: Expired due to Term

• Current Assignee: Bull HN Information Systems Incorporated (Atos SE)
• Original Assignee: Bull HN Information Systems Incorporated (Atos SE)
• Inventors: Wertz, John A., Hunter, John C.
• Primary Examiner(s): Bowler, Alyssa H.
• Assistant Examiner(s): HARRITY, JOHN E
• Application Number:
  US07/993,884
• Time in Patent Office: 797 Days
• Field of Search: 395/800, 395/325, 395/425
• US Class Current: 711/148
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration

Abstract

A computer cluster architecture including a plurality of CPUs at each of a plurality of nodes. Each CPU has the property of coherency and includes a primary cache. A local bus at each node couples: all the local caches, a local main memory having physical space assignable as-shared space and non-shared space and a local external coherency unit (ECU). An inter-node communication bus couples all the ECUs. Each ECU includes a monitoring section for monitoring the local and inter-node busses and a coherency section for a) responding to a non-shared cache-line request appearing on the local bus by directing the request to the non-shared space of the local memory and b) responding to a shared cache-line request appearing on the local bus by examining its coherence state to further determine if inter-node action is required to service the request and, if such action is required, transmitting a unique identifier and a coherency command to all the other ECUs. Each unit of information present in the shared space of the local memory is assigned, by the local ECU, a coherency state which may be: exclusive (the local copy of the requested information is unique in the cluster); 2) modified (the local copy has been updated by a CPU in the same node); 3) invalid (a local copy either does not exist or is known to be out-of-date); or 4) shared (the local copy is one of a plurality of current copies present in a plurality of nodes).

Claims

1. A computer cluster architecture comprising:

A) a plurality of nodes;

B) at least one central processing unit resident at each said node, at least one of said central processing units including at least one primary cache, each said central processing unit having a property of coherency;

C) a node-local communications means resident at each said node;

each said node-local communications means coupling all said primary caches resident in the same node with said node-local communications means;

D) a local main memory resident at each said node, each said local main memory having physical space assignable as shared physical space and non-shared physical space;

E) an external coherency unit resident at each said node, each said external coherency unit being coupled to the node-local communications means and to the local main memory resident in the same node with said external coherency unit;

F) inter-node communication means coupling all said external coherency units;

G) each said external coherency unit comprising;

1) monitoring means adapted to monitor both said inter-node communication means and said node-local communications means resident in the same node with said external coherency unit; and

2) coherency means adapted to;

a) respond to said monitoring means sensing a cache-line request appearing on said node-local communications means in the same node with said external coherency unit and determining that such cache-line request is non-shared by directing said non-shared cache-line request to said non-shared physical space of said local main memory; and

b) respond to said monitoring means sensing a cache-line request appearing on said node-local communications means in the same node with said external coherency unit and determining that such cache-line request is shared by examining its coherence state to further determine if inter-node action is required to service the request and, if such inter-node action is required, transmitting a unique identifier and a coherency command over said inter-node communication means to all other said external coherency units;

(H) each said external coherency unit assigning a coherency state to each unit of information stored in the shared memory spaces of said cluster, said coherency states comprising;

1) exclusive indicating that a copy of the requested information present in the shared memory space of said local main memory resident in the same node as said external coherency unit is unique in the cluster;

2) modified indicating that a copy of the requested information present in said local main memory resident in the same node as said external coherency unit has been updated by a central processing unit in said same node;

3) invalid indicating that a copy of the requested information present in said local main memory resident in the same node as said external coherency unit either does not exist or is known to be out-of-date; and

4) shared indicating that a copy of the requested information present in said local main memory resident in the same node as said external coherency unit is one of a plurality of current copies of the requested information in a plurality of nodes; and

I) each said coherency means in each said external coherency unit responding to a local cache-line request associated with a block of shared information as follows;

1) if the requested cache-line state is exclusive or modified, the requested block is retrieved directly from the local main memory resident in the requesting node with no remote action taken because no current copies exist elsewhere in the cluster;

2) if the requested cache-line state is invalid, a read-with-intent-to-write command is transmitted over said communications means to all other said nodes, and a remote node that contains the requested cache-line information in the exclusive or modified state responds thereto by siphoning the requested cache-line information over said communications means to the requesting node; and

3) if the requested cache-line state is shared, an invalidating command is sent to all other nodes such that the requested cache-line state is set to invalid in such all other nodes;

whereby, the most recent instantiation of the requested data available in the shared physical spaces among all said local memories in the cluster are provided to the one of said external coherency units requesting the same.

2. The computer architecture of claim 1 in which said inter-node communications means comprises an inter-node external coherency unit bus.

3. The computer architecture of claim 1 in which said inter-node communications means comprises direct coupling means.