Multiprocessor system with distributed shared memory having hot plug function for main memories

• US 6,553,465 B1
• Filed: 01/28/2000
• Issued: 04/22/2003
• Est. Priority Date: 01/28/1999
• Status: Expired due to Fees

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Takusagawa, Junichi
• Primary Examiner(s): Portka, Gary J
• Application Number:
  US09/493,241
• Time in Patent Office: 1,180 Days
• Field of Search: 711/115, 711/148, 711/152, 711/163, 711/165, 711/170, 711/162, 711/155, 712/31, 714/6, 714/7, 707/10
• US Class Current: 711/152
• CPC Class Codes: G06F 11/1666 where the redundant component is memory or memory area
• CPC Class Codes: G06F 11/2043 where the redundant components share a common memory address space
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 15/17 using an input/output type connection, e.g. channel, I/O port

Abstract

A multiprocessor system of a distributed shared memory structure has a hot plug function for main memories. Each of nodes of the multiprocessor system has a processor, an IO unit, a main memory, a mover, and a routing control unit. If a memory read access request is issued from the processor, the IO unit, or the mover to the main memory of a master node, the routing control unit instructs the master node to transfer the memory read access request. If a memory write access request is issued from the processor, the IO unit, or the mover to the main memory of the master node, the routing control unit instructs both the master node and a slave node to transfer the memory write access request when in a multicasting mode, and instructs only the master node to transfer the memory write access request when not in the multicasting mode.

Claims

1. A multiprocessor system of a distributed shared memory structure having a plurality of nodes each comprising at least one processor and a main memory, comprising:

means for managing the same memory addresses in two nodes;

means for holding information of the two nodes as a master node and a slave node;

means for transferring a read access request from one of a processor and an I/O unit with respect to said main memory to a master node;

a flag indicative of whether a write access request from one of the processor and the I/O unit with respect to said main memory is to be transferred to the master node or to the master node and the slave node;

means for transferring the write access request to the master node if said flag indicates that the write access request is to be transferred to the master node;

means for transferring the write access request to the master node and the slave node if said flag indicates that the write access request is to be transferred to the master node and the slave node;

means for holding address information of a memory space of the main memory of each of the nodes;

lock read access means for reading data from the main memory of the master node based on said address information in response to a locked read access request;

unlock write access means for writing the data read by said lock read access means into the main memories of the master node and the slave node in response to an unlocked write access request;

memory copying means for inseparably executing said lock read access request and said unlocked write access request; and

means for effecting memory copying by said memory copying means on the memory space of the main memory of said master node.

2. A multiprocessing system according to claim 1, further comprising:

a lock address buffer for inseparably executing a read access request and a write access request for said memory copying;

means for registering an address of said locked read access request in said lock address buffer in response to said locked read access request;

means for deleting the same address as said unlocked write access request from said lock address buffer in response to said unlocked write access request; and

means for inhibiting said read access request and said write access request until the memory address of a memory access request is deleted from said lock address buffer if the memory address of the memory access request sent from said processor or said IO unit to said main memory has already been registered in said lock address buffer.

3. A multiprocessor system of a distributed shared memory structure having a plurality of nodes each comprising at least one processor, an IO unit, a main memory, and a system control unit, said system control unit comprising a routing control unit, a memory control unit, an inter-node interface control unit, a lock control unit, and a mover;

said routing control unit having means for being supplied with a memory access request issued from said processor, said IO unit, or said mover, determining which node the memory access request is destined for the main memory in, adding routing information to the memory access request, and transferring the memory access request to said inter-node interface control unit;

said inter-node interface control unit having means for, if the memory access request transferred from said routing control unit is destined for the main memory of another node, issuing a memory access request to said other node via a system bus, and, if a memory access request other than a locked or unlocked memory access request received from said routing control unit or the system bus is destined for the main memory of its own node, transferring a lock address buffer index request to said lock control unit, and, if said lock control unit indicates that a transferred address is not locked, transferring the memory access request via said memory control unit to said main memory, and, if said lock control unit indicates that the transferred address is locked, holding the memory access request and transferring a memory access request address again to said lock control unit, and repeating the above process until the memory access request address is unlocked, and means for, if the memory access request transferred from said routing control unit or the system bus is a locked memory read request destined for its own node, transferring a lock request to said lock control unit, if an indication of locking success is received from said lock control unit, transferring the memory read request to said memory control unit, if an indication of locking failure is received from said lock control unit, eliminating the locked memory read request, and if the memory access request transferred from said routing control unit or the system bus is an unlocked memory write request destined for its own node, transferring the memory write request to said memory control unit, and transferring an unlock request to said lock control unit;

said lock control unit having means for determining whether the transferred address is locked or not when said lock address buffer index request is transferred, and indicating the determined result to said inter-node interface control unit, and, if the address of said lock request is unlocked, locking said address and sending an indication of locking success to said inter-node interface control unit, and, if the address of said lock request is locked, sending an indication of locking failure to said.inter-node interface control unit and a lock bus, and unlocking the address of said unlock request when said unlock request is received.

4. A multiprocessor system according to claim 3, wherein said routing control unit comprises:

a first register for holding write data when the memory write request is received and outputting the write data to said inter-node interface control unit;

a second register for holding the type of the memory access request and outputting the type of the memory access request to said inter-node interface control unit;

a third register for holding the memory address of the memory access request and outputting the memory address to said inter-node interface control unit;

a flag for indicating the transfer of the memory write request to a plurality of nodes;

a fourth register for holding master node information;

a fifth register for holding slave node information;

sixth and seventh registers for holding lower and upper limit values, respectively, of the address space of the main memory of each of said nodes;

first combining circuits associated respectively with said sixth and seventh registers, for comparing the memory address held by said third register with the lower and upper limit values of the address space which are held respectively by said sixth and seventh registers, and making respective output signals active if the memory address is between said lower and upper limit values of the address space; and

a second combining circuit for outputting the node corresponding to the active output signal of the output signals from said first combining circuits, as routing information indicative of a routing destination to said inter-node interface control unit if the transfer of the memory write request to a plurality of nodes is not indicated, and adding a slave node held by said fifth register as a routing destination and outputting the routing destination as routing information to said inter-node interface control unit if the transfer of the memory write request to a plurality of nodes is indicated, a node corresponding to the active output signal from said first combining circuits is a master node held by said fourth register, and the memory access request held by said second register is a memory write request.

5. A multiprocessor system according to claim 4, wherein said lock control unit comprises:

a lock address buffer for holding a locked main memory address;

a comparing circuit for comparing the data held by said lock address buffer with the memory addresses of a lock request, an unlock request, and a lock address buffer index request transferred from said inter-node interface control unit to determine whether they agree with each other; and

a lock control circuit for receiving a lock address buffer access request from said inter-node interface control unit, and, if the compared result from said comparing circuit indicates that the same address as the memory access request has already been registered in said lock address buffer, indicating that the transferred address is locked to said inter-node interface control unit, and, if the compared result from said comparing circuit indicates that the same address as the memory access request has not been registered in said lock address buffer, indicating that the transferred address is not locked to said inter-node interface control unit, and, if the compared result from said comparing circuit indicates that the same address as the locked memory read request has already been registered in said lock address buffer when the lock request is transferred from said inter-node interface control unit, issuing an indication of locking failure to said inter-node interface control unit and the lock bus, and, if the compared result from said comparing circuit indicates that the same address as the locked memory read request has not been registered in said lock address buffer, issuing an indication of locking success to said inter-node interface control unit and the lock bus, and instructing said lock address buffer to register the address of the lock request, and instructing said lock address buffer to delete an entry which agrees with the address of the unlock request among those addresses which are registered in the lock address buffer, in response to the compared result from said comparing circuit when the unlock request is transferred.

6. A multiprocessor system according to claim 4, wherein said mover comprises:

a flag for indicating the activation of the mover;

a first register for indicating a node to which the mover carries out memory copying;

a first selector for being supplied with an address held by said seventh register of said routing control unit;

a second selector for being supplied with an address held by said sixth register of said routing control unit;

a second register for holding a memory address of a memory access request outputted from said second selector and issued to said routing control unit;

an adder for adding a particular value to the memory address outputted from said second register and entering the sum into said selector;

a comparator for comparing an output from said first selector and the memory address outputted from said second register with each other; and

a mover control circuit for, if said flag indicates the activation of the mover, controlling said second selector to set the start address of a memory access request in said second register based on node information indicated by said first register, and controlling said first selector to enter the end address of the memory access request into said comparator, issuing a locked memory read request to said routing control unit based on the address entered from said second register, and, if an indication of locking failure is returned for the locked memory read request, issuing a locked memory read request again to said routing control unit using the same address, and, if reply data is returned for the locked memory read request, issuing an unlocked memory write request to the same address, and, when the unlocked memory write request is completed, causing said second selector to select an address counted up by said adder and entering the selected address into said second register, and, while counting up the address, repeatedly issuing a locked memory read request and an unlocked memory write request to said routing control unit, and, if said comparator detects when the address of said second register exceeds the address from said first selector, resetting said flag, and finishing the issuance of the memory access request to said routing control unit.

7. A multiprocessor system according to claim 5, wherein said mover comprises:

a flag for indicating the activation of the mover;

a first register for indicating a node to which the mover carries out memory copying;

a first selector for being supplied with an address held by said seventh register of said routing control unit;

a second selector for being supplied with an address held by said sixth register of said routing control unit;

a second register for holding a memory address of a memory access request outputted from said second selector and issued to said routing control unit;

an adder for adding a particular value to the memory address outputted from said second register and entering the sum into said selector;

a comparator for comparing an output from said first selector and the memory address outputted from said second register with each other; and

a mover control circuit for, if said flag indicates the activation of the mover, controlling said second selector to set the start address of a memory access request in said second register based on node information indicated by said first register, and controlling said first selector to enter the end address of the memory access request into said comparator, issuing a locked memory read request to said routing control unit based on the address entered from said second register, and, if an indication of locking failure is returned for the locked memory read request, issuing a locked memory read request again to said routing control unit using the same address, and, if reply data is returned for the locked memory read request, issuing an unlocked memory write request to the same address, and, when the unlocked memory write request is completed, causing said second selector to select an address counted up by said adder and entering the selected address into said second register, and, while counting up the address, repeatedly issuing a locked memory read request and an unlocked memory write request to said routing control unit, and, if said comparator detects when the address of said second register exceeds the address from said first selector, resetting said flag, and finishing the issuance of the