Shared memory multiprocessor system and method with address translation between partitions and resetting of nodes included in other partitions

• US 6,510,496 B1
• Filed: 02/11/2000
• Issued: 01/21/2003
• Est. Priority Date: 02/16/1999
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Kawamoto, Shinichi, Tarui, Toshiaki, Okochi, Toshio
• Primary Examiner(s): Gossage, Glenn
• Application Number:
  US09/501,978
• Time in Patent Office: 1,075 Days
• Field of Search: 711/147, 711/153, 711/202, 711/173, 711/141, 711/170, 711/206, 711/146, 711/209, 709/213, 709/215
• US Class Current: 711/147
• CPC Class Codes: G06F 12/0284 Multiple user address space allocation, e.g. using different base addresses interprocessor communication G06F15/163
• CPC Class Codes: G06F 12/0831 using a bus scheme, e.g. with bus monitoring or watching means
• CPC Class Codes: G06F 12/1072 Decentralised address translation, e.g. in distributed shared memory systems
• CPC Class Codes: G06F 2212/656 Address space sharing

Abstract

A symmetric multiprocessor (SMP) of hierarchical connection realizing an inter-partition shared memory has at the gateway of an inter-node connection switch from each node, a translator for translating an address of an access command for an area shared between partitions, between a real address used in a partition and a shared area address used in common between partitions. Thereby, the address of a local area of each partition is freely set, and cache coherent control of a shared area is conducted at high speed by using a snoop command of the hierarchical connection SMP. Fault containment between partitions is realized by checking conformity between the address of the access command issued from another partition and the shared area configuration. Nodes included in other partitions may be reset from each partition. In addition, the configuration information of the shared area between partitions may be dynamically modified.

Claims

1. A shared memory multiprocessor system having a plurality of nodes and a network for interconnecting nodes, each of the plurality of nodes including at least one CPU, at least one cache, and at least one main memory, cache coherent control being conducted between nodes sharing a main memory via said network, said plurality of nodes being divided into a plurality of partitions, each of said partitions including at least one node, the partitions respectively locally sharing a main memory,

wherein a memory address of said system comprises a local real address and an address of common memory space, said local real address being local to each of said plurality of partitions, said address of common memory space being used in common in a memory space used in common between said plurality of partitions,

wherein a main memory of each node is accessed by using a local real address of a partition the node belongs to, and

wherein as for access from each CPU to a main memory of another node, data of a main memory of a node in its partition is accessed via said network by coherently using a local real address of the partition, and access to data of a main memory of a node included in another partition is conducted by translating an access address to an address of said shared memory space when an access command is issued to said network and translating an address of said shared memory space to a local real address of said another partition when said access command enters said node included in said another partition.

2. A shared memory multiprocessor system having a plurality of nodes and a network for interconnecting nodes, each of the plurality of nodes including at least one CPU, at least one cache, and at least one main memory, cache coherent control being conducted between nodes sharing a main memory via said network, said plurality of nodes being divided into a plurality of partitions each including at least one node locally sharing a main memory, a local real address local to each of said plurality of partitions is used as a memory address of the partition,

wherein said shared memory multiprocessor includes an address translation means provided at a gateway of each of said plurality of nodes to said network, said address translation means being used when a shared area shared by at least two of said partitions is set in any of said main memories, an address specified by a command for accessing said shared area is subjected in said address translation means to mutual translation between said local real address used in a partition and an address of the shared are used in said network, and

wherein a main memory of each node is accessed by using said local real address no matter whether it is said shared area.

3. A shared memory multiprocessor according to claim 2, wherein a command for accessing an area which is not shared between said partitions is exchanged between each node and the network for interconnecting nodes without conducting address translation.

4. A shared memory multiprocessor according to claim 2, wherein when sending out a snoop command from each of said plurality of nodes to the inter-node connection network, a command for accessing an area which is not shared between partitions is multicast to only nodes in the partition, and a command for accessing an area shared by partitions is broadcasted to all nodes belonging to at least partitions sharing the area.

5. A shared memory multiprocessor according to claim 2, wherein in said plurality of partitions, every node included in a partition having said shared area set in a main memory included in its partition has mapping means for mutually mapping between an address in a shared memory space and a local real address of the partition of said shared area.

6. A shared memory multiprocessor according to claim 5, wherein said mapping means conducts mapping in accordance with a table having mapping information set for each shared area, and as a result, a plurality of shared areas can be set as a whole.

7. A shared memory multiprocessor according to claim 2, wherein in said plurality of partitions, on a local real address space of a partition sharing a shared area set in a main memory of another partition, a window area for accessing a shared area set in the main memory of said another partition is provided, said window area does not have a main memory in its own partition, and every node in the partition having said window area has means for mutually mapping between a local real address of the own partition of said window area and said shared memory space address of said shared area.

8. A shared memory multiprocessor according to claim 7, wherein it is made possible for two or more partitions to share one shared area by permitting mapping between window areas respectively set in a plurality of partitions and a shared memory space address of one shared area.

9. A shared memory multiprocessor system having a plurality of nodes and a network for interconnecting nodes, each of the plurality of nodes including at least one CPU, at least one cache, and at least one main memory, cache coherent control being conducted between nodes sharing a main memory via said network,

wherein each of said plurality of nodes comprises information addition means, said information addition means being used when said plurality of nodes are divided into a plurality of partitions each including at least one node locally sharing a main memory and a shared area shared by two or more of said partitions, said shared area being provided in any of said main memories, and said information addition means adds information indicating whether a command sent out from a node to said network is a command for accessing said shared area or a command for accessing a local area in said partition which includes said node having said information addition means, to the command,

wherein said local area is addressed by a local address local to said partition and said shared area is addressed by a shared address of said shared area, and

wherein when said command is a command for accessing said shared area a shared address associated with said command is subjected to address translation to a local address to address a main memory in which said shared area is provided.

10. A shared memory multiprocessor system having a plurality of nodes and a network for interconnecting nodes, each of the plurality of nodes including at least one CPU, at least one cache, and at least one main memory, cache coherent control being conducted between nodes sharing a main memory via said network,

wherein each of said plurality of nodes comprises:

information addition means used when said plurality of nodes are divided into a plurality of partitions, each including at least one node locally sharing a main memory, and a shared memory area shared by two or more of said partitions is provided in any of said main memories, said information addition means adding information, indicating whether a command sent out from that node to said network is a command for accessing said shared area or a command for accessing a local area in said partitions, to the command, and when said command is a command for accessing said shared area, said information addition means translates an address specified by said command from a real address local to a partition including its own node to an address in the shared area and sends out a resultant address into said network, and

means responsive to added information of a command received from said network indicating that the command is a command accessing said shared area, for translating an address of the shared area specified by said command to a real address local to a partition including its own node,

wherein a main memory of each node is accessed by using the real address no matter whether that main memory is said shared area.

11. A shared memory multiprocessor system having a plurality of nodes and a network for interconnecting nodes, each of the plurality of nodes including at least one CPU, at least one cache, and at least one main memory, cache coherent control being conducted between nodes sharing a main memory via said network,

wherein each of said plurality of nodes comprises:

information addition means used when said plurality of nodes are divided into a plurality of partitions, each including at least one node locally sharing a main memory, and a shared area shared by two or more of said partitions is provided in any of said main memories, said information addition means adding information, indicating whether a command sent out from that node to said network is a command for accessing said shared area or a command for accessing a local area in said partitions, to the command,

storage means for setting nodes included in a partition a node belongs to, and

means responsive to a command received from said network being a command for accessing said local area, for checking whether the accessing node of the command is a node included in a partition that node belongs to, and responsive to a negative result, for suppressing access of said command,

wherein said local area is addressed by a local address local to said partition and said shared area is addressed by a shared address of said shared area, and

wherein when said command is a command for accessing said shared area a shared address associated with said command is subjected to address translation to a local address to address a main memory in which said shared area is provided.

12. A shared memory multiprocessor system having a plurality of nodes and a network for interconnecting nodes, each of the plurality of nodes including at least one CPU, at least one cache, and at least one main memory, cache coherent control being conducted between nodes sharing a main memory via said network, and when said plurality of nodes are divided into a plurality of partitions each including at least one node locally sharing a main memory, a command for locally accessing a main memory in each of said partitions and a command for accessing a shared area of a main memory shared by a plurality of partitions are used,

wherein each of said plurality of nodes comprises:

storage means responsive to determination of partitions sharing said shared area, for setting nodes included in said partitions, and

means responsive to a command received from said network being a command for accessing the local area, for checking whether the accessing node of said command is included in the nodes set in said storage means, and responsive to a negative result, for suppressing access of said command,

wherein said local area is addressed by a local address local to said partition and said shared area is addressed by a shared address of said shared area, and

wherein when said command is a command for accessing said shared area a shared address associated with said command is subjected to address translation to a local address to address a main memory in which said shared area is provided.

13. A shared memory multiprocessor system according to claim 12, wherein a plurality of said shared areas are set independently, and nodes of a set of partitions authorized to share each of said plurality of shared areas are independently set in said storage means.

14. In a shared memory multiprocessor system having a plurality of nodes and a network for interconnecting nodes, each of the plurality of nodes including at least one CPU, at least one cache, and at least one main memory, cache coherent control being conducted between nodes sharing a main memory via said network, and when said plurality of nodes are divided into a plurality of partitions each including at least one node locally sharing a main memory, a local real address local to each of said plurality of partitions is used as a memory address of the partition,

a shared area generation method of a shared memory multiprocessor comprising the steps of:

when generating a shared area shared by two or more partitions of said partitions, allocating an area used as a shared area to a main memory of a node in one partition included in partitions attempting to share the shared area, said one partition being own partition; and

defining address mapping to the allocated area and a shared area,

wherein a local area is addressed by a local real address local to said partition and said shared area is addressed by a shared address of said shared area, and

wherein when accessing said shared area a shared address associated with said accessing is subjected to address translation to a local real address to address a main memory in which said shared area is provided.

15. A shared area generation method according to claim 14, wherein the allocation of an area used as said shared area and the definition of said mapping are carried out by system software prepared in said one partition.

16. In a shared memory multiprocessor system having a plurality of nodes and a network for interconnecting nodes, each of the plurality of nodes including at least one CPU, at least one cache, and at least one main memory, cache coherent control being conducted between nodes sharing a main memory via said network, and when said plurality of nodes are divided into a plurality of partitions each including at least one node locally sharing a main memory, a local real address local to each of said plurality of partitions is used as a memory address of the partition,

a shared area generation method of a shared memory multiprocessor comprising the steps of:

when generating a shared area shared by two or more partitions in any of said partitions, allocating a window area in a partition in a local real address space in the partition, to partitions attempting to share the shared area, other than a partition having an area used as the shared area allocated in a main memory; and

defining address mapping of the allocated area and a shared area,

wherein a local area is addressed by a local real address local to said partition and said shared area is addressed by a shared address of said shared area, and

wherein when accessing said shared area a shared address associated with said accessing is subjected to address translation to a local real address to address a main memory in which said shared area is provided.

17. A shared area generation method according to claim 16, wherein the allocation of the window area and the definition of mapping are carried out respectively by system software pieces respectively prepared in partitions other than a partition having an area used as a shared area and allocated in a main memory.