Remote translation mechanism for a multinode system

• US 20050044340A1
• Filed: 08/18/2003
• Published: 02/24/2005
• Est. Priority Date: 08/18/2003
• Status: Active Grant

• Current Assignee: CRAY Incorporated (Hewlett-Packard Enterprise Company)
• Original Assignee: CRAY Incorporated (Hewlett-Packard Enterprise Company)
• Inventors: Hastings, Andrew B., Sheets, Kitrick
• Granted Patent: US 7,577,816 B2
• Time in Patent Office: Days
• US Class Current: 711/206
• CPC Class Codes: G06F 12/1036 for multiple virtual address spaces, e.g. segmentation G06F12/1045 takes precedence
• CPC Class Codes: G06F 12/1072 Decentralised address translation, e.g. in distributed shared memory systems

Abstract

The present invention provides a method of initializing shared memory in a multinode system. The method includes building a local address space in each of a plurality of nodes and exporting the local address space from each of the plurality of nodes to a Remote Translation Table (RTT) in each of the plurality of nodes. The present invention further provides system including a plurality of nodes, each node having one or more processors and a memory controller operatively coupled to the one or more processors, wherein the memory controller includes a RTT for holding translation information for an entire virtual memory address space for the node, further wherein the RTT is initialized upon the start of a process by building a local address space in the node, and exporting the local address space from the node to a RTT in each of the plurality of other nodes.

Claims

1. A method of accessing shared memory in a computer system having a plurality of nodes, including a first node, wherein each node includes a processor and local memory, the method comprising:

distributing an application across the plurality of nodes;

building an application virtual address space, wherein building an application virtual address space includes;

building a local virtual address space for the application in each of the plurality of nodes, wherein the local virtual address space translates a virtual address generated by the application executing on that node to a physical address in local memory for that node; and

exporting the local virtual address space for each node to a Remote Translation Table (RTT) associated with that node; and

performing a memory reference to a memory location in the application virtual address space, wherein performing a memory reference to a memory location in the application virtual address space includes translating bits of the application virtual address into a node address associated with the first node and translating bits of the application virtual address using the RTT associated with the first node.

2. The method of claim 1, wherein the local address space is read from a Translation Look-aside Buffer (TLB).

3. The method of claim 1, wherein building an application virtual address space further includes performing a synchronization operation that causes at least some of the plurality of nodes to wait for all nodes to complete exporting their respective local virtual address spaces.

4. A system comprising:

a plurality of nodes, each node including;

one or more processors;

a memory; and

a memory controller operatively coupled to the memory and the one or more processors, wherein the memory controller includes a Remote Translation Table (RTT), wherein the RTT translates a virtual address received as part of a memory request received from another node into a memory request with physical addresses into the memory on the node associated with the RTT;

further wherein the RTT is initialized upon the start of a process associated with an application by building virtual to physical address translations for local virtual address space in the node corresponding to the application, and exporting the virtual to physical address translations for the local virtual address space from the node to the Remote Translation Table (RTT) associated with that node.

5. The system of claim 4, wherein each of the plurality of nodes executes a synchronization operation that causes at least some of the plurality of nodes to wait for all of the plurality of nodes to complete exporting the virtual to physical address translations to their respective Remote Translation Tables.

6. A device-readable medium having instructions thereon that, when executed on a properly programmed information-processing device having a plurality of nodes, including a first node, each node having one or more processors, a memory, and a memory controller and coupled to the memory and the one or more processors, causes the information-processing device to perform a method comprising:

distributing an application across the plurality of nodes;

building an application virtual address space, wherein building an application virtual address space includes;

building a local virtual address space for the application in each of the plurality of nodes, wherein the local virtual address space translates a virtual address generated by the application executing on that node to a physical address in local memory for that node; and

exporting the local virtual address space for each node to a Remote Translation Table (RTT) associated with that node; and

performing a memory reference to a memory location in the application virtual address space, wherein performing a memory reference to a memory location in the application virtual address space includes translating bits of the application virtual address into a node address associated with the first node and translating bits of the application virtual address using the RTT associated with the first node.

7. The device-readable medium of claim 6, wherein building a local virtual address space further includes performing a synchronization operation that causes at least some of the plurality of nodes to wait for all nodes complete exporting their respective address space.

8. The device-readable medium of claim 6, wherein the local address space is read from a Translation Look-aside Buffer (TLB).

9. A multinode system for implementing remote address translation, the system comprising:

a plurality of nodes, including a first node, each of the plurality of nodes including;

one or more processors,

a memory, and

a memory controller operatively coupled to the memory and the one or more processors;

means for distributing an application across the plurality of nodes;

means for building an application virtual address space, wherein the means for building an application virtual address space includes;

means for building a local virtual address space for the application in each of the plurality of nodes, wherein the local virtual address space translates a virtual address generated by the application executing on that node to a physical address in local memory for that node; and

means for exporting the local virtual address space for each node to a Remote Translation Table (RTT) associated with that node; and

means for performing a memory reference to a memory location in the application virtual address space, wherein performing a memory reference to a memory location in the application virtual address space includes;

means for translating bits of the application virtual address into a node address associated with the first node, and means for translating bits of the application virtual address using the RTT associated with the first node.

10. The multinode system of claim 9, wherein building an application virtual address space further includes a means for performing a synchronization operation that causes at least some of the plurality of nodes to wait for all nodes to complete exporting their respective local virtual address spaces.

11. A multi-node system for implementing remote address translation, the system comprising:

a network;

a source node coupled to the network, wherein the source node includes a first remote-translation table (RTT);

a remote node coupled to the network, wherein the remote node includes a second RTT;

wherein on the remote node the second RTT is built using a first local address space on the source node exported from the source node to the remote node; and

wherein on the source node the first RTT is built using a second local address space on the remote node exported from the remote node to the source node.