Multi-node system in which home memory subsystem stores global to local address translation information for replicating nodes

• US 20050005074A1
• Filed: 04/02/2004
• Published: 01/06/2005
• Est. Priority Date: 04/04/2003
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Hagersten, Erik E., Landin, Anders
• Granted Patent: US 7,765,381 B2
• Time in Patent Office: Days
• US Class Current: 711/148
• CPC Class Codes: G06F 12/0815 Cache consistency protocols
• CPC Class Codes: G06F 12/0817 using directory methods
• CPC Class Codes: G06F 12/1072 Decentralised address translation, e.g. in distributed shared memory systems

Abstract

A system may include a plurality of nodes. Each node may include an active device and a memory subsystem coupled to the active device. An active device in one of the nodes is configured to generate a global address that identifies a coherency unit and associated translation information identifying a translation function to be performed on the global address. A memory subsystem included in the node is configured to perform the translation function identified by the translation information on the global address to generate a physical address of the coherency unit within the memory subsystem. An additional memory subsystem included in an additional one of the nodes is configured to store the translation information identifying the translation function used in the node. In response to a request for access to the coherency unit, the additional memory subsystem is configured to send the translation information to the node.

Claims

1. A system, comprising:

a plurality of nodes, wherein each node includes an active device and a memory subsystem coupled to the active device;

wherein an active device in a node of the plurality of nodes is configured to generate a global address and translation information identifying a translation function, wherein the global address identifies a coherency unit;

wherein a memory subsystem included in the node is configured to perform the translation function identified by the translation information on the global address to generate a physical address of the coherency unit within the memory subsystem;

wherein an additional memory subsystem included in an additional node of the plurality of nodes is configured to store the translation information identifying the translation function used in the node, wherein in response to a request for access to the coherency unit, the additional memory subsystem is configured to send the translation information to the node.

2. The system of claim 1, wherein the plurality of nodes are coupled by an inter-node network, and wherein each of the plurality of nodes includes an interface to the inter-node network;

wherein an additional interface included in the additional node is configured to receive the translation information for the coherency unit from the additional memory subsystem;

wherein the additional interface is configured to provide the translation information and the global address to an interface included in the node via the inter-node network; and

wherein the interface included in the node is configured to provide the translation information and the global address received via the inter-node network to the memory subsystem.

3. The system of claim 1, wherein the additional memory subsystem is configured to perform a different translation function on the global address to obtain a local physical address of the coherency unit within the additional memory subsystem.

4. The system of claim 1, wherein the additional memory subsystem is configured to store translation information associated with the coherency unit for several other nodes included in the plurality of nodes, wherein different translation information is associated with each of the several other nodes.

5. The system of claim 1, wherein each active device included in the plurality of nodes is configured to use at least a portion of the global address of the coherency unit to determine whether a copy of the coherency unit is locally cached by that active device.

6. The system of claim 1, wherein an additional active device in the additional node is configured to initiate a coherency transaction to gain access to the coherency unit by sending the request for access to the coherency unit to the additional memory subsystem, wherein the request for access includes the global address and additional translation information, and wherein the additional translation information is associated with the coherency unit in the additional node;

wherein the additional memory subsystem is configured to send an packet indicating the coherency transaction to an additional interface included in the additional node, wherein the packet includes the global address and the translation information for the node,

wherein in response to the packet, the additional interface is configured to communicate the global address and the translation information to an interface included in the node.

7. The system of claim 1, wherein no memory subsystem included in an other node of the plurality of nodes maps the global address, wherein an other active device included in the other node is configured to request access to the coherency unit by sending a packet including the global address and translation information associated with the coherency unit in the other node on a network included in the other node, wherein the translation information associated with the coherency unit in the other node indicates that no memory subsystem included in the other node maps the coherency unit.

8. The system of claim 7, wherein an other interface included in the other node and coupled to the network is configured to forward the global address to an additional interface included in the additional node in response to receiving the packet.

9. The system of claim 1, wherein a memory controller included in the memory subsystem is integrated in a same integrated circuit as the processing subsystem.

10. A method for use in a system comprising a plurality of nodes, wherein each node includes an active device and a memory subsystem coupled to the active device, the method comprising:

an active device in a node of the plurality of nodes generating a global address and translation information identifying a translation function, wherein the global address identifies a coherency unit;

a memory subsystem included in the node performing the translation function identified by the translation information on the global address to generate a physical address of the coherency unit within the memory subsystem;

an additional memory subsystem included in an additional node of the plurality of nodes storing the translation information identifying the translation function used in the node; and

in response to a request for access to the coherency unit, the additional memory subsystem sending the translation information to the node.

11. The method of claim 10, further comprising:

an additional interface included in the additional node receiving the translation information for the coherency unit in the node from the additional memory subsystem;

the additional interface providing the translation information and the global address to an interface included in the node via an inter-node network; and

the interface included in the node providing the translation information and the global address received via the inter-node network to the memory subsystem.

12. The method of claim 10, further comprising the additional memory subsystem performing a different translation function on the global address to obtain a local physical address of the coherency unit within the additional memory subsystem.

13. The method of claim 10, further comprising the additional memory subsystem storing translation information associated with the coherency unit for several other nodes included in the plurality of nodes, wherein different translation information is associated with each of the several other nodes.

14. The method of claim 10, further comprising each active device included in the plurality of nodes using at least a portion of the global address of the coherency unit to determine whether a copy of the coherency unit is locally cached by that active device.

15. The method of claim 10, further comprising an additional active device in the additional node initiating a coherency transaction to gain access to the coherency unit by sending the request for access to the coherency unit to the additional memory subsystem, wherein the request for access includes the global address and additional translation information, and wherein the additional translation information is associated with the coherency unit in the additional node;

wherein in response to the request for access, the additional memory subsystem sends an packet indicating the coherency transaction to an additional interface included in the additional node, wherein the packet includes the global address and the translation information for the node,

wherein in response to the packet, the additional interface communicates the global address and the translation information to an interface included in the node.

16. The method of claim 10, further comprising an other active device included in an other node of the plurality of nodes requesting access to the coherency unit by sending a packet including the global address and translation information associated with the coherency unit in the other node on a network included in the other node, wherein no memory subsystem included in the other node maps the global address, and wherein the translation information associated with the coherency unit in the other node indicates that no memory subsystem included in the other node maps the coherency unit.

17. The method of claim 16, further comprising an other interface included in the other node and coupled to the network is configured to forward the global address to an additional interface included in the additional node in response to receiving the packet.

18. The method of claim 10, wherein a memory controller included in the memory subsystem is integrated in a same integrated circuit as the processing subsystem.

19. The method of claim 10, further comprising an operating system executing on the active device creating a translation lookaside buffer entry corresponding to the virtual address, wherein the translation lookaside buffer entry includes the global address and the information identifying the translation function, wherein the operating system selects the translation function in order to map the global address to the local physical address within the memory.

20. The method of claim 10, further comprising an operating system executing on the active device in the node creating a translation lookaside buffer entry corresponding to a virtual address in response to deciding to replicate the coherency unit to the node from the additional node, wherein the translation lookaside buffer entry corresponding to the virtual address specifies the global address and the information identifying the translation function.