Non-uniform memory access (NUMA) data processing system having remote memory cache incorporated within system memory

• US 6,760,809 B2
• Filed: 06/21/2001
• Issued: 07/06/2004
• Est. Priority Date: 06/21/2001
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fields, James Stephen Jr., Arimilli, Ravi Kumar, Dodson, John Steven
• Primary Examiner(s): Padmanabhan, Mano
• Assistant Examiner(s): BAKER, PAUL A
• Application Number:
  US09/885,992
• Publication Number:
  US 20030009623A1
• Time in Patent Office: 1,111 Days
• Field of Search: 711/119, 711/122
• US Class Current: 711/119
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0817 using directory methods
• CPC Class Codes: G06F 12/0831 using a bus scheme, e.g. with bus monitoring or watching means

Abstract

A non-uniform memory access (NUMA) computer system and associated method of operation are disclosed. The NUMA computer system includes at least a remote node and a home node coupled to an interconnect. The remote node contains at least one processing unit coupled to a remote system memory, and the home node contains at least a home system memory. To reduce access latency for data from other nodes, a portion of the remote system memory is allocated as a remote memory cache containing data corresponding to data resident in the home system memory. In one embodiment, access bandwidth to the remote memory cache is increased by distributing the remote memory cache across multiple system memories in the remote node.

Claims

1. A non-uniform memory access (NUMA) computer system, comprising:

a node interconnect; and

a first node and a second node coupled to said node interconnect for communication of data therebetween, wherein;

said first node includes a first system memory at a lowest level of volatile memory within said first node; and

said second node includes at least one processing unit coupled to a second memory hierarchy within said second node, said second memory hierarchy including one or more upper levels of volatile memory and a second system memory at a lowest level of volatile memory within said second node, wherein said second system memory includes;

a first set of storage locations allocated as a remote memory cache for storing data corresponding to data resident in said first system memory for which the first node is the home node;

a remote memory directory specifying contents of said remote memory cache; and

a second set of storage locations allocated for storage of data for which the second node is the home node.

2. The NUMA computer system of claim 1, said second node including a plurality of processing units and a plurality of second system memories, wherein said remote memory cache is distributed among said plurality of second system memories.

3. The NUMA computer system of claim 1, wherein said second system memory comprises one or more dynamic random access memory (DRAM) chips.

4. The NUMA computer system of claim 1, said at least one processing unit including a cache controller for said remote memory cache.

5. The NUMA computer system of claim 1, wherein said one or more upper levels of volatile memory comprise a cache hierarchy and wherein said remote memory cache is inclusive of data from said first node held in said cache hierarchy.

6. A second node for a non-uniform memory access (NUMA) computer system including first node coupled to the second node by a node interconnect, said second node comprising:

a local interconnect;

a node controller coupled to said local interconnect and the node interconnect;

at least one processing unit coupled to said local interconnect; and

a remote memory hierarchy coupled to said at least one processing unit, wherein said remote memory hierarchy includes one or more upper levels of volatile memory and a remote system memory at a lowest level of volatile memory within said remote node, wherein said remote system memory includes;

a first set of storage locations allocated as a remote memory cache for storing data corresponding to data resident in a home system memory in the first node for which the first node is the home node;

a remote memory directory specifying contents of said remote memory cache; and

a second set of storage locations allocated for storage of data for which the second node is the home node.

7. The second node of claim 6, said second node including a plurality of processing units and a plurality of remote system memories, wherein said remote memory cache is distributed among said plurality of remote system memories.

8. The second node of claim 6, wherein said remote system memory comprises one or more dynamic random access memory (DRAM) chips.

9. The second node of claim 6, said at least one processing unit including a cache controller for said remote memory cache.

10. The second node of claim 6, wherein said one or more upper levels of volatile memory comprises a cache hierarchy and wherein said remote memory cache is inclusive of data from said first node held in said cache hierarchy.

11. A method of operating a non-uniform memory access (NUMA) computer system including a first node and a second node coupled by a node interconnect, said method comprising:

in a second system memory in said second node, allocating a first set of storage locations for storing data for which the second node is a home node and allocating a second set of storage locations as a remote memory cache for storing data corresponding to data resident in a first system memory in said first node for which said first node is the home node

creating within said second system memory a remote memory directory specifying contents of said remote memory cache; and

in response to a request by a processing unit within said second node that specifies an address associated with a storage location in said first system memory, servicing said request by accessing said remote memory in said second system memory.

12. The method of claim 11, wherein said second node includes a plurality of processing units and a plurality of second system memories, and wherein allocating a remote memory cache comprises creating a remote memory cache distributed among said plurality of second system memories.

13. The method of claim 11, wherein allocating a remote memory cache in said second system memory comprises creating said remote memory cache within one or more dynamic random access memory (DRAM) chips.

14. The method of claim 11, wherein said accessing step comprises accessing said remote memory caches utilizing a remote memory cache controller within said at least one processing unit.

15. The method of claim 11, wherein said one or more upper levels of memory comprise a cache hierarchy and wherein allocating said remote memory cache comprises allocating a remote memory cache that is inclusive of data from said first node held in said cache hierarchy.