Reservation management in a non-uniform memory access (NUMA) data processing system

• US 6,275,907 B1
• Filed: 11/02/1998
• Issued: 08/14/2001
• Est. Priority Date: 11/02/1998
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fields, James Stephen Jr., Dean, Mark Edward, Carpenter, Gary Dale, Glasco, David Brian, Baumgartner, Yoanna, Elman, Anna
• Primary Examiner(s): Nguyen, Hiep T.
• Application Number:
  US09/184,395
• Time in Patent Office: 1,016 Days
• Field of Search: 711/119, 711/120, 711/122, 711/143, 711/144, 711/145, 711/148, 711/153, 711/121, 711/156
• US Class Current: 711/143
• CPC Class Codes: G06F 12/0828 with concurrent directory accessing, i.e. handling multiple concurrent coherency transactions
• CPC Class Codes: G06F 9/3004 to perform operations on memory
• CPC Class Codes: G06F 9/30072 to perform conditional operations, e.g. using predicates or guards
• CPC Class Codes: G06F 9/30087 Synchronisation or serialisation instructions

Abstract

A non-uniform memory access (NUMA) computer system includes a plurality of processing nodes coupled to a node interconnect. The plurality of processing nodes include at least a remote processing node, which contains a processor having an associated cache hierarchy, and a home processing node. The home processing node includes a shared system memory containing a plurality of memory granules and a coherence directory that indicates possible coherence states of copies of memory granules among the plurality of memory granules that are stored within at least one processing node other than the home processing node. If the processor within the remote processing node has a reservation for a memory granule among the plurality of memory granules that is not resident within the associated cache hierarchy, the coherence directory associates the memory granule with a coherence state indicating that the reserved memory granule may possibly be held non-exclusively at the remote processing node. In this manner, the coherence mechanism can be utilized to manage processor reservations even in cases in which a reserving processor'"'"'s cache hierarchy does not hold a copy of the reserved memory granule.

Claims

1. A method of reservation management in a multiprocessor computer system including a remote processing node and a home processing node coupled to said node interconnect, wherein said remote processing node includes a processor having an associated cache hierarchy and said home processing node includes a shared system memory containing a plurality of memory granules and a coherence directory that indicates possible coherence states of remote copies of said plurality of mimory granules, said method comprising:

at said remote processing node, maintaining a reservation, for said processor, reserving a mimory granule among said plurality of memory granules that is not resident within said associated cache hierarchy; and

while said mimory granule is not resident within said associated cache hierarchy, setting a coherence indicator within said coherence directory of said home processing node to a state indicating that said reserved memory granule is possibly held non-exclusively at said remote processing node, such that cache coherence communication will reset said reservation if necessary.

2. The method of claim 1, and further comprising:

storing said memory granule in said associated cache hierarchy;

setting said reservation at said remote processing node;

setting said coherence indicator to a state indicating that said memory granule is held only at said remote processing node; and

thereafter, deallocating said memory granule from said associated cache hierarchy and writing back said memory granule to said shared system memory.

3. The method of claim 2, wherein writing back said memory granule comprises transmitting a writeback transaction from said remote processing node to said home processing node, wherein said writeback transaction indicates that said coherence indicator should be updated to shared state.

4. The method of claim 3, said remote processing node further including a node controller, wherein writing back said memory granule further comprises:

transmitting a writeback transaction to said node controller that indicates that said coherence indicator should be updated to invalid state; and

in response to receipt of said writeback transaction at said node controller, converting said writeback transaction prior to transmission of said writeback transaction to said home processing node, wherein said converted writeback transaction indicates that said coherence indicator should be updated to shared state.

5. The method of claim 4, wherein said conversion is performed only if said memory granule is modified with respect to said shared system memory and said reservation is valid.

6. The method of claim 1, and further comprising:

in response to receipt of a transaction at said home processing node indicating an update to said memory granule, forwarding said transaction from said home processing node to said remote processing node; and

in response to receipt of said forwarded transaction at said remote processing node, cancelling said reservation if said reservation is valid; and

updating said coherence indicator to an invalid state.

7. The method of claim 1, wherein setting said coherence indicator comprises setting said state on response to a writeback of said reserved memory granule to said home processing node by said remote processing node.

8. A computer system, comprising:

a node interconnect; and

a plurality of processing nodes coupled to said node interconnect, wherein said plurality of processing nodes include at least a remote processing node and a home processing node, said remote processing node including a processor having an associated cache hierarchy and said home processing node including;

a shared system memory containing a plurality of memory granules; and

a coherence directory that indicates possible coherence states of copies of memory granules among said plurality of memory granules that are stored within at least one processing node other than said home processing node;

a controller that, while said processor has a reservation for a memory granule among said plurality of memory granules that is not resident within said cache hierarchy, sets a coherence indicator to a state indicating that said reserved memory granule is possibly held non-exclusively at said remote processing node, such that cache coherence communication will reset said reservation if necessary.

9. The computer system of claim 8, wherein said controller updates said coherence indicator from a state indicating that said memory granule is held only at said remote processing node to said state indicating that said reserved memory granule is possibly held non-exclusively at said remote processing node in response to receipt from said remote processing node of a writeback transaction specifying said memory granule.

10. The computer system of claim 9, wherein said writeback transaction indicates that said coherence indicator should be updated to shared state.

11. The computer system of claim 9, said remote processing node further including a node controller that receives said writeback transaction from said processor, said writeback transaction indicating that said coherence indicator should be updated to invalid state, wherein said node controller converts said writeback transaction prior to transmitting said writeback transaction to said home processing node, such that said converted writeback transaction indicates that said coherence indicator should be updated to shared state.

12. The computer system of claim 11, wherein said node controller performs said conversion only if said memory granule is modified with respect to said shared system memory and said reservation is valid.

13. The computer system of claim 8, wherein said computer system further comprises:

means, responsive to receipt of a transaction at said home processing node indicating an update to said memory granule, for forwarding said transaction from said home processing node to said remote processing node;

means, responsive to receipt of said forwarded transaction at said remote processing node, for cancelling said reservation if said reservation is valid; and

wherein said controller updates said coherence indicator to an invalid state in response to receipt of a transaction at said home processing node indicating an update to said memory granule.

14. A home processing node for a multi-node computer system containing a remote processing node including a processor having an associated cache hierarchy, said home processing node comprising:

a shared system memory containing a plurality of memory granules;

a coherence directory that indicates possible coherence states of copies of memory granules among said plurality of memory granules that are stored within at least one processing node other than said home processing node; and

a controller that, while said processor has a reservation for a memory granule among said plurality of memory granules that is not resident within said cache hierarchy, sets a coherence indicator to a state indicating that said reserved memory granule is possibly held non-exclusively at said remote processing node, such that cache coherence communication will reset said reservation if necessary.

15. The home processing node of claim 14, wherein said controller updates said coherence indicator from a state indicating that said memory granule is held only at said remote processing node to said state indicating that said reserved memory granule is possibly held non-exclusively at said remote processing node in response to receipt from said remote processing node of a writeback transaction specifying said memory granule.

16. The home processing node of claim 15, wherein said writeback transaction indicates that said coherence indicator should be updated to shared state.

17. The home processing node of claim 14, said controller further comprising:

means, responsive to receipt of a transaction at said home processing node indicating an update to said memory granule, for forwarding said transaction from said home processing node to said remote processing node;

means for updating said coherence indicator from said state indicating that said reserved memory granule is possibly held non-exclusively by said remote processing node to an invalid state in response to receipt of a transaction at said home processing node indicating an update to said memory granule.

18. A remote processing node for a multi-node computer system, wherein said multi-node computer system contains a home processing node including a shared system memory containing a plurality of memory granules and a coherence directory that indicates possible coherence states of copies of memory granules that are stored within at least one processing node other than said home processing node, said remote processing node comprising:

a processor having an associated cache hierarchy; and

a node controller that receives a writeback transaction from said processor, said writeback transaction indicating that a coherence indicator in said home processing node associated with a memory granule should be updated to invalid state, wherein said node controller converts said writeback transaction prior to transmitting said writeback transaction to said home processing node, such that said converted writeback transaction indicates that said coherence indicator should be updated to shared state.

19. The remote processing node of claim 18, wherein said node controller performs said conversion only if said memory granule is modified with respect to said shared system memory and if said processor has a valid reservation for the memory granule.