Managing sparse directory evictions in multiprocessor systems via memory locking

• US 20050251626A1
• Filed: 04/24/2003
• Published: 11/10/2005
• Est. Priority Date: 04/24/2003
• Status: Active Grant

• Current Assignee: Sanmina-SCI Corporation
• Original Assignee: Newisys, Inc. (Sanmina-SCI Corporation)
• Inventors: Glasco, David B.
• Granted Patent: US 7,024,521 B2
• Time in Patent Office: Days
• US Class Current: 711/133
• CPC Class Codes: G06F 12/082 Associative directories G06F12/0822 takes precedence
• CPC Class Codes: G06F 12/0828 with concurrent directory accessing, i.e. handling multiple concurrent coherency transactions
• CPC Class Codes: G06F 12/12 Replacement control
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users

Abstract

Cache coherence directory eviction mechanisms are described for use in computer systems having a plurality of multiprocessor clusters. Interaction among the clusters is facilitated by a cache coherence controller in each cluster. A cache coherence directory is associated with each cache coherence controller identifying memory lines associated with the local cluster that are cached in remote clusters. Techniques are provided for managing eviction of entries in the cache coherence directory by locking memory lines in a home cluster without causing a memory controller to generate probes to processors in the home cluster.

Claims

1. A computer system, comprising a plurality of processor clusters, each cluster including a plurality of local nodes and a cache coherence controller interconnected by a local point-to-point architecture, the computer system further comprising memory corresponding to a global memory space, each cluster corresponding to a contiguous portion of the global memory space, selected ones of the plurality of local nodes in each cluster having a memory controller associated therewith, each memory controller in each cluster being responsible for a memory range within the corresponding contiguous portion of the global memory space, the cache coherence controller in each cluster having a cache coherence directory associated therewith, entries in the cache coherence directory in each cluster corresponding to memory lines within the corresponding contiguous portion of the global memory space that are cached in remote clusters, the cache coherence controller being operable to facilitate an eviction of a first one of the entries corresponding to a copy of a first memory line by sending a lock request identifying the first memory line to a first one of the memory controllers corresponding to the first memory line.

2. The computer system of claim 1, wherein the cache coherence directory is operable to designate the first entry to be evicted and maintain the first entry therein at least until the first memory controller allows the eviction to proceed.

3. The computer system of claim 2, wherein the cache coherence directory includes an eviction buffer, the cache coherence directory being operable to designate the first entry by placing the first entry in the eviction buffer.

4. The computer system of claim 1, wherein the copy of the first memory line is a modified copy.

5. The computer system of claim 1, wherein the copy of the first memory line is an unmodified copy.

6. The computer system of claim 1, wherein the cache coherence controller is operable to generate an invalidating probe to a remote cluster having a cached copy of the first memory line, in response to which all copies of the first memory line in the cache memories of the remote cluster are invalidated.

7. The computer system of claim 3, wherein the cache coherence directory is further operable to invalidate the first entry in the eviction buffer in response to a communication from the first memory controller corresponding to the eviction.

8. The computer system of claim 4, wherein the lock request comprises a unique memory locking transaction.

9. The computer system of claim 5, wherein the lock request comprises an existing transaction in which semantics are overloaded to implement locking semantics.

10. The computer system of claim 6, wherein the cache coherence controller is operable to enable interaction by the associated processing nodes with processing nodes in others of the clusters in accordance with the associated cache coherence directory.

11. The computer system of claim 9, wherein the existing transaction comprises a non-coherent zero-sized write.

12. The computer system of claim 9, wherein the existing transaction comprises a non-coherent zero-sized read.

13. The computer system of claim 10, wherein the cache coherence controller includes the cache coherence directory.

14. The computer system of claim 10, wherein the cache coherence controller is operable using the cache coherence directory to forward the invalidating probes only to clusters having at least one copy of the first memory line in the associated cache memories.

15. A cache coherence controller for use in a computer system comprising a plurality of processor clusters, each cluster including a plurality of local nodes and an instance of the cache coherence controller interconnected by a local point-to-point architecture, the computer system further comprising memory corresponding to a global memory space, each cluster corresponding to a contiguous portion of the global memory space, selected ones of the plurality of local nodes in each cluster having a memory controller associated therewith, each memory controller in each cluster being responsible for a memory range within the corresponding contiguous portion of the global memory space, the cache coherence controller including a cache coherence directory, entries in the cache coherence directory in each cluster corresponding to memory lines within the corresponding contiguous portion of the global memory space which are cached in remote clusters, the cache coherence controller being operable to facilitate an eviction of a first one of the entries corresponding to a copy of a first memory line by sending a lock request identifying the first memory line to a first one of the memory controllers corresponding to the first memory line.

16. An integrated circuit comprising the cache coherence controller of claim 15.

17. A set of semiconductor processing masks representative of at least a portion of the cache coherence controller of claim 15.

18. At least one computer-readable medium having data structures stored therein representative of the cache coherence controller of claim 15.

19. The integrated circuit of claim 16, wherein the integrated circuit comprises an application-specific integrated circuit.

20. The at least one computer-readable medium of claim 18, wherein the data structures comprise a simulatable representation of the cache coherence controller.

21. The at least one computer-readable medium of claim 18, wherein the data structures comprise a code description of the cache coherence controller.

22. The at least one computer-readable medium of claim 20, wherein the simulatable representation comprises a netlist.

23. The at least one computer-readable medium of claim 21, wherein the code description corresponds to a hardware description language.

24. A computer-implemented method for evicting entries in a cache coherence directory, the cache coherence directory being associated with a computer system comprising a plurality of processor clusters, each cluster including a plurality of local nodes and a cache coherence controller interconnected by a local point-to-point architecture, the computer system further comprising memory corresponding to a global memory space, each cluster corresponding to a contiguous portion of the global memory space, selected ones of the plurality of local nodes in each cluster having a memory controller associated therewith, each memory controller in each cluster being responsible for a memory range within the corresponding contiguous portion of the global memory space, the cache coherence controller in a first cluster having the cache coherence directory associated therewith, entries in the cache coherence directory corresponding to memory lines within the contiguous portion of the global memory space corresponding to the first cluster which are cached in remote clusters, the method comprising:

determining that a first one of the entries in the cache coherence directory should be evicted, the first entry corresponding to a copy of a first memory line;

generating a lock request identifying the first memory line directed to a first one of the memory controllers corresponding to the first memory line;

in response to the lock request, locking the first memory line and generating a locked response to the cache coherence controller in the first cluster;

invalidating the first entry in the cache coherence directory in response to the locked response received by the cache coherence controller;

sending a probe to a remote cluster having at least one copy of the first memory line; and

invalidating all copies of the first memory line in cache memories of nodes in the remote cluster.

25. The computer-implemented method of claim 24, further comprising generating a response from the remote cluster to the cache coherence controller.

26. The computer-implemented method of claim 25, wherein the response from the remote cluster to the cache coherence controller includes a dirty data response.

27. The computer-implemented method of claim 25, further comprising generating an unlock response from the cache coherence controller to the first one of the memory controllers.

28. The computer-implemented method of claim 26, further comprising generating an unlock response from the cache coherence controller to the first one of the memory controllers.

29. The computer-implemented method of claim 27, further comprising unlocking the first memory line.

30. The computer-implemented method of claim 28, further comprising:

unlocking the first memory line; and

writing the dirty data to a memory.