Add-on memory coherence directory

• US 9,842,050 B2
• Filed: 04/30/2015
• Issued: 12/12/2017
• Est. Priority Date: 04/30/2015
• Status: Active Grant

• Current Assignee: Beijing Zitiao Network Technology Company Limited (ByteDance Ltd.)
• Original Assignee: International Business Machines Corporation
• Inventors: Daly, David M.
• Primary Examiner(s): Elmore, Reba I
• Application Number:
  US14/700,526
• Publication Number:
  US 20160321191A1
• Time in Patent Office: 957 Days
• Field of Search: 711136, 711146, 711153, 711154, 711170, 709136, 709218, 709226, 709229, 710 11, 710 23, 710120, 712 14, 712 32, 712 43
• CPC Class Codes: G06F 12/0817 using directory methods
• CPC Class Codes: G06F 12/0831 using a bus scheme, e.g. with bus monitoring or watching means
• CPC Class Codes: G06F 2212/621 Coherency control relating to peripheral accessing, e.g. from DMA or I/O device
• CPC Class Codes: G06F 2212/622 State-only directory, i.e. not recording identity of sharing or owning nodes

Abstract

A mechanism is provided for memory coherence in a multiple processor system. Responsive to a memory access resulting in a cache miss in a given processor, the processor determines whether a memory region being accessed is marked as directory-based. Responsive to the given processor determining the memory region is marked as directory-based, the given processor accesses a directory entry corresponding to the memory region to identify a home chip for the page using a directory-based protocol. The given processor forwards the memory access request to the home chip to perform the memory access.

Claims

1. An apparatus comprising:

a processing chip in a multiprocessing data processing system;

wherein the processing chip comprises a plurality of processors;

wherein each processor within the plurality of processors comprises a processing core and a cache; and

wherein a given processor in the processing chip is configured to;

responsive to determining a number of sharers of a given memory region is greater than one and less than a threshold, mark the memory region as directory-based in an effective-to-real address translation table and storing a new directory entry for the memory re ion in an add-on cache directory;

responsive to a memory access resulting in a cache miss in the given processor, determine whether the accessed memory region is marked as directory-based in the effective-to-real address translation table;

responsive to the given processor determining the accessed memory region is not marked as directory-based, perform the memory access using a snooping protocol; and

responsive to the given processor determining the accessed memory region is marked as directory-based in the effective-to-real address translation table, access a directory entry corresponding to the accessed memory region from the add-on cache directory to identify a home chip for the accessed memory region using a directory-based protocol and forward the memory access request from the given processor to the home chip to perform the memory access.

2. The apparatus of claim 1, wherein the directory entry comprises a sharing vector, wherein the sharing vector comprises a hit vector having a bit for each chip in the multiple processor system and wherein each bit in the bit vector is asserted if the corresponding chip is sharing the accessed memory region.

3. The apparatus of claim 1, wherein the memory access request is a load request and wherein the home chip forwards a read request to a sharing node, the sharing node services the read request, and the sharing node forwards data for the read request directly to the given processor.

4. The apparatus of claim 3, wherein the directory entry comprises least recently used (LRU) information for identifying candidate directory entries in the cache to replace.

5. The apparatus of claim 1, wherein the memory access request is a store request and wherein the given processor forwards a read with intent to modify (RWITM) request to the home chip, the home chip forwards the RWITM request and a number of sharers to a sharing node, the home chip sends invalidate notifications to the sharers, the sharing node services the RWITM request, and the sharing node forwards data for the RWITM request directly to the given processor.

6. The apparatus of claim 5, wherein the given processor in the processing chip is further configured to:

receive the data for the RWITM request and the number of sharers;

count confirmation notifications indicating the sharers have invalidated their copies of the data; and

responsive to the number of confirmation notifications equaling the number of sharers, send a final confirmation message to the home node.

7. The apparatus of claim 5, wherein the sharing node services the RWITM request on a given chip locally using a directory protocol.

8. The apparatus of claim 5, wherein the home node updates a sharing vector in the directory entry.

9. The apparatus of claim 1, wherein the given processor in the processing chip is further configured to attempt to service the memory access locally using a directory protocol in parallel with forwarding the memory access request to the home chip to perform the directory-based memory access.

10. The apparatus of claim 1, wherein storing the new directory entry for the accessed memory region in the add-on cache directory comprises:

identifying a directory entry to be replaced from the add-on cache directory; and

replacing the identified directory entry with the new directory entry in the add-on cache directory.

11. The apparatus of claim 10, wherein identifying the directory entry to be replaced comprises identifying a least recently used directory entry in the add-on cache directory.

12. The apparatus of claim 10, wherein identifying the directory entry to be replaced comprises identifying a directory entry having a highest number of sharers.

13. The apparatus of claim 10, wherein identifying the directory entry to be replaced comprises identifying a subset of directory entries in the add-on cache directory having a number of sharers greater than the threshold and identifying a least recently used directory entry within the subset of directory entries.

14. The apparatus of claim 1, wherein the memory access request is a load request and wherein the home chip forwards a read request to a sharing node, the sharing node services the read request, and the sharing node forwards data for the read request directly to the given processor.

15. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

responsive to determining a number of sharers of a given memory region is greater than one and less than a threshold, mark the memory region as directory-based in an effective-to-real address translation table and storing a new directory entry for the memory region in an add-on cache directory;

responsive to a memory access resulting in a cache miss in a given processor, determine whether the accessed memory region is marked as directory-based in the effective-to-real address translation table;

responsive to the given processor determining the accessed memory region is not marked as directory-based, perform the memory access using a snooping protocol; and

responsive to the given processor determining the accessed memory region is marked as directory-based in the effective-to-real address translation table, access a directory entry corresponding to the accessed memory region from the add-on cache directory to identify a home chip far the accessed memory region using a directory-based protocol and forward the memory access request from the given processor to the home chip to perform the memory access.

16. The computer program product of claim 15, wherein storing the new directory entry for the accessed memory region in the add-on cache directory comprises:

identifying a directory entry to be replaced from the add-on cache directory; and

replacing the identified directory entry with the new directory entry in the add-on cache directory.

17. The computer program product of claim 16, wherein identifying the directory entry to be replaced comprises identifying a least recently used directory entry in the add-on cache directory.

18. The computer program product of claim 16, wherein identifying the directory entry to be replaced comprises identifying a directory entry having a highest number of sharers.

19. The computer program product of claim 16, wherein identifying the directory entry to be replaced comprises identifying a subset of directory entries in the add-on cache directory having a number of sharers greater than the threshold and identifying a Least recently used directory entry within the subset of directory entries.