Aggregation of cache-updates in a multi-processor, shared-memory system

• US 6,678,799 B2
• Filed: 10/18/2001
• Issued: 01/13/2004
• Est. Priority Date: 10/18/2001
• Status: Expired due to Fees

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Ang, Boon Seong
• Primary Examiner(s): Sparks, Donald
• Assistant Examiner(s): CHACE, CHRISTIAN
• Application Number:
  US10/002,993
• Publication Number:
  US 20030079085A1
• Time in Patent Office: 817 Days
• Field of Search: 709/213, 711/130, 711/133, 711/141, 711/147, 711/156
• US Class Current: 711/141
• CPC Class Codes: G06F 12/0817 using directory methods
• CPC Class Codes: G06F 12/0824 Distributed directories, e.g. linked lists of caches
• CPC Class Codes: G06F 2212/272 Cache only memory architecture [COMA]

Abstract

Method and arrangement for cache management in a shared memory system. Each of a plurality of intercoupled processing nodes includes a higher-level cache and a lower-level cache having corresponding cache lines. At each node, update-state information is maintained in association with cache lines in the higher-level cache. The update-state information for a cache line tracks whether there is pending update that needs to be distributed from the node. In response to a write-back operation referencing an address cached at a node, the node generates difference data that specifies differences between data in a cache line for the address in the higher-level cache and data in a corresponding cache line in the lower-level cache. The difference data are then provided to one or more other nodes with cached versions of the cache line for the address.

Claims

1. A cache memory arrangement for a shared memory system including storage implemented on a plurality of intercoupled processing nodes, comprising at each node:

a higher-level cache and a lower-level cache, wherein the higher and lower-level caches include respective pluralities of cache lines and the higher-level cache checks for presence of a requested address before conditionally presenting the requested address to the lower-level cache;

a coherence controller coupled to the higher and lower-level caches and to the storage elements, the coherence controller configured to maintain cache coherency for the higher-level cache consistent with an invalidation-based cache coherence protocol while maintaining cache coherency for the lower-level cache consistent with an update-based cache coherence protocol.

2. The arrangement of claim 1, wherein the higher-level cache is a first-level cache, and the lower-level cache is a second-level cache.

3. The arrangement of claim 1, wherein the higher-level cache is a second-level cache, and the lower-level cache is a third-level cache.

4. The arrangement of claim 1, wherein the coherence controller is configured to generate a write-back request signal to the higher-level cache in response to receipt of a cache synchronization signal, the write-back request signal including an address, and the higher-level cache configured to generate a write-back operation signal in response to the write-back request signal if the address in the write-back request signal is present in the higher-level cache.

5. A method for cache management in a shared memory system implemented on a plurality of intercoupled processing nodes, each processing node including a higher-level cache and a lower-level cache having corresponding cache lines, comprising:

maintaining cache coherency for the higher-level cache consistent with an invalidation-based cache coherence protocol;

while

maintaining cache coherency for the lower-level cache consistent with an update-based cache coherence protocol.

6. The method of claim 5, further comprising:

in response to a write-back operation referencing an address cached at a node, generating difference data that specifies differences between data in a cache line for the address in the higher-level cache and data in a corresponding cache line in the lower-level cache; and

providing the difference data to one or more other nodes with cached versions of the cache line for the address.

7. The method of claim 6, further comprising in response to receipt of the difference data at a node, purging a version of the cache line from the higher-level cache at the node and updating a version of the cache line in the lower-level cache at a node.

8. An apparatus for cache management in a shared memory system implemented on a plurality of intercoupled processing nodes, each processing node including a higher-level cache and a lower-level cache having corresponding cache lines, comprising:

means for maintaining cache coherency for the higher-level cache consistent with an invalidation-based cache coherence protocol;

while

maintaining cache coherency for the lower-level cache consistent with an update-based cache coherence protocol.

9. A cache memory arrangement for a shared memory system implemented on a plurality of intercoupled processing nodes, comprising at each node:

a higher-level cache and a lower-level cache, wherein the higher and lower-level caches include respective pluralities of cache lines and the higher-level cache checks for presence of a requested address before conditionally presenting the requested address to the lower-level cache;

a plurality of storage elements for storage of update-state information of the cache lines in the higher-level cache;

a coherence controller coupled to the higher and lower-level caches and to the storage elements, the coherence controller configured to,

generate a cache-line-fetch request with write permission for a requested address in a store operation if the requested address is not present in the lower-level cache,

in response to data received for the cache-line-fetch request with write permission, store update-state information in one of the storage elements associated with the cache-line of the requested address, and

in response to a write-back operation signal, clear the update-state information associated with the cache line, generate difference data that specifies differences between data in a cache line referenced by the cache-line code in the higher-level cache and data in a corresponding cache line in the lower-level cache, and provide the difference data to one more nodes with cached versions of the cache line.

10. The arrangement of claim 9, wherein the plurality of storage elements are respectively associated with the cache lines in the higher-level cache.

11. The arrangement of claim 9, wherein the higher-level cache is a first-level cache, and the lower-level cache is a second-level cache.

12. The arrangement of claim 9, wherein the higher-level cache is a second-level cache, and the lower-level cache is a third-level cache.

13. A cache memory arrangement for a shared memory system implemented on a plurality of intercoupled processing nodes, comprising at each node:

a higher-level cache and a lower-level cache, wherein the higher and lower-level caches include respective pluralities of cache lines and the higher-level cache checks for presence of a requested address before conditionally presenting the requested address to the lower-level cache;

an update-pending queue, each entry in the update-pending queue identifying a cache line in the higher-level cache;

a coherence controller coupled to the higher and lower-level caches and to the update-pending queue, the coherence controller configured to,

in response to a memory-write cache-line fetch request received from a requester processing node, the memory-write cache-line fetch request including a requested address, provide a cache line with the requested address to the requester node,

in response to a receipt of a cache line with write permission for a requested address, enter a cache-line code that identifies the cache line of the requested address in the update-pending queue,

in response to a write-back operation signal, remove a cache-line code from the update-pending queue, generate difference data that specifies differences between data in the higher-level cache for a cache line referenced by the cache-line code and a corresponding cache line in the lower-level cache, and provide the difference data to a home node that hosts the cache line, and

in response to receipt of difference data, distribute the difference data to one or more nodes having cached versions of the associated cache line.

14. The arrangement of claim 13, wherein the difference data includes data that identifies which addresses have modified data and associated modified data.

15. The arrangement of claim 13, wherein the lower-level cache is implemented with DRAM and the coherence controller is further configured to read from the lower-level cache, the cache line corresponding to the cache-line code, and simultaneously write the cache line from the higher-level cache and referenced by the cache-line code to the lower-level cache.

16. The arrangement of claim 13, wherein the higher-level cache is a first-level cache, and the lower-level cache is a second-level cache.

17. The arrangement of claim 13, wherein the higher-level cache is a second-level cache, and the lower-level cache is a third-level cache.

18. The arrangement of claim 13, wherein the coherence controller is configured to generate a write-back request signal to the higher-level cache when the update-pending queue is full, the write-back request signal including an address, and the higher-level cache is configured to generate a write-back operation signal in response to the write-back request signal if the address in the write-back request signal is present in the higher-level cache.

19. The arrangement of claim 13, wherein the coherence controller is configured to generate a write-back request signal to the higher-level cache when an associated entry in the update-pending queue is to be replaced, the write-back request signal including an address, and the higher-level cache is configured to generate a write-back operation signal in response to the write-back request signal if the address in the write-back request signal is present in the higher-level cache.

20. The arrangement of claim 13, wherein the coherence controller is configured to generate a write-back request signal to the higher-level cache in response to receipt of a cache synchronization signal, the write-back request signal including an address, and the higher-level cache is configured to generate a write-back operation signal in response to the write-back request signal if the address in the write-back request signal is present in the higher-level cache.

21. The arrangement of claim 13, wherein the coherence controller is configured to generate the write-back request signal to the higher-level cache for a particular entry in the update-pending queue after passage of a selected period of time following placement of the entry in the update-pending queue, the write-back request signal including an address, and the higher-level cache is configured to generate a write-back operation signal in response to the write-back request signal if the address in the write-back request signal is present in the higher-level cache.

22. A method for cache management in a shared memory system implemented on a plurality of intercoupled processing nodes, each processing node including a higher-level cache and a lower-level cache having corresponding cache lines, comprising:

maintaining update-state information in association with cache lines in the higher-level cache, wherein the update-state information for a cache line indicates pending updates from the node with a cached version of the cache line;

in response to a write-back operation referencing an address cached at a node, generating difference data that specifies differences between data in a cache line for the address in the higher-level cache and data in a corresponding cache line in the lower-level cache; and

providing the difference data to one or more other nodes with cached versions of the cache line for the address.

23. The method of claim 22, further comprising in response to receipt of the difference data at a node, purging a version of the cache line from the higher-level cache at the node and updating a version of the cache line in the lower-level cache at a node.

24. The method of claim 23, wherein maintaining the update-state information comprises at each node, entering in an update-pending queue cache-line codes that identify cache lines cached with write permission at the node, and further comprising:

in response to the write-back operation, removing from the update-pending queue the cache-line code associated with the cache line for the address.

25. The method of claim 23, wherein the each memory address is hosted by a home node and further comprising:

providing the difference data to the home node; and

distributing the difference data from the home node to the one or more other nodes.

26. The method of claim 25, further comprising selecting at each node that hosts a range of memory addresses, an update-based or invalidation-based cache coherence protocol for each address requested with write permission, wherein the update-state information for a cache line indicates write permission at the node with a cached version of the cache line with update-based cache coherence protocol.

27. The method of claim 26, further comprising maintaining at each node that hosts a range of memory addresses, a directory having entries that identify cache lines that are cached in the hosted range of addresses, read-write permissions associated with the cache lines, and cache coherence protocols associated with the cache lines.

28. An apparatus for cache management in a shared memory system implemented on a plurality of intercoupled processing nodes, each processing node including a higher-level cache and a lower-level cache having corresponding cache lines, comprising:

means for maintaining update-state information in association with cache lines in the higher-level cache, wherein the update-state information for a cache line indicates pending updates from the node with a cached version of the cache line;

means, responsive to a write-back operation referencing an address cached at a node, for generating difference data that specifies differences between data in a cache line for the address in the higher-level cache and data in a corresponding cache line in the lower-level cache; and

means for providing the difference data to one or more other nodes with cached versions of the cache line for the address.