System and method for event monitoring in cache coherence protocols without explicit invalidations

• US 10,387,312 B2
• Filed: 01/02/2015
• Issued: 08/20/2019
• Est. Priority Date: 01/03/2014
• Status: Active Grant

• Current Assignee: Eta Scale AB
• Original Assignee: Eta Scale AB
• Inventors: Kaxiras, Stefanos, Ros, Alberto
• Primary Examiner(s): Yi, David
• Assistant Examiner(s): Goldschmidt, Craig S
• Application Number:
  US15/108,948
• Publication Number:
  US 20160321181A1
• Time in Patent Office: 1,691 Days
• Field of Search: None
• CPC Class Codes: G06F 12/0811 with multilevel cache hierarchies
• CPC Class Codes: G06F 12/0815 Cache consistency protocols
• CPC Class Codes: G06F 12/082 Associative directories G06F12/0822 takes precedence
• CPC Class Codes: G06F 12/0831 using a bus scheme, e.g. with bus monitoring or watching means
• CPC Class Codes: G06F 12/084 with a shared cache
• CPC Class Codes: G06F 12/128 adapted to multidimensional cache systems, e.g. set-associative, multicache, multiset or multilevel
• CPC Class Codes: G06F 13/1663 Access to shared memory
• CPC Class Codes: G06F 2212/314 In storage network, e.g. network attached cache
• CPC Class Codes: G06F 2212/621 Coherency control relating to peripheral accessing, e.g. from DMA or I/O device

Abstract

Synchronization events associated with cache coherence are monitored without using invalidations. A callback-read is issued to a memory address associated with the synchronization event, which callback-read either reads the last value written in the memory address or blocks until a next write takes place in the memory address and reads a newly written value.

Claims

1. A computer system comprising:

multiple processor cores;

at least one local cache memory associated with and operatively coupled to each core for storing one or more cache lines accessible only by the associated core;

a shared memory, the shared memory being operatively coupled to the local cache memories and accessible by the cores, the shared memory being capable of storing a plurality of cache lines; and

a callback directory containing a set of callback (CB) bits associated with a memory address, wherein each CB bit in the set corresponds to a core;

wherein a core issuing a callback-read to the memory address either reads the last value written in the memory address, or is blocked from reading from the memory address until the next write takes place in the memory address and then reads a new value of said next write, such that the callback-read enables event monitoring for coherence of the at least one local cache and the shared memory without using explicit invalidations,

when a CB bit corresponding to the core that issued the callback-read is set, the callback-read is completed by the core reading the last value in the memory address;

when the CB bit corresponding to the core that issued the callback-read is unset, the callback-read triggers setting of the CB bit and the callback-read is completed when the new value is written in the memory address and the new value is forwarded to the core that issued the callback-read; and

when the new value is written in the memory address, the new value is forwarded to all of the cores that have their corresponding CB bit set for the memory address, CB bits previously set for the memory address are cleared, and CB bits previously unset for the memory address are set.

2. The computer system of claim 1 where the CB bits correspond to program threads of the cores.

3. The computer system of claim 1, wherein the callback read to the memory address is issued by a compare-and-callback instruction wherein the core either continues to a next instruction or issues the callback read to the memory address.

4. The computer system of claim 3, wherein the compare-and-callback instruction takes the core to a power-saving state when the compare-and-callback instruction is awaiting a response to a callback read.

5. The computer system of claim 1, further comprising a callback queue which stores pending callback requests issued for the memory address.

12. The computer system of claim 1, wherein each of the multiple processor cores is configured to issue at least one of a callback read and a callback write to access the callback directory using one of a corresponding special load/store or atomic instruction.

13. The computer system of claim 1 wherein the callback directory is located at one or more cache levels where writes become globally visible in the computer system.

6. A computer system comprising:

multiple processor cores;

at least one local cache memory associated with and operatively coupled to each core for storing one or more cache lines accessible only by the associated core;

a shared memory, the shared memory being operatively coupled to the local cache memories and accessible by the cores, the shared memory being capable of storing a plurality of cache lines; and

a callback directory containing a set of callback (CB) bits and full/empty (F/E) bits associated with a memory address, wherein each CB bit and each F/E bit of the set corresponds to a different one of the cores;

wherein a core issuing a callback-read to the memory address either reads the last value written in the memory address, or is blocked from reading from the memory address until the next write takes place in the memory address and then reads a new value of said next write, such that the callback-read enables event monitoring for coherence of the at least one local cache and the shared memory without using explicit invalidations

when an F/E bit corresponding to the core that issued the callback-read is set, the callback-read from the core to the memory address with the set of CB and F/E bits is completed by the core reading the last value in the memory address;

when the F/E bit corresponding to the core that issued the callback-read is unset, the callback-read triggers setting of a CB bit which corresponds to the memory address, and the call-back read is completed when the new value is written in the memory address and the new value is forwarded to the core that issued the callback-read; and

when the new value is written in the memory address, the new value is forwarded to all of the cores that have a corresponding CB bit set for the memory address, corresponding CB bits previously set are cleared, and F/E bits that correspond to the CB bits previously unset are set.

7. The computer system of claim 6 wherein a read through read of the memory address issued by a core bypasses the at least one local cache memory associated with the core and resets an F/E bit of the memory address that corresponds to the core.

8. The computer system of claim 6 wherein the callback directory also contains an All/One bit associated with the memory address;

and wherein when the All/One bit is set to one, the F/E bits for the memory address operate in unison such that the F/E bits for the memory address act as a single F/E bit; and

wherein when the All/One bit is set to all, the F/E bits for the memory address operate independently of one another such that each F/E bit has its value changed independently by a corresponding core.

9. The computer system of claim 8 wherein a write callback one write instruction to the memory address sets the All/One bit to one;

wherein the write callback one write instruction selects a single core that has its corresponding CB bit set for the memory address and forwards a written value to only this single core and resets the corresponding CB bit; and

further wherein when no core has its corresponding CB bit set for the memory address, the write callback one write instruction sets all the F/E bits which correspond to the memory address.

10. The computer system of claim 8 wherein a write callback zero write instruction to the memory address sets the All/One bit which corresponds to the memory address to one.

11. The computer system of claim 8 wherein a write callback all write instruction to the memory address sets the All/One bit to all;

forwards a written value to all of the cores that have their corresponding CB bit set for the memory address, clears the corresponding CB bits previously set and sets the F/E bits that correspond to CB bits previously unset.

14. The computer system of claim 6 wherein, when there is no free space in the callback directory, the memory address is evicted to accommodate a new memory address and a current data value of the evicted memory address is forwarded to all the cores that have the corresponding CB bit set for the evicted memory address;

and

wherein the CB bits for the new memory address that replaces the evicted address are initialized to unset and the F/E bits for the new memory are initialized to set.