Satisfying memory ordering requirements between partial reads and non-snoop accesses

• US 8,694,736 B2
• Filed: 08/21/2012
• Issued: 04/08/2014
• Est. Priority Date: 07/07/2008
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Beers, Robert H., Chou, Ching-Tsun, Safranek, Robert J., Vash, James
• Primary Examiner(s): YU, JAE UN
• Application Number:
  US13/591,157
• Publication Number:
  US 20120317369A1
• Time in Patent Office: 595 Days
• Field of Search: 711/118, 711/146, 711/141, 711/143, 711/154
• US Class Current: 711/146
• CPC Class Codes: G06F 12/0806 Multiuser, multiprocessor or multiprocessing cache systems
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0824 Distributed directories, e.g. linked lists of caches
• CPC Class Codes: G06F 12/0831 using a bus scheme, e.g. with bus monitoring or watching means
• CPC Class Codes: G06F 12/0833 in combination with broadcast means (e.g. for invalidation or updating)
• CPC Class Codes: G06F 12/12 Replacement control
• CPC Class Codes: G06F 12/128 adapted to multidimensional cache systems, e.g. set-associative, multicache, multiset or multilevel
• CPC Class Codes: G06F 2212/2542 Non-uniform memory access [NUMA] architecture
• CPC Class Codes: G06F 2212/62 Details of cache specific to multiprocessor cache arrangements
• CPC Class Codes: G06F 2212/621 Coherency control relating to peripheral accessing, e.g. from DMA or I/O device
• CPC Class Codes: G06F 2212/70 Details relating to dynamic memory management
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A method and apparatus for preserving memory ordering in a cache coherent link based interconnect in light of partial and non-coherent memory accesses is herein described. In one embodiment, partial memory accesses, such as a partial read, is implemented utilizing a Read Invalidate and/or Snoop Invalidate message. When a peer node receives a Snoop Invalidate message referencing data from a requesting node, the peer node is to invalidate a cache line associated with the data and is not to directly forward the data to the requesting node. In one embodiment, when the peer node holds the referenced cache line in a Modified coherency state, in response to receiving the Snoop Invalidate message, the peer node is to writeback the data to a home node associated with the data.

Claims

1. An apparatus comprising:

receiving logic to receive a snoop invalidate message referencing data;

a cache memory including a cache line to hold the data; and

protocol logic coupled to the receiving logic and the cache memory, the protocol logic, in response to the receiving logic receiving the snoop invalidate message referencing the data and the cache line being held in a first cache coherency state, to generate a writeback of the data to a home node associated with the data and to initiate a transition of the cache line from the first cache coherency state to an invalid cache coherency state, wherein the protocol logic, in response to the receiving logic receiving the snoop invalidate message referencing the data, is further to not directly provide the data to a requesting agent associated with the snoop invalidating message regardless of the cache line being held in any cache coherency state.

2. The apparatus of claim 1, wherein the first cache coherency state is a Modified cache coherency state.

3. The apparatus of claim 2, wherein the protocol logic, in response to the receiving logic receiving the snoop invalidate message referencing the data and the cache line being held in an Exclusive or Shared state, is further to provide a Response Invalidate message to a home node associated with the data.

4. The apparatus of claim 1, wherein the protocol logic and the receiving logic are logically organized into an interconnect layer stack, wherein the interconnect layer stack includes at least a physical layer to include the receiving logic, a link layer, and a protocol layer to include the protocol logic.

5. The apparatus of claim 1, wherein the receiving logic is to be coupled to a high speed point-to-point link.

6. An apparatus comprising:

receiving logic to receive a snoop invalidate message referencing data;

a cache memory including a cache line to hold the data; and

protocol logic coupled to the receiving logic and the cache memory, the protocol logic, in response to the receiving logic receiving the snoop invalidate message referencing the data and the cache line being held in a first cache coherency state, to generate a writeback of the data to a home node associated with the data and to initiate a transition of the cache line from the first cache coherency state to an invalid cache coherency state, wherein the snoop invalidate message is selected from a group consisting of a Snoop Invalidate No Forward message, a Snoop Invalidate Invalid to Exclusive message, and a Snoop Invalidate X to Invalid message.

7. The apparatus of claim 6, wherein the protocol logic and the receiving logic are logically organized into an interconnect layer stack, wherein the interconnect layer stack includes at least a physical layer to include the receiving logic, a link layer, and a protocol layer to include the protocol logic.

8. The apparatus of claim 6, wherein the receiving logic is to be coupled to a high speed point-to-point link.

9. The apparatus of claim 6, wherein the protocol logic, in response to the receiving logic receiving the snoop invalidate message referencing the data, is further to not directly provide the data to a requesting agent associated with the snoop invalidating message regardless of the cache line being held in any cache coherency state.

10. A system comprising:

a first peer node to implement a first interconnect stack including a first protocol layer to generate a first message and a second message referencing a data element, and a first physical layer to transmit the first message to a home node and the second message to a second peer node;

wherein the second peer node is to be coupled to the first peer node through a point-to-point link, the second peer node to include a cache memory to hold a cached copy of the data element and to implement a second interconnect stack including a second physical layer to receive the second message and a second protocol layer to not forward the cached copy of the data element directly to the first peer node and to cause the cached copy of the data element to be transitioned to an Invalid coherency state in response to the second physical layer receiving the second message; and

wherein the home node is to be coupled to the first peer node through a point-to-point link and is to implement a third interconnect stack including a third physical layer to receive the first message and a third protocol layer to generate a data message to provide a correct copy of the data element to the first peer node in response to the third physical layer receiving the first message.

11. The apparatus of claim 10, wherein the second protocol layer is further to generate a response message to indicate to the home node that the cached copy is to be left in the Invalid state and the second physical layer is further to transmit the response message to the home node in response to the second physical layer receiving the second message and the cached copy of the data element being held in an Exclusive or Shard coherency state in the cache memory.

12. The apparatus of claim 10, wherein the second protocol layer is further to generate a writeback message to provide the cached copy of the data element to the home node and the second physical layer is further to transmit the writeback message to provide the cached copy of the data element to the home node in response to the second physical layer receiving the second message and the cached copy of the data element being held in a Modified coherency state in the cache memory.

13. The apparatus of claim 12, wherein the third protocol layer to generate the data message to provide the correct copy of the data element to the first peer node comprises the third protocol layer to generate the data message to provide the cached copy of the data element in response to the third physical layer receiving the first message and receiving the cached copy of the data element from the second peer node.

14. The apparatus of claim 13, wherein the first message includes a Read Invalidate message, the second message includes a Snoop Invalidate message, and the data message includes a writeback marker message and a writeback data message and wherein the first protocol layer is to generate the Read Invalidate message and the Snoop Invalidate message in response to a partial read of the data element.

15. The apparatus of claim 14, wherein the Read Invalidate message includes a Read Invalidate No Forward (RdInvNoFwd) message, the Snoop Invalidate message includes a Snoop Invalidate No Forward (SnpInvNoFwd) message, the writeback marker message includes a Response Invalid Writeback (RspIWb) message, and the writeback data message includes a Writeback Invalid Data (WbIData) message.

16. A method comprising:

generating a first message referencing data with a requesting node;

receiving the first message referencing data with a peer node;

not directly forwarding a copy of the data held in a cache line of a cache memory in the peer node to the requesting node and invalidating the cache line, in response to receiving the first message with the peer node;

determining if the cache line is in a Modified cache coherency state in response to receiving the first message with the peer node;

pushing the copy of the data held in the cache line of a cache memory in the peer node to a home node, in response to determining the cache line is in the Modified cache coherency state; and

transmitting a response message from the peer node to the home node to indicate the copy of the data is held in a cache line in an Invalid cache coherency state, in response to determining the cache line is not in the Modified cache coherency state.

17. The method of claim 16, wherein generating a first message referencing data with a requesting node comprises generating the first message referencing data in a protocol layer of an interconnect state implemented in the requesting node in response to a partial read request by the requesting node.

18. The method of claim 16, further comprising:

forwarding the data from the home node to the requesting node, in response to the home node receiving the response message from the peer node, and

forwarding the copy of the data, pushed from the peer node to the home node, from the home node to the requesting node, in response to the home node receiving the copy of the data pushed from the peer node to the home node.

19. The method of claim 18, wherein the first message includes a Snoop Invalidate No Forward (SnpInvNoFwd) message.

20. The method of claim 18, wherein the requesting node, the peer node, and the home node are each interconnected to each other through a point-to-point serial link.