Method and apparatus for a coherence transformer with limited memory for connecting computer system coherence domains

• US 5,829,034 A
• Filed: 07/01/1996
• Issued: 10/27/1998
• Est. Priority Date: 07/01/1996
• Status: Expired due to Term

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Hagersten, Erik E., Wood, David A., Hill, Mark Donald
• Primary Examiner(s): Swann, Tod R.
• Assistant Examiner(s): Tzeng, Fred Fei
• Application Number:
  US08/677,014
• Time in Patent Office: 848 Days
• Field of Search: 395/468, 395/441, 395/442, 395/420, 395/200.02, 395/200.08, 395/200.15, 395/200.2, 395/471, 395/448, 395/473, 395/463, 395/472, 395/458
• US Class Current: 711/141
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/082 Associative directories G06F12/0822 takes precedence
• CPC Class Codes: G06F 12/0826 Limited pointers directories; State-only directories without pointers
• CPC Class Codes: G06F 12/0831 using a bus scheme, e.g. with bus monitoring or watching means
• CPC Class Codes: G06F 2212/622 State-only directory, i.e. not recording identity of sharing or owning nodes

Abstract

A coherence transformer for allowing a computer node and one or more external devices to share memory blocks having local physical addresses at a memory module of the computer node. The coherence transformer includes logic for ascertaining whether a memory access request from the external device for a memory block should be responded to using a snoop-only approach or an Mtag-only approach. The snoop-only approach requires a tag in a snoop tag array of the coherence transformer be available to track the memory block for an entire duration that the memory block is cached by the external device. The Mtag-only approach only temporarily stores the memory block until a global state associated with the memory block can be written back into the memory module of the computer node. The snoop tag array allows the coherence transformer to snoop the bus of the computer node to intervene and respond to memory access requests pertaining to a memory block externally cached and tracked by the snoop tag array.

Claims

1. In a computer system having a computer node which has a common bus, a method for enabling an external device in an external domain that is external to said computer node to share memory blocks having local physical addresses in a memory module at said computer node irrespective whether said external device and said common bus both employ a common protocol and irrespective whether said external device and said common bus both operate at the same speed, each of said memory blocks having an associated Mtag for tracking a global state associated with said each of said memory block, including a global exclusive state for indicating that said each of said memory blocks is exclusive to said computer node, a global shared state for indicating that said each of said memory blocks is shared by said computer node with said external device, and a global invalid state for indicating that said each of said memory blocks is invalid in said computer node, said method comprising:

receiving through a coherence transformer that is coupled to said common bus, a first memory access request for a first memory block from said external device;

obtaining a first copy of said first memory block, using said coherence transformer, from said common bus, said coherence transformer having a snoop tag array having a plurality of snoop tags, each of said plurality of snoop tags being configured to track an external state of a copy of one of said memory blocks if cached by said external device, said external state including one of an external exclusive state for indicating that said copy of one of said memory blocks is exclusive to said external domain, an external shared state for indicating that said copy of one of said memory blocks is shared by said external domain, and an external invalid state for indicating that said copy of one of said memory blocks is invalid in said external domain;

if at least one tag in said plurality of snoop tags is available for tracking said external state of said first copy of said first memory block, responding to said first memory access request using a snoop-only approach by tracking said external state of said first copy of said first memory block using said at least one tag;

else if at least one tag in said plurality of snoop tags is not available for tracking said external state of said first copy of said first memory block, responding to said first memory access request using an Mtag-only approach by modifying a first Mtag associated with said first memory block in said memory module at said computer node using said coherence transformer to reflect that said external device is caching said first copy of said first memory block; and

sending said first copy of said first memory block from said coherence transformer to said external device.

2. The method of claim 1 wherein said first memory access request from said external device represents a request for an exclusive copy of said first memory block and said step of responding to said first memory access request using said Mtag-only approach further includes a step of changing said first Mtag in said memory module to a global invalid state.

3. The method of claim 2 wherein said step of responding to said first memory access request using said Mtag-only approach further comprising a step of invalidating all valid copies of said first memory block at said computer node.

4. The method of claim 1 wherein said first memory access request from said external device represents either a request for an exclusive copy of said first memory block or a request for a shared copy of said first memory block, said step of responding to said first memory access request using said Mtag-only approach further includes the steps of:

prior to said modifying step, examining said first Mtag associated with said first memory block; and

proceeding with said modifying step and said sending step only if said first Mtag does not represent a global invalid state.

5. The method of claim 1 wherein said first memory access request from said external device represents a request for a shared copy of said first memory block and said step of responding to said first memory access request using said Mtag-only approach further includes a step of changing said first Mtag in said memory module to a global shared state.

6. The method of claim 5 wherein said step of responding to said first memory access request using said Mtag-only approach further includes the steps of:

prior to said modifying step, examining said first Mtag associated with said first memory block;

proceeding with said modifying step and said sending step only if said first Mtag does not represent a global invalid state; and

if said first Mtag represents a global invalid state, flagging an error condition.

7. The method of claim 1 further comprising the steps of:

receiving a write back request for a second memory block from said external device at said coherence transformer;

obtaining said first copy of said second memory block, using said coherence transformer, from said external device;

writing said first copy of said second memory block from said coherence transformer to said memory module at said computer node; and

if said first copy of said first memory block is not tracked in a snoop tag of said snoop tag array, modifying, using said coherence transformer, an Mtag associated with said second memory block in said memory module at said computer node to reflect that said computer node has an exclusive copy of said second memory block.

8. The method of claim 1 wherein said global state for said each of said memory blocks is employed as said external state for said each of said memory blocks, whereby a global exclusive state represents an external invalid state, a global shared state represents an external shared state, and a global invalid state represents an external exclusive state.

9. The method of claim 1 further comprising the steps of:

receiving a writeback request for said first memory block from said external device at said coherence transformer;

obtaining said first copy of said first memory block, using said coherence transformer, from said external device;

writing said first copy of said first memory block from said coherence transformer to said memory module at said computer node; and

if said first copy of said first memory block was tracked in a snoop tag of said snoop tag array prior to said writing step, unallocating said snoop tag of said snoop tag array, thereby rendering said snoop tag available for tracking other externally cached memory blocks and causing said first copy of said first memory block to be no longer tracked by said snoop tag array.

10. The method of claim 1 further comprising the step of responding, through said coherence transformer, to a second memory access request on said common bus on behalf of said external device, comprising:

monitoring memory access requests on said common bus, using said coherence transformer, to determine whether a second memory access request of said memory access requests on said common bus pertains to any one of memory blocks tracked in snoop tags of said snoop tag array; and

if said second memory access request pertains to a second memory block, said second memory block representing said one of memory blocks tracked in said snoop tags of said snoop tag array, responding to said second memory access request using said snoop-only approach, including responding to said second memory access request using said coherence transformer.

11. The method of claim 10 wherein said coherence transformer responds to said second memory access request in said snoop-only approach only if a snoop tag tracking said second memory block in said snoop tag array indicates that a first copy of said second memory block is valid at said external device.

12. The method of claim 11 wherein said second memory access request is a request for an exclusive copy and said snoop tag tracking said second memory block indicates that said first copy of said second memory block at said external device is an exclusive copy of said second memory block, said step of responding to said second memory access request using said snoop-only approach comprises:

obtaining, using said coherence transformer, a second copy of said second memory block from said first copy of said second memory block at said external device;

invalidating said first copy of said second memory block at said external device; and

forwarding said second copy of said second memory block from said coherence transformer to said common bus to enable a progenitor of said second memory access request to obtain said second copy of said second memory block; and

unallocating said snoop tag of said snoop tag array, thereby rendering said snoop tag available for tracking other externally cached memory blocks and causing said second memory block to be no longer tracked by said snoop tag array.

13. The method of claim 11 wherein said second memory access request is a request for an exclusive copy and said snoop tag tracking said second memory block indicates that said first copy of said second memory block at said external device is a shared copy of said second memory block, said step of responding to said second memory access request using said snoop-only approach comprises:

invalidating said first copy of said second memory block at said external device;

obtaining, using said coherence transformer, a second copy of said second memory block from said computer node via said common bus;

invalidating, using said coherence transformer, any valid copy of said second memory block in said computer node; and

forwarding said second copy of said second memory block from said coherence transformer to said common bus to enable a progenitor of said second memory access request to obtain said second copy of said second memory block; and

unallocating said snoop tag of said snoop tag array, thereby rendering said snoop tag available for tracking other externally cached memory blocks and causing said second memory block to be no longer tracked by said snoop tag array.

14. The method of claim 11 wherein said second memory access request is a request for a shared copy and said snoop tag tracking said second memory block indicates that said first copy of said second memory block at said external device is a shared copy of said second memory block, said step of responding to said second memory access request using said snoop-only approach comprises:

obtaining, using said coherence transformer, a second copy of said second memory block from said computer node via said common bus; and

forwarding said second copy of said second memory block from said coherence transformer to said common bus to enable a progenitor of said second memory access request to obtain said second copy of said second memory block.

15. The method of claim 11 wherein said second memory access request is a request for a shared copy and said snoop tag tracking said second memory block indicates that said first copy of said second memory block at said external device is an exclusive copy of said second memory block, said step of responding to said second memory access request using said snoop-only approach comprises:

obtaining, using said coherence transformer, a second copy of said second memory block from said external device;

changing said snoop tag tracking said second memory block to indicate that said first copy of said memory block at said external device is a shared copy of said second memory block; and

forwarding said second copy of said second memory block from said coherence transformer to said common bus to enable a progenitor of said second memory access request to obtain said second copy of said second memory block.

16. In a computer system having a computer node which has a common bus, a method for enabling an external device in an external domain that is external to said computer node to share memory blocks having local physical addresses in a memory module at said computer node through a coherence transformer coupled to a common bus of said computer node irrespective whether said external device and said common bus both employ a common protocol and irrespective whether said external device and said common bus both operate at the same speed, each of said memory blocks having an associated Mtag for tracking a global state associated with said each of said memory block, including a global exclusive state for indicating that said each of said memory blocks is exclusive to said computer node, a global shared state for indicating that said each of said memory blocks is shared by said computer node with said external device, and a global invalid state for indicating that said each of said memory blocks is invalid in said computer node, said method comprising:

receiving, at said memory module via said common bus, a first memory access request for a valid copy of a first memory block of said memory blocks from a progenitor of said first memory access request, said progenitor being an entity different from said coherence transformer, said coherence transformer having a snoop tag array having a plurality of snoop tags, each of said plurality of snoop tags being configured to track an external state of a copy of one of said memory blocks if cached by said external device, said external state including one of an external exclusive state for indicating that said copy of one of said memory blocks is exclusive to said external domain, an external shared state for indicating that said copy of one of said memory blocks is shared by said external domain, and an external invalid state for indicating that said copy of one of said memory blocks is invalid in said external domain; and

if said first memory block is not tracked in said snoop tag array because said snoop tag array is full, responding to said first memory access request using an Mtag-only approach, including;

sending a first copy of said first memory block, along with a first Mtag corresponding to said first memory block, from said memory module to said progenitor of said first memory block,

examining, using said progenitor of said first memory block, said first Mtag state, and

if said first Mtag state is invalid, issuing a second memory access request pertaining to said first memory block from said progenitor of said first memory access request to request said coherence transformer to service said second memory access request, thereby enabling said progenitor of said first memory access request to obtain said valid copy of said first memory block.

17. The method of claim 16 further comprising:

if said first memory block is tracked in said snoop tag array, responding to said first memory access request using a snoop-only approach, including;

intervening to respond to said first memory access request, using said coherence transformer, thereby preventing said memory module from responding to said first memory access request.

18. The method of claim 17 wherein said step of responding to s aid first memory access request using said snoop-only approach includes:

obtaining a first copy of said first memory block, using said coherence transformer, from said external domain; and

forwarding said first copy of said first memory block from said coherence transformer to said progenitor of said first memory access request.

19. An apparatus for facilitating the sharing of memory blocks between a computer node and an external device, said memory blocks having local physical addresses at said computer node, each of said memory blocks have an associated Mtag for tracking a global state that is associated with said each of said memory blocks including a global exclusive state for indicating that said each of said memory blocks is exclusive to said computer node, a global shared state for indicating that said each of said memory blocks is shared by said computer node with said external device, and a global invalid state for indicating that said each of said memory blocks is invalid in said computer node, said method comprising:

snooping logic that is configured to be coupled with a common bus of said computer node, such that said snooping logic monitors memory access requests on said common bus when coupled to said common bus;

a snoop tag array coupled to said snooping logic, said snoop tag array having a plurality of snoop tags, each of said plurality of snoop tags a configured to track an external state of a copy of one of said memory blocks if cached by said external device, said external state including one of an external exclusive state for indicating that said copy of one of said memory blocks is exclusive to said external domain, an external shared state for indicating that said copy of one of said memory blocks is shared by said external domain, and an external invalid state for indicating that said copy of one of said memory blocks is invalid in said external domain; and

logic for ascertaining whether a first memory access request from said external device for a first memory block of said memory blocks should be responded to using a snoop-only approach or an Mtag-only approach when said snoop tag array is full, said snoop-only approach requires a tag in said snoop tag array to track said first memory block for an entire duration that said first memory block is cached by said external device, said Mtag-only approach only temporarily stores said first memory block until a global state associated with said first memory block can be written back into said memory module.

20. The apparatus of claim 19 further comprising logic for ascertaining whether a second memory access for a second memory block on said common bus should be responded to using said snoop-only approach or said Mtag-only approach, said second memory access is responded to by said coherence transformer using said snoop-only approach when said second memory block is tracked by said snoop tag array, said second memory block is responded to by said memory module using said Mtag-only approach when said second memory access is not tracked by said snoop tag array.