Efficient region coherence protocol for clustered shared-memory multiprocessor systems

• US 8,397,030 B2
• Filed: 06/24/2008
• Issued: 03/12/2013
• Est. Priority Date: 06/24/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cantin, Jason F., Kunkel, Steven R.
• Primary Examiner(s): NGUYEN, THAN VINH
• Application Number:
  US12/144,759
• Publication Number:
  US 20090319726A1
• Time in Patent Office: 1,722 Days
• Field of Search: None
• US Class Current: 711/141
• CPC Class Codes: G06F 12/0822 Copy directories local copy tags for implementing a bus snooping protocol G06F12/0831
• CPC Class Codes: G06F 12/0833 in combination with broadcast means (e.g. for invalidation or updating)

Abstract

A system and method of a region coherence protocol for use in Region Coherence Arrays (RCAs) deployed in clustered shared-memory multiprocessor systems which optimize cache-to-cache transfers by allowing broadcast memory requests to be provided to only a portion of a clustered shared-memory multiprocessor system. Interconnect hierarchy levels can be devised for logical groups of processors, processors on the same chip, processors on chips aggregated into a multichip module, multichip modules on the same printed circuit board, and for processors on other printed circuit boards or in other cabinets. The present region coherence protocol includes, for example, one bit per level of interconnect hierarchy, such that the one bit has a value of “1” to indicate that there may be processors caching copies of lines from the region at that level of the interconnect hierarchy, and the one bit has a value of “0” to indicate that there are no cached copies of any lines from the region at that respective level of the interconnect hierarchy.

Claims

1. A method of controlling region coherence in a clustered shared-memory multiprocessor system, the method comprising:

generating a request by a processor for a line of data storable in a system memory;

determining, via examination of at least one entry in a plurality of entries of a region coherence array, at least one level of a multi-level interconnect hierarchy which has recently cached at least one line of data of a region of the system memory, such that the region includes the requested line of data, wherein each entry in the region coherence array has one or more of a valid bit, one or more parity bits, a region address tag, and a plurality of line-count bits, and a non-zero bit, and wherein each entry in the region coherence array also has a region coherence state field, the region coherence state field having one region coherence state bit per level of the multi-level interconnect hierarchy; and

updating the at least one entry in the region coherence array associated with the requesting processor responsive to determining the at least one level of the multi-level interconnect hierarchy.

2. The method of claim 1, further comprising:

determining whether a processor of the determined at least one level of the multi-level interconnect hierarchy has recently cached the requested line of data.

3. The method of claim 1, wherein the request for a line of data is generated by a processor and provided only to processors associated with a same SMP node as the requesting processor.

4. The method of claim 1, wherein the request for a line of data is generated by a processor and provided only to processors associated with a different SMP node than the SMP node of the requesting processor.

5. The method of claim 1, wherein:

the determining of the at least one level of a multi-level interconnect hierarchy includes examination of at least one region coherence state bit; and

the updating of the region coherence array includes setting the region coherence state bit corresponding to the at least one level of the multi-level interconnect hierarchy that includes the requested line of data based on a region snoop response.

6. The method of claim 1, wherein the updating of the region coherence array includes setting the region coherence state bit corresponding to the at least one level of the multi-level interconnect hierarchy that includes the requested line of data.

7. The method of claim 1, wherein each level in the multi-level interconnect hierarchy comprises at least one of:

a group of processors;

a group of processors on a same chip;

a group of processors on a same node;

a group of processors on one or more chips aggregated into a multichip module;

a group of processors on one or more multichip modules on a same printed circuit board;

a group of processors on other printed circuit boards; and

a group of processors in one or more physical cabinets.

8. A region coherence protocol control system, comprising:

a clustered shared-memory multiprocessor system, including;

a processor included within a multi-level interconnect hierarchy of the multiprocessor system;

a region coherence array associated with the processor to provide region coherence status of a region of the system memory, such that the region includes the requested line of data; and

a system memory having stored thereon instructions, that when executed by the processor;

generate a request for a line of data storable in a system memory;

determine via examination of at least one entry in a plurality of entries of the region coherence array at least one level of a multi-level interconnect hierarchy which has recently cached at least one line of data of a region of the system memory, such that the region includes the requested line of data, wherein each entry in the region coherence array has one or more of a valid bit, one or more parity bits, a region address tag, and a plurality of line-count bits, and a non-zero bit, and wherein each entry in the region coherence array also has a region coherence state field, the region coherence state field having one region coherence state bit per level of the multi-level interconnect hierarchy; and

update the at least one entry in the region coherence array associated with the requesting processor responsive to determining the at least one level of the multi-level interconnect hierarchy.

9. The region coherence protocol control system of claim 8, wherein the request for a line of data is generated by a processor and provided only to processors associated with a same SMP node as the requesting processor.

10. The region coherence protocol control system of claim 8, wherein the request for a line of data is generated by a processor and provided only to processors associated with a different SMP node than the SMP node of the requesting processor.

11. The region coherence protocol control system of claim 8, wherein:

the region coherence array is updated based on an examination of at least one line count bit; and

the updating of the region coherence array includes setting the least one region coherence state bit corresponding to the at least one level of the multi-level interconnect hierarchy that includes the requested line of data based on a region snoop response.

12. The region coherence protocol control system of claim 8, wherein the updating of the region coherence array includes setting the region coherence state bit corresponding to the at least one level of the multi-level interconnect hierarchy that includes the requested line of data.

13. A computer program product, comprising:

a tangible computer storage medium; and

program code embodied in said tangible computer storage medium that when executed by a computer provides functionality, including;

generating a request by a processor for a line of data storable in a system memory;

determining via examination of at least one entry in a plurality of entries of a region coherence array at least one level of a multi-level interconnect hierarchy which has recently cached at least one line of data of a region of the system memory, such that the region includes the requested line of data, wherein each entry in the region coherence array has one or more of a valid bit, one or more parity bits, a region address tag, and a plurality of line-count bits, and a non-zero bit, and wherein each entry in the region coherence array also has a region coherence state field, the region coherence state field having one region coherence state bit per level of the multi-level interconnect hierarchy; and

updating the at least one entry in the region coherence array associated with the requesting processor responsive to determining the at least one determined level of the multi-level interconnect hierarchy.

14. The computer program product of claim 13, wherein each level in the multi-level interconnect hierarchy comprises at least one of:

a group of processors;

a group of processors on a same chip;

a group of processors on a same node;

a group of processors on one or more chips aggregated into a multichip module;

a group of processors on one or more multichip modules on a same printed circuit board;

a group of processors on other printed circuit boards; and

a group of processors in one or more physical cabinets.

15. The computer program product of claim 14, wherein the requesting processor determines whether the indicated at least one level of the multi-level interconnect hierarchy includes a processor that has recently cached the requested line of data.

16. The computer program product of claim 13, further comprising the function of:

determining whether a processor of the determined at least one level of the multi-level interconnect hierarchy has recently cached the requested line of data.

17. The computer program product of claim 13, wherein the request for a line of data is generated by a processor and provided only to processors associated with a same SMP node as the requesting processor.

18. The computer program product of claim 13, wherein the request for a line of data is generated by a processor and provided only to processors associated with a different SMP node than the SMP node of the requesting processor.

19. The computer program product of claim 13, wherein:

the determining of the at least one level of a multi-level interconnect hierarchy includes examination of at least one region coherence state bit; and

the updating of the region coherence array includes setting the region coherence state bit corresponding to the at least one level of the multi-level interconnect hierarchy that includes the requested line of data based on a region snoop response.

20. The computer program product of claim 13, wherein each level in the multi-level interconnect hierarchy comprises at least one of:

a group of processors;

a group of processors on a same chip;

a group of processors on a same node;

a group of processors on one or more chips aggregated into a multichip module;

a group of processors on one or more multichip modules on a same printed circuit board;

a group of processors on other printed circuit boards; and

a group of processors in one or more physical cabinets.