Variable-grained memory sharing for clusters of symmetric multi-processors using private and shared state tables

• US 5,950,228 A
• Filed: 02/03/1997
• Issued: 09/07/1999
• Est. Priority Date: 02/03/1997
• Status: Expired due to Term

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Digital Equipment Corporation (HP Inc.)
• Inventors: Aggarwal, Anshu, Scales, Daniel J., Gharachorloo, Kourosh
• Primary Examiner(s): Swann, Tod R.
• Assistant Examiner(s): Langjahr, David
• Application Number:
  US08/794,172
• Time in Patent Office: 946 Days
• Field of Search: 711/122, 711/148, 395/800.06, 395/825
• US Class Current: 711/148
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 12/0817 using directory methods
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: H04L 47/6225 Fixed service order, e.g. Round Robin
• CPC Class Codes: H04L 49/90 Buffering arrangements
• CPC Class Codes: H04L 49/9052 with buffers of different sizes

Abstract

In a distributed shared memory system, clusters of symmetric multi-processors are connected to each other by a network. Each symmetric multi-processor includes a plurality of processors, a memory having addresses, and an input/output interface to interconnect the processors. A software implemented method enables data sharing between the clusters of symmetric multi-processors using variable sized quantities of data called blocks. A set of the addresses of the memories are designated as virtual shared addresses to store shared data, and a portion of the virtual shared addresses are allocated to store a shared data structure as one or more blocks. The size of a particular allocated block can vary for different shared data structures. Each block includes an integer number of lines, and each line includes a predetermined number of bytes of shared data. Directory information of a particular block is stored in the memory of a processor designed as the home of the block. The directory information includes the size of the particular block, the identity of the processor that last modified the data in the particular block and the identity of all processors having a copy of the block.

Claims

1. In a computer system including a plurality of symmetric multi-processors, each symmetric multi-processor including a plurality of processors, a memory having addresses, and an input/output interface connected to each other by a bus, the input/output interfaces connecting the symmetric multi-processors to each other by a network, a software implemented method for sharing access to data stored in the memories of the symmetric multi-processors, comprising the steps of:

designating a set of the addresses of the memories as virtual shared addresses to store shared data;

allocating a portion of the virtual shared addresses to store a shared data structure as one or more blocks accessible by programs executing in any of the processors, the size of a particular allocated block to vary with the size of the shared data structure, each block including an integer number of lines, each line including a predetermined number of bytes of shared data;

maintaining a shared state table including a plurality of shared state entries, there being one shared table entry for each line of the one or more blocks, each shared state entry to indicate a possible state of the line, the possible states being invalid, shared, exclusive, and pending; and

maintaining a private state table for each processor of the plurality of symmetric multi-processors, each private state table having a plurality of private state entries, the private state table entries of a particular private state table to indicate a possible state of a particular line accessed by the associated particular processor;

storing directory information of a particular block of the shared data structure in the memory of a home processor, the directory information including the size of the particular block;

instrumenting the programs with instructions which access the shared data to check whether the data are available; and

in response to receiving an access request from a requesting one of the processors to access the shared data, sending a particular block including the particular line and the size of the particular block to the requesting processor via the network to enable the processors to exchange shared data structures stored in variable sized blocks via the network.

2. The method of claim 1 further comprising:

storing the directory information in a directory maintained by the home processor, the directory including an entry for each line of the one or more blocks of the shared data structure, each entry including the size the particular block including the line.

3. The method of claim 2 further comprising:

maintaining, in the entry for each line of the particular block, an identity of a controlling one of the processors, the controlling processor last having an exclusive copy of the particular block including the particular line.

4. The method of claim 3 further comprising:

maintaining, in the entry, a bit vector, the bit vector including one bit for each processor, each bit to indicate whether a corresponding processor has a shared copy of the particular block.

5. The method of claim 1 further comprising:

dynamically changing the size of the one or more blocks allocated for the shared data structure while the programs are executing.

6. The method of claim 1 further comprising:

locking the shared state table before modifying one of the shared table entries.

7. The method of claim 6 further comprising:

setting the state of each line of the one or more blocks to invalid before dynamically changing the size of the one or more blocks.

8. The method of claim 7 further comprising:

modifying one of the private state tables only by the processor associated with the private state table.

9. The method of claim 8 further comprising:

selectively sending a message from a particular one of the processors of a particular symmetric multi-processor to other processors of the particular symmetric multi-processor when downgrading states in the private state table associated with the particular processor.

10. The method of claim 1 wherein the number of lines of the one or more blocks of a first shared data structure are different than the number of lines of the one or more blocks of a second data structure.

11. The method of claim 1 wherein the number of bytes in one of the lines of the first data structure in one program are different than the number of bytes in one of the lines of a second data structure in another program.

12. The method of claim 1, wherein the designating step includes a private data section used to store data structures for use by a single local processor.

13. The method of claim 1, wherein the designating step provides a state table which includes the shared state table and the private state table.

14. The method of claim 13, wherein the state table includes exclusion tables as a part thereof, the exclusion tables including a shared portion and a private portion.

15. A system comprising:

a network;

a plurality of symmetric multi-processors interconnected by the network, each symmetric multi-processor including a plurality of processors; and

a memory having a layout of addresses for each symmetric multi-processor, each memory address having a designated set of virtual shared addresses to store shared data, a portion of the virtual shared addresses storing a shared data structure as one or more blocks accessible by programs executing in any of the processors, the size of a particular allocated block varying with a size of the shared data structure, each block including an integer number of lines, each line including a predetermined number of bytes of shared data;

wherein the layout includes;

i) a shared state table including a plurality of shared state entries, one shared entry for each line of the one or more entries, each shared entry indicating a possible state of the line, the possible states being invalid, shared, exclusive and pending;

ii) a private state table for each processor of the plurality of symmetric multi-processors, each private state table having a plurality of private state entries, the private state entries of a particular private state table indicating a possible state of a particular line accessed by the associated particular processor;

wherein the shared data is accessed to check whether the data are available and a particular block is sent to a requesting processor from one other processor via the network to exchange shared data structures stored in variable sized blocks.

16. The system of claim 15, wherein the private state table includes an exclusion table.

17. The system of claim 16, wherein the exclusion table includes a shared portion and a private portion.

18. The system of claim 15, wherein the shared state table is capable of being locked before one of the shared table entries is modified.

19. The system of claim 15, wherein the private state table of a particular processor is capable of modification only by that processor associated with the private state table.

20. The system of claim 15, wherein one of the processors is designated as a home processor for storing directory information of a particular block of the shared data structure in the memory, the directory information including the size of the block.