Message passing system for distributed shared memory multiprocessor system and message passing method using the same

• US 5,617,537 A
• Filed: 10/03/1994
• Issued: 04/01/1997
• Est. Priority Date: 10/05/1993
• Status: Expired due to Fees

• Current Assignee: Nippon Telegraph and Telephone Corporation
• Original Assignee: Nippon Telegraph and Telephone Corporation
• Inventors: Tanaka, Satoshi, Maruyama, Katsumi, Kubota, Minoru, Yamada, Shigeki
• Primary Examiner(s): Lim, Krisna
• Assistant Examiner(s): Najjar, Saleh
• Application Number:
  US08/317,647
• Time in Patent Office: 911 Days
• Field of Search: 395/650, 395/800, 395/480, 395/448, 395/200.01, 395/200.07, 395/200.08
• US Class Current: 709/214
• CPC Class Codes: G06F 15/17 using an input/output type connection, e.g. channel, I/O port

Abstract

In a multiprocessor system, each processor module comprises a processor, a distributed shared memory, a distributed memory coupler for controlling copying between distributed shared memories and a distributed memory protector for protecting said distributed shared memory against illegal access. The distributed shared memories are assigned global addresses common to all the processor modules, and the distributed shared memory of each processor module has its addresses shared with the distributed shared memory of each processor module which is the destinatiion of data transfer. Message buffers and message control areas on the distributed shared memory are divided into areas specified by a combination of sending and receiving processor modules. A processing request area on the distributed shared memory is divided corresponding to each receiving processor module and arranged accordingly. The processing request area on the receiver'"'"'s side distributed shared memory has a FIFO structure. The sender'"'"'s side distributed memory coupler stores identifying information of the destination processor module between the processor module communication and, upon occurrence of a write into the distributed shared memory, sends a write address and write data to the destination processor module. The receiver'"'"'s side distributed memory coupler copies the received write data into the distributed shared memory of the processor module to which the distributed shared memory coupler belongs, by receiving write address and write data from the sender'"'"'s side distributed memory coupler.

Claims

1. A multiprocessor system which has a plurality of processor modules which are interconnected via a channel for communication and each having unique identifying information and being capable of executing an application object, each one of said plurality of processor modules comprising:

a distributed shared memory which has addresses common to all of said processor modules and includes a plurality of communication areas for sending use which are dedicatedly assigned to respective ones of all possible combinations of one-to-one communication between said one of the plurality of processor modules at a sender'"'"'s side and each of said plurality of processor modules at a receiver'"'"'s side and a plurality of communication areas for receiving use which are dedicatedly assigned to respective ones of all possible combinations of point-to-point communication between said one of the plurality of processor modules at a receiver'"'"'s side and each of said processor modules at a sender'"'"'s side, each of said communications areas for sending use and for receiving use having message buffers each for writing therein a message to be transferred, each pair of sender'"'"'s side and receiver'"'"'s side processor modules defining point-to-point communication being assigned shared addresses of a dedicated communication area, by which said shared addresses are associated with said identifying information of said sender'"'"'s side and receiver'"'"'s side processor modules forming said each pair;

message buffer management means which responds to a request by said application object for writing a message into one of said communication areas corresponding to one of said processor modules designated as a receiver'"'"'s side processor module to acquire a vacant one of said message buffers in said communication area at said sender'"'"'s side corresponding to said receiver'"'"'s side processor module; and

distributed shared memory control means which, as said application object writes said message into said acquired vacant message buffer in said sender'"'"'s side communication area corresponding to said receiver'"'"'s side processor module, sends the address of said acquired message buffer and said message to said receiver'"'"'s side processor module specified by said application object to write said message into a receiver'"'"'s side message buffer of the same address shared with said acquired message buffer.

2. The system of claim 1, wherein said distributed shared memory control means of each of said processor modules has a data memory in which pieces of identifying information of all of said processor modules with which said each processor module communicates are prestored as a table in correspondence with their shared addresses, and said distributed shared memory control means sends the address of said acquired message buffer and said message to said receiver'"'"'s side processor module specified by the identifying information obtained by referring to said table in said data memory to write said message into said receiver'"'"'s side message buffer of the same address shared with said acquired message buffer.

3. The system of claim 1, wherein the identifying information of said sender'"'"'s side processor module and the identifying information of said receiver'"'"'s side processor module of said each pair defined by each of said shared addresses are defined in different bit positions of said each shared address.

4. The system of claim 1, wherein each of said communication areas of said distributed shared memory of said each processor module includes a control area for storing control data which controls a message transfer, said distributed shared memory control means responds to a write request to write said control data into an address shared between said sender'"'"'s side processor module and said receiver'"'"'s side processor module, and said receiver'"'"'s side processor module responds to a processing request from said sender'"'"'s side processor module to process said message written in said receiver'"'"'s side message buffer specified by said control data stored in said control area at the receiver'"'"'s side.

5. The system of claim 4, wherein said control area of said each communication area includes a descriptor ring for storing the address of said message buffer where to write a message to be transferred and execution right information which specifies the execution right of processing by said sender'"'"'s side and receiver'"'"'s side processor modules, said distributed shared memory control means responds to said write request to write said descriptor ring in the corresponding address of said distributed shared memory of said receiver'"'"'s side processor module, and said receiver'"'"'s side processor module responds to said processing request from said sender'"'"'s side processor module to process said message written in said receiver'"'"'s side message buffer specified by said descriptor ring.

6. The system of claim 4, wherein said message buffer management means of said each processor module includes a management map area provided in said control area of said each of said communication area, for indicating the state of use of each of said message buffers in said each communication area.

7. The system of claim 1, wherein said distributed shared memory of said each processor module has a processing request area exclusively for writing a processing request for each of said processor modules, and said processing request area for said processor module to which said distributed shared memory belongs has a FIFO structure.

8. The system of claim 1, 4, or 7, wherein said each processor module further comprises memory protection means including:

memory capability storage means for storing the access right for said each message buffer of said distributed shared memory;

current capability storage means for storing access right information for an application object being executed in said each processor module; and

capability comparison means for comparing the stored contents of said memory capability storage means and said current capability storage means to decide whether access to said each message buffer is legal.

9. A communication method using the multiprocessor system defined in claim 1 comprising the following steps:

a) said sender'"'"'s side processor module acquires, by said message buffer management means, a vacant sender'"'"'s side message buffer in said communication area corresponding to said receiver'"'"'s side processor module;

b) said sender'"'"'s side processor module writes a message in said acquired sender'"'"'s side message buffer;

c) said sender'"'"'s side distributed memory control means extracts the identifying information of said receiver'"'"'s side processor module on the basis of the address of said sender'"'"'s side message buffer and transfers the address of said sender'"'"'s side message buffer and write data on said message written therein to said receiver'"'"'s side processor module;

d) said receiver'"'"'s side distribution shared memory control means copies said message to that one of the receiver'"'"'s side message buffers in said receiver'"'"'s side distributed shared memory which has the same address as that of said sender'"'"'s side message buffer, on the basis of the address of said sender'"'"'s side message buffer and said write data; and

e) said receiver'"'"'s side processor module reads out said message from said receiver'"'"'s side message buffer.

10. The method of claim 9, wherein said distributed shared memory control means of each of said processor modules has a data memory in which pieces of identifying information of all of said processor modules with which said each processor module communicates are prestored as a table in correspondence with their shared addresses, and said distributed shared memory control means sends the address of said acquired message buffer and said message to said receiver'"'"'s side processor module specified by the identifying information obtained by referring to said table in said data memory to write said message into said receiver'"'"'s side message buffer of the same address shared with said acquired message buffer.

11. The method of claim 9, wherein the identifying information of said sender'"'"'s side processor module and the identifying information of said receiver'"'"'s side processor module of said each pair defined by each of said shared addresses are defined in different bit positions of said each shared address.

12. The method of claim 9, wherein each of said communication areas of said distributed shared memory of said each processor module includes a control area for storing control data which controls the transfer of said message, said method further comprising the following steps:

f) said sender'"'"'s side distributed shared memory control means extracts the identifying information of said receiver'"'"'s side processor module on the basis of the address of said control area and transfers the address of said control area and write data to said receiver'"'"'s side processor module;

g) said receiver'"'"'s side distributed shared memory control means copies said control data to the control area of said receiver'"'"'s side distributed shared memory which has its address shared with said sender'"'"'s side control area, on the basis of the address of said sender'"'"'s side control area and said write data;

h) said receiver'"'"'s side processor module reads out said control data from said receiver'"'"'s side control area, recognizes the contents of control on said message, executes the corresponding process and writes response control data in said receiver'"'"'s side control area;

i) said receiver'"'"'s side distributed shared control means takes out the identifying information of said sender'"'"'s side processor module on the basis of the address of said receiver'"'"'s side control area and transfers the address of said receiver'"'"'s side control area and write data on said response control data to said sender'"'"'s side processor module;

j) said senders side distributed shared memory control means copies said control data to said sender'"'"'s side control area of said sender'"'"'s side distributed shared memory which has its address shared with said receiver'"'"'s side control area, on the basis of the address of said receiver'"'"'s side control area and said response control data received from said receiver'"'"'s side distributed shared memory control means; and

k) said senders side processor module reads out said response control data from said sender'"'"'s side control area and recognizes the results of processing at the receiver'"'"'s side, thereby implementing a bidirectional communication between said processor modules.

13. The method of claim 12, wherein said sender'"'"'s side and receiver'"'"'s side control areas each includes a descriptor ring area for writing thereinto the address of said message buffer where said message to be transferred is written and enqueue data indicating the transfer of the process execution right from said sender'"'"'s side to said receiver'"'"'s side;

said step f) includes a step wherein said sender'"'"'s side distributed shared memory control means writes said enqueue data in said descriptor ring area to cause it to copy said enqueue data to said receiver'"'"'s side descriptor ring area having its address shared with said sender'"'"'s side descriptor ring area; and

said step i) includes a step wherein dequeue data indicating the return of the process execution right from said receiver'"'"'s side to said sender'"'"'s side is written in said receiver'"'"'s side descriptor ring area to cause it to copy said dequeue data to said sender'"'"'s side descriptor ring area having its address shared with said receiver'"'"'s side descriptor ring area.

14. The method of claim 12, wherein said control area of each of said communication areas includes a message buffer management map area corresponding to one of said message buffers;

said step a) includes a step wherein said sender'"'"'s side distributed shared memory control means acquires a vacant one of said message buffers and writes data indicating the occupied state of said acquired message buffer in the corresponding message buffer management map area to cause it to copy said occupied state data to the message buffer management map area of said receiver'"'"'s side distributed shared memory which has its address shared with said sender'"'"'s side message buffer management map area; and

said step i) includes a step wherein when an application object at said receiver'"'"'s side completes the readout of said message from said receiver'"'"'s side message buffer, said receiver'"'"'s side distributed shared memory control means writes data indicating the vacant state of said receiver'"'"'s side message buffer in the corresponding message buffer management map area to cause it to copy said vacant state data to the message buffer management map area of said sender'"'"'s side distributed shared memory which has its address shared with said receiver'"'"'s side message buffer management map area.

15. The method of claim 9, wherein said distributed shared memory of said each processor module has a processing request area exclusively for writing a processing request for each of said processor modules, and said processing request area for said processor module to which said distributed shared memory belongs has a FIFO structure;

said method further comprising the steps;

wherein when said plurality of sender'"'"'s side processor modules send processing requests to one of said receiver'"'"'s side processor modules, said sender'"'"'s side processor modules each write processing request data in an area on said sender'"'"'s side distributed shared memory for writing processing requests to said one receiver'"'"'s side processor module;

wherein said each sender'"'"'s side distributed shared memory control means takes out the identifying information of said receiver'"'"'s side processor module on the basis of the address of said processing request area and transfers said processing revest area address and write data to said one receiver'"'"'s side processor module;

wherein, on the basis of said processing request are address and said write data transferred thereto, said receiver'"'"'s side distributed shared memory control means writes said processing request data in said processing request area of said FIFO structure which has its address shared with said processing request area at said sender'"'"'s side, thereby storing in said FIFO-structured processing request area a plurality of pieces of processing request data in order of arrival; and

wherein said one receiver'"'"'s side processor module detects said processing requests by reading out said plurality of pieces of processing request data from said FIFO-structured processing request area on said receiver'"'"'s side distributed shared memory in a sequential order.

16. The method of claim 9, 12, or 15, wherein said each processor module further comprises memory protection means including:

memory capability storage means for storing the access right for said each message buffer of said distributed shared memory;

current capability storage means for storing access right information for an application object being executed in said each processor module; and

capability comparison means for comparing the contents of said memory capability storage means and said current capability storage means to decide whether access to said each message buffer is legal;

when a sender object which is executed by one of said processor modules transfers a message to a receiver application object which is executed by a different one of said processor modules, said method performs the steps;

wherein said receiver'"'"'s side processor module acquires a sender'"'"'s side message buffer on said distributed shared memory in said sender'"'"'s side processor module and sets said access right information in said current capability storage means and said memory capability storage means in said sender'"'"'s side processor module corresponding to said sender'"'"'s side message buffer, and upon each occurrence of memory access from a processor in said sender'"'"'s side processor module to said sender'"'"'s side message buffer on said distributed shared memory thereof, said capability comparison means compares the contents of said memory capability storage means and said current capability storage means to make a check to determine if said memory access is legal; and

wherein when the message written in said sender'"'"'s side message buffer is transferred to said distributed shared memory of said receiver'"'"'s side processor module, said receiver'"'"'s side processor module sets said access right information in said current capability storage means and said memory capability storage means corresponding to said receiver'"'"'s side message buffer, and upon each occurrence of memory access from said processor in said receiver'"'"'s side processor module to said receiver'"'"'s side message buffer on said receiver'"'"'s side distributed shared memory, said capability comparison means compares the contents of said memory capability storage means and said current capability storage means to make a check to determine if said memory access is legal.

17. The method of claim 16, wherein when a sender application object which is executed by one of said sender'"'"'s side processor modules transfers a message to a receiver application object which is executed by a different one of said processor modules, said method performs the steps:

wherein said sender'"'"'s side processor module transfers said access right information from said sender'"'"'s side to said receiver'"'"'s side while holding it in one of a message buffer and message control area on said sender'"'"'s side distributed shared memory; and

wherein said receiver'"'"'s side processor module receives said access right information from said sender'"'"'s side by reading it out from one of a message buffer and message control area on said receiver'"'"'s side distributed shared memory corresponding to said one of the message buffer and the message control area on said sender'"'"'s side distributed shared memory.

18. The method of claim 9, wherein when said sender'"'"'s side processor module and said receiver'"'"'s side processor module are the same, said receiver'"'"'s side processor module reads out a message directly from said sender'"'"'s side processor module to thereby implement message communication in the same processor module.

19. The method of claim 18, wherein said each communication area of said distributed shared memory of said each processor module includes a descriptor ring area in which the address of said message buffer for writing a message to be transferred and execution right information specifying the execution right for processing by said sender'"'"'s side and receiver'"'"'s side processor modules are stored in correspondence with each other, said method further comprising the step:

wherein said receiver'"'"'s side processor module reads out control data directly from said sender'"'"'s side control area, recognizes the contents of message control, executes the corresponding process and writes response control data in said sender'"'"'s side control area, and said sender'"'"'s side processor module reads out said response control data from said sender'"'"'s side control area and recognizes the result of processing at the receiver'"'"'s side, thereby implementing a bidirectional communication in the same processor module.

20. The method of claim 18, wherein said control area of said each communication area includes a message buffer management map area corresponding to each of said message buffers;

said step a) includes a step wherein said sender'"'"'s side distributed shared memory control means acquires a vacant one of said message buffers and writes data indicating the occupied state of said acquired message buffer in the corresponding message buffer management map area; and

another step is included wherein when an application object at said sender'"'"'s side completes the read out of said message from said receiver'"'"'s side message buffer, said receiver'"'"'s side distributed shared memory control means writes data indicating the vacant state of said receiver'"'"'s side message buffer in the corresponding message buffer management map area.

21. The method of claim 18, wherein said distributed shared memory of said each processor module has a processing request area exclusively for writing a processing request for each of said processor modules, and said processing request area for said processor module to which said distributed shared memory belongs has a FIFO structure;

said method further comprising the steps;

wherein when said plurality of sender'"'"'s side processor modules send processing requests to one of said receiver'"'"'s side processor modules, each of said sender'"'"'s side processor modules writes processing request data in said processing request area on the sender'"'"'s side distributed shared memory destined for said receiver'"'"'s side processor module thereby storing in said FIFO structured processing request area of said receiver'"'"'s side distributed shared memory a plurality of pieces of processing request data in order of arrival; and

wherein said one receiver'"'"'s side processor module detects said processing requests by reading out said plurality of pieces of processing request data from said FIFO-structured processing request area on said receiver'"'"'s side distributed shared memory in a sequential order.

22. The method of claim 9, wherein said receiver'"'"'s side processor module is a sending relay processor having a network adaptor connected to another multiprocessor system via a communication network, said method further comprising the steps:

f) wherein when the identifying information of a receiver object contained in said received message is identifying information of an application object belonging to said another processor system, said sending relay processor module enqueues an adaptor sending descriptor for said received message;

g) wherein said network adaptor detects said enqueued adaptor sending descriptor, reads out said message from said message buffer of said relay processor module and provides it on said communication network, using as its destination the identifying information of said receiver object indicated in said message;

h) wherein a network adaptor of a third processor module of the multiprocessor system specified by said identifying information of said receiver object stores said received message in a local buffer and enqueues an adaptor receiving descriptor; and

i) wherein, upon detection of said enqueued adaptor receiving descriptor, said third processor module makes a check to determine if the destination of said message stored in said message buffer is said third processor module itself, and processes said message in accordance with the result of said check.

23. The method of claim 22, wherein said step i) is a step wherein when the destination of said message is another processor module other than said third one, a vacant message buffer in the distributed shared memory corresponding to said destination processor module is acquired, said message is written in said vacant message buffer, and at the same time, said message is copied to the corresponding message buffer of said distributed shared memory of said destination processor module by said distributed shared memory control means.

24. The method of claim 9, wherein said receiver'"'"'s side processor module is a sending relay processor module having a network adaptor connected to another multiprocessor system via a communication network, said method further comprising the steps:

f) wherein when the identifying information of a receiver object contained in said received message is identifying information of another multiprocessor system, said sending relay processor module enqueues an adaptor sending descriptor for said received message;

g) wherein said network adaptor detects said enqueued adaptor sending descriptor, reads out said message from said message buffer and provides it onto said communication network, using as its destination the identifying information of said receiver object indicated in said received message; and

h) wherein a network adaptor of a third processor module of the multiprocessor system specified by said identifying information of said receiver object writes said message directly into a communication area in the distributed shared memory corresponding to the processor module of the destination specified by the identifying information of said receiver object indicated in said message.