Messaging facility with hardware tail pointer and software implemented head pointer message queue for distributed memory massively parallel processing system

• US 5,581,705 A
• Filed: 12/13/1993
• Issued: 12/03/1996
• Est. Priority Date: 12/13/1993
• Status: Expired due to Term

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: CRAY Incorporated (Hewlett-Packard Enterprise Company)
• Inventors: Oberlin, Steven M., Passint, Randal S., Fromm, Eric C.
• Primary Examiner(s): Lee, Thomas C.
• Assistant Examiner(s): DINH, DUNG C
• Application Number:
  US08/166,443
• Time in Patent Office: 1,086 Days
• Field of Search: 395/200.14, 395/200.13, 395/800, 395/250
• US Class Current: 709/200
• CPC Class Codes: G06F 15/17381 Two dimensional, e.g. mesh, torus

Abstract

A messaging facility is described that enables the passing of packets of data from one processing element to another in a globally addressable, distributed memory multiprocessor without having an explicit destination address in the target processing element'"'"'s memory. The messaging facility can be used to accomplish a remote action by defining an opcode convention that permits one processor to send a message containing opcode, address and arguments to another. The destination processor, upon receiving the message after the arrival interrupt, can decode the opcode and perform the indicated action using the argument address and data. The messaging facility provides the primitives for the construction of an interprocessor communication protocol. Operating system communication and message-passing programming models can be accomplished using the messaging facility.

Claims

1. A messaging facility in a multiprocessor computer system having a plurality of processing elements interconnected by a n-dimensional interconnect network, each processing element including a processor and a local memory, wherein globally addressable portions of local memory of each processing element form a distributed memory, the messaging facility comprising:

assembling means in a source processing element for assembling a message to be sent from the source processing element to a destination processing element based on information provided from the source processing element'"'"'s processor;

a network router for transmitting the assembled message from the source processing element to the destination processing element via the interconnect network;

a message queue in a designated area of the local memory of the destination processing element for storing the transmitted message;

tail pointer hardware circuitry for indexing into the message queue to indicate a location where the transmitted message is to be stored in the message queue;

interrupt hardware circuitry in the destination processing element for providing an interrupt to the destination processing element'"'"'s processor in response to the message being stored in the message queue; and

message facility software providing a head pointer for indexing into the message queue to indicate a location where the message is stored in the message queue for reading by the destination processing element'"'"'s processor in response to the interrupt.

2. The messaging facility of claim 1 further comprising:

a queue limit counter to provide a limit count representative of the remaining available locations in the message queue.

3. The messaging facility of claim 2 further comprising:

means for generating an acknowledged message based on the limit count indicating that there is an available location for storing the transmitted message and for generating a not acknowledged message based on the limit count indicating that there are no available locations for storing the transmitted message.

4. The messaging facility of claim 2 further comprising:

means for causing the queue limit counter to increment the limit count as messages are read from the message queue; and

means for causing queue limit counter to decrement the limit count as messages are stored in the message queue.

5. The messaging facility of claim 3 wherein the source processing element includes a message queue and a queue limit counter and wherein the messaging facility further comprises:

means for causing the queue limit counter of the source processing element to decrement the limit count after the message is assembled in the assembling means to thereby reserve a location in the message queue of the source processing element for a not acknowledged message generated by the destination processing element in response to the transmitted message.

6. The messaging facility of claim 5 further comprising:

means for causing the queue limit counter of the source processing element to increment the limit count in response to receiving an acknowledged message from the destination processing element indicating that the destination processing element accepted the transmitted message.

7. The messaging facility of claim 1 wherein the tail pointer hardware circuitry circularly indexes the message queue.

8. The messaging facility of claim 7 wherein the message queue is a first-in, first-out circular buffer.

9. The messaging facility of claim 1 farther comprising:

means for detecting possible error conditions and for providing an error interrupt to the destination processing element'"'"'s processor in response to detecting an error condition.

10. The messaging facility of claim 9 wherein the means for detecting includes:

means for detecting error conditions resulting from a message being corrupted during the transmission from the source processing element to the destination processing element;

means for detecting that the message queue is full when the message is to be stored in the message queue; and

means for detecting that the message has been misrouted through the interconnect network and is at the wrong destination processing element.

11. The messaging facility of claim 9 wherein one type of possible error condition results from a message being corrupted during the transmission from the source processing element to the destination processing element and wherein the messaging facility further comprises:

means for responding to an error condition indicating that the transmitted message is corrupted to transform the corrupted message into an error message to be stored in the message queue for retrieval during software error recovery.

12. A method of passing messages in a multiprocessor computer system having a plurality of processing elements interconnected by a n-dimensional interconnect network, each processing element including a processor and a local memory, wherein globally addressable portions of local memory of each processing element from a distributed memory, the method comprising:

assembling a message to be sent from the source processing element to a destination processing element based on information provided from the source processing element'"'"'s processor;

transmitting the assembled message from the source processing element to the destination processing element via the interconnect network;

indexing, with hardware circuitry, into a designed area of the local memory of the destination processing element'"'"'s local memory to indicate a location where the transmitted message is to be stored;

storing the transmitted message in the indexed location in the designated area of the local memory of the destination processing element;

generating an interrupt, with hardware circuitry in the destination processing element to the destination processing element'"'"'s processor in response to the message being stored in the indexed location; and

indexing, with message facility software, into the designated area of the local memory of the destination processing element'"'"'s local memory to indicate a location where the message is stored for reading by the destination processing element'"'"'s processor in response to the interrupt.

13. The method of claim 12 further comprising the step of providing a limit count representative of the remaining available locations in the designated area of the local memory of the destination processing element.

14. The method of claim 13 further comprising the steps of generating an acknowledged message based on the limit count indicating that there is an available location for storing the transmitted message and generating a not acknowledged message based on the limit count indicating that there are no available locations for storing the transmitted message.

15. The method of claim 13 further comprising the steps of incrementing the limit count as messages are read from the designated area of the local memory of the destination processing element and decrementing the queue limit count as messages are stored in the designated area.

16. The method of claim 14 further comprising the steps of generating a second limit count representative of the remaining available locations in the designated area of the local memory of the source processing element, of decrementing the second limit count after the message is assembled in the buffer means to thereby reserve a location in the designated area of the local memory of the source processing element for a not acknowledged message generated by the destination processing clement in response to the transmitted message.

17. The method of claim 16 further comprising the steps of incrementing the second limit count in response to receiving an acknowledged message from the destination processing element indicating that the destination processing element accepted the transmitted message.

18. The method of claim 12 wherein the step of indexing into a designated area includes the step of circularly indexing into the designated area after indexing through a selected location in the designated area.

19. The method of claim 12 further comprising the steps of detecting possible error conditions and providing an error interrupt to the destination processing element'"'"'s processor in response to detecting an error condition.

20. The method of claim 19 wherein the step of detecting includes detecting error conditions resulting from a message being corrupted during the transmission from the source processing element to the destination processing element, detecting that the designated area is full when the message is to be stored in the designated area, and detecting that the message has been misrouted through the interconnect network and is at the wrong destination processing element.

21. The method of claim 19 wherein one type of possible error condition results from a message being corrupted during the transmission from the source processing element to the destination processing element and wherein the method further comprises the step of transforming the corrupted message into an error message to be stored in the message queue for retrieval during software error recovery.

22. A multiprocessor computer system comprising:

a plurality of processing elements, each processing element including a processor, a local memory having a designated message queue area, and shell circuitry, each shell circuitry including;

assembly means for assembling a message to be sent from a source processing element to a destination processing clement based on information provided from the source processing element'"'"'s processor,

a network router for transmitting the assembled messages from the source processing element to the destination processing element via the interconnect network to be stored in the designated message queue area of the local memory of the destination processing element,

tail pointer circuitry to index into the designated message queue area of the local memory of the destination processing element to indicate a location where the transmitted message is to be stored, and

interrupt circuitry providing an interrupt to the destination processing element'"'"'s processor in response to the message being stored in the designated message queue arae;

messaging facility software providing head pointers to indicate locations where messages are stored in the designated message queue areas of the local memories of the destination processing elements for reading by the destination processing element'"'"'s processors in response to the interrupts from the destination processing element'"'"'s interrupt circuitry; and

a n-dimensional interconnect network interconnecting the plurality of processing elements.