PROVIDING IMPROVED MESSAGE HANDLING PERFORMANCE IN COMPUTER SYSTEMS UTILIZING SHARED NETWORK DEVICES

• US 20100082788A1
• Filed: 09/29/2008
• Published: 04/01/2010
• Est. Priority Date: 09/29/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Mundy, Michael Basil
• Granted Patent: US 8,166,146 B2
• Time in Patent Office: Days
• US Class Current: 709/223
• CPC Class Codes: G06F 9/52 Program synchronisation; Mutual exclusion, e.g. by means of semaphores
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes

Abstract

In a massively parallel computer system embodiment, when receiving a message at a compute node from an input/output node, the compute node performs the steps of: obtaining a lock on a collective network device; checking a shared storage location for a message pending for a thread; if such a message is pending, receiving the message'"'"'s remaining packets directly to a user'"'"'s buffer, unlocking, and returning; if no such message is pending, receiving one packet from the network device; if the packet indicates that the message is for the thread, receiving the message'"'"'s remaining packets directly to the user'"'"'s buffer, unlocking, and returning; and if the packet indicates that the message is for another thread, updating the shared storage location with a thread id of the other thread, unlocking, waiting for a time out, locking, and repeating from the checking step. Accordingly, data copying is eliminated with an attendant performance benefit.

Claims

1. A parallel computer system, comprising:

an input/output (I/O) node;

a plurality of compute nodes coupled to each other and to the I/O node via a collective network, each compute node comprising;

a compute logic block having a plurality of processors, wherein one of the processors runs a first thread;

a memory array block shared by the processors;

a network logic block having one or more communication blocks, wherein at least one of the communication blocks comprises a collective network device for facilitating communication of messages between the compute node and the I/O node, each message comprising a plurality of packets;

wherein when receiving a message at the compute node, the compute node performs the steps of;

(a) obtaining a lock on the network device;

(b) checking a shared storage location of the memory array block to see if a message is pending for the first thread;

(c) if a message is pending for the first thread based on the checking step (b), receiving the remaining packets in the message directly to a user'"'"'s buffer, unlocking the network device, and returning;

(d) if no message is pending for the first thread based on the checking step (b), receiving at least one packet of a message from the network device;

(e) if the at least one packet received in step (d) indicates that the message is for the first thread, receiving the remaining packets in the message directly to the user'"'"'s buffer, unlocking the network device, and returning;

(f) if the at least one packet received in step (d) indicates that the message is for a thread other than the first thread, updating the shared storage location of the memory array block with a thread id of the other thread, unlocking the network device, waiting for a time out to expire, obtaining a lock on the network device, and repeating from the checking step (b).

2. The parallel computer system as recited in claim 1, wherein the collective network has a point-to-point mode that allows messages to be sent to a specific node in the collective network, and wherein when sending a message to one of the compute nodes from the I/O node, all of the packets in the message are sent together so a complete message with the packets in order is delivered to the compute node.

3. The parallel computer system as recited in claim 2, wherein each of the messages has a one packet header that includes a thread id identifying a thread to which the message is to be delivered, wherein step (b) comprises checking the shared storage location of the memory array block for a one packet header containing a thread id identifying the first thread, and wherein step (d) comprises receiving a one packet header of the message from the network device.

4. The parallel computer system as recited in claim 1, wherein an I/O node daemon runs on the I/O node, wherein a compute node kernel (CNK) runs on each of the processors, wherein a first one of the processors runs a main process of an application program in a symmetric multi-processing (SMP) mode, wherein the first thread is spawned to run on a second one of the processors by the application program'"'"'s main process running on the first processor, wherein steps (a)-(f) are performed by the CNK running on the second processor, and wherein a third one of the processors runs a second thread spawned by the application program'"'"'s main process running on the first processor.

5. The parallel computer system as recited in claim 1, wherein an I/O node daemon runs on the I/O node, wherein a compute node kernel (CNK) runs on each of the processors, wherein a first one and a second one of the processors each runs a main process of an application program in a dual mode, wherein the first thread is spawned to run on a third one of the processors by the application program'"'"'s main process running on the first processor, wherein steps (a)-(f) are performed by the CNK running on the third processor, and wherein a fourth one of the processors runs a second thread spawned by the application program'"'"'s main process running on the second processor.

6. The parallel computer system as recited in claim 1, wherein the network logic block includes a JTAG communication block, a torus communication block, a tree communication block, a barrier communication block and an Ethernet communication block, and wherein the network device comprises the tree communication block.

7. A computer-implemented method for providing improved message handling performance in a parallel computer system utilizing a shared network device, wherein the parallel computer system comprises an input/output (I/O) node and a plurality of compute nodes coupled to each other and to the I/O node via a collective network, each compute node comprises:

a compute logic block having a plurality of processors, wherein one of the processors runs a first thread;

a memory array block shared by the processors;

a network logic block having a collective network device for facilitating communication of messages between the compute node and the I/O node, each message comprising a plurality of packets;

wherein when receiving a message at the compute node, the compute node performs the computer-implemented method comprising the steps of;

(a) obtaining a lock on the network device;

(b) checking a shared storage location of the memory array block to see if a message is pending for the first process;

(c) if a message is pending for the first thread based on the checking step (b), receiving the remaining packets in the message directly to a user'"'"'s buffer, unlocking the network device, and returning;

(d) if no message is pending for the first thread based on the checking step (b), receiving at least one packet of a message from the network device;

(e) if the at least one packet received in step (d) indicates that the message is for the first thread, receiving the remaining packets in the message directly to the user'"'"'s buffer, unlocking the network device, and returning;

(f) if the at least one packet received in step (d) indicates that the message is for a thread other than the first thread, updating the shared storage location of the memory array block with a thread id of the other thread, unlocking the network device, waiting for a time out to expire, obtaining a lock on the network device, and repeating from the checking step (b).

8. The computer-implemented method as recited in claim 7, wherein the collective network has a point-to-point mode that allows messages to be sent to a specific node in the collective network, and wherein when sending a message to one of the compute nodes from the I/O node, all of the packets in the message are sent together so a complete message with the packets in order is delivered to the compute node.

9. The computer-implemented method as recited in claim 8, wherein each of the messages has a one packet header that includes a thread id identifying a thread to which the message is to be delivered, wherein step (b) comprises checking the shared storage location of the memory array block for a one packet header containing a thread id identifying the first thread, and wherein step (d) comprises receiving a one packet header of the message from the network device.

10. The computer-implemented method as recited in claim 7, wherein an I/O node daemon runs on the I/O node, wherein a compute node kernel (CNK) runs on each of the processors, wherein a first one of the processors runs a main process of an application program in a symmetric multi-processing (SMP) mode, wherein the first thread is spawned to run on a second one of the processors by the application program'"'"'s main process running on the first processor, wherein steps (a)-(f) are performed by the CNK running on the second processor, and wherein a third one of the processors runs a second thread spawned by the application program'"'"'s main process running on the first processor.

11. The computer-implemented method as recited in claim 7, wherein an I/O node daemon runs on the I/O node, wherein a compute node kernel (CNK) runs on each of the processors, wherein a first one and a second one of the processors each runs a main process of an application program in a dual mode, wherein the first thread is spawned to run on a third one of the processors by the application program'"'"'s main process running on the first processor, wherein steps (a)-(f) are performed by the CNK running on the third processor, and wherein a fourth one of the processors runs a second thread spawned by the application program'"'"'s main process running on the second processor.

12. A computer readable medium for providing improved message handling performance in a parallel computer system utilizing a shared network device, wherein the parallel computer system comprises an input/output (I/O) node and a plurality of compute nodes coupled to each other and to the I/O node via a collective network, each compute node comprises:

a compute logic block having a plurality of processors, wherein one of the processors runs a first thread;

a memory array block shared by the processors; and

a network logic block having a collective network device for facilitating communication of messages between the compute node and the I/O node, each message comprising a plurality of packets;

the computer readable medium comprising instructions that when executed by one or more of the processors of the compute node cause the compute node when receiving a message to perform the steps of;

(a) obtaining a lock on the network device;

(b) checking a shared storage location of the memory array block to see if a message is pending for the first thread;

(c) if a message is pending for the first thread based on the checking step (b), receiving the remaining packets in the message directly to a user'"'"'s buffer, unlocking the network device, and returning;

(d) if no message is pending for the first thread based on the checking step (b), receiving at least one packet of a message from the network device;

(e) if the at least one packet received in step (d) indicates that the message is for the first thread, receiving the remaining packets in the message directly to the user'"'"'s buffer, unlocking the network device, and returning;

(f) if the at least one packet received in step (d) indicates that the message is for a thread other than the first thread, updating the shared storage location of the memory array block with a thread id of the other thread, unlocking the network device, waiting for a time out to expire, obtaining a lock on the network device, and repeating from the checking step (b).

13. The computer readable medium as recited in claim 12, wherein the collective network has a point-to-point mode that allows messages to be sent to a specific node in the collective network, and wherein when sending a message to one of the compute nodes from the I/O node, all of the packets in the message are sent together so a complete message with the packets in order is delivered to the compute node.

14. The computer readable medium as recited in claim 13, wherein each of the messages has a one packet header that includes a thread id identifying a thread to which the message is to be delivered, wherein step (b) comprises checking the shared storage location of the memory array block for a one packet header containing a thread id identifying the first thread, and wherein step (d) comprises receiving a one packet header of the message from the network device.

15. The computer readable medium as recited in claim 12, wherein an I/O node daemon runs on the I/O node, wherein a compute node kernel (CNK) runs on each of the processors, wherein a first one of the processors runs a main process of an application program in a symmetric multi-processing (SMP) mode, wherein the first thread is spawned to run on a second one of the processors by the application program'"'"'s main process running on the first processor, wherein steps (a)-(f) are performed by the CNK running on the second processor, and wherein a third one of the processors runs a second thread spawned by the application program'"'"'s main process running on the first processor.

16. The computer readable medium as recited in claim 12, wherein an I/O node daemon runs on the I/O node, wherein a compute node kernel (CNK) runs on each of the processors, wherein a first one and a second one of the processors each runs a main process of an application program in a dual mode, wherein the first thread is spawned to run on a third one of the processors by the application program'"'"'s main process running on the first processor, wherein steps (a)-(f) are performed by the CNK running on the third processor, and wherein a fourth one of the processors runs a second thread spawned by the application program'"'"'s main process running on the second processor.

17. A computer-implemented method for providing improved message handling performance in a distributed computer system utilizing a shared network device, wherein the parallel computer system comprises a control system and a plurality of compute nodes coupled to the control system via a network, each compute node comprises:

a compute logic block having a plurality of processors, wherein one of the processors runs a first thread;

a memory array block shared by the processors;

a network logic block having a network device for facilitating communication of messages between the compute node and the control system, each message comprising a plurality of packets;

wherein when receiving a message at the compute node from the control system, the compute node performs the computer-implemented method comprising the steps of;

(a) obtaining a lock on the network device;

(b) checking a shared storage location of the memory array block to see if a message is pending for the first thread;

(c) if a message is pending for the first thread based on the checking step (b), receiving the remaining packets in the message directly to a user'"'"'s buffer, unlocking the network device, and returning;

(d) if no message is pending for the first thread based on the checking step (b), receiving at least one packet of a message from the network device;

(e) if the at least one packet received in step (d) indicates that the message is for the first thread, receiving the remaining packets in the message directly to the user'"'"'s buffer, unlocking the network device, and returning;

(f) if the at least one packet received in step (d) indicates that the message is for a thread other than the first thread, updating the shared storage location of the memory array block with a thread id of the other thread, unlocking the network device, waiting for a time out to expire, obtaining a lock on the network device, and repeating from the checking step (b).

18. The computer-implemented method as recited in claim 17, wherein the network has a point-to-point mode that allows messages to be sent to a specific node in the network, and wherein when sending a message to one of the compute nodes from the control system, all of the packets in the message are sent together so a complete message with the packets in order is delivered to the compute node.

19. The computer-implemented method as recited in claim 18, wherein each of the messages has a one packet header that includes a thread id identifying a thread to which the message is to be delivered, wherein step (b) comprises checking the shared storage location of the memory array block for a one packet header containing a thread id identifying the first thread, and wherein step (d) comprises receiving a one packet header of the message from the network device.

20. The computer-implemented method as recited in claim 17, wherein a compute node kernel (CNK) runs on each of the processors, wherein a first one of the processors runs a main process of an application program in a symmetric multi-processing (SMP) mode, wherein the first thread is spawned to run on a second one of the processors by the application program'"'"'s main process running on the first processor, wherein steps (a)-(f) are performed by the CNK running on the second processor, and wherein a third one of the processors runs a second thread spawned by the application program'"'"'s main process running on the first processor.

21. The computer-implemented method as recited in claim 17, wherein a compute node kernel (CNK) runs on each of the processors, wherein a first one and a second one of the processors each runs a main process of an application program in a dual mode, wherein the first thread is spawned to run on a third one of the processors by the application program'"'"'s main process running on the first processor, wherein steps (a)-(f) are performed by the CNK running on the third processor, and wherein a fourth one of the processors runs a second thread spawned by the application program'"'"'s main process running on the second processor.