Fencing network direct memory access data transfers in a parallel active messaging interface of a parallel computer

• US 9,075,759 B2
• Filed: 11/05/2010
• Issued: 07/07/2015
• Est. Priority Date: 11/05/2010
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Blocksome, Michael A., Mamidala, Amith R.
• Primary Examiner(s): Vu, Viet
• Assistant Examiner(s): BELCHER, HERMAN A
• Application Number:
  US12/940,282
• Publication Number:
  US 20120117138A1
• Time in Patent Office: 1,705 Days
• Field of Search: 709/201, 709/223, 712/225
• US Class Current: 1/1
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues
• CPC Class Codes: H04L 67/133 Protocols for remote procedure calls [RPC]

Abstract

Fencing direct memory access (‘DMA’) data transfers in a parallel active messaging interface (‘PAMI’) of a parallel computer, the PAMI including data communications endpoints, each endpoint including specifications of a client, a context, and a task, the endpoints coupled for data communications through the PAMI and through DMA controllers operatively coupled to a deterministic data communications network through which the DMA controllers deliver data communications deterministically, including initiating execution through the PAMI of an ordered sequence of active DMA instructions for DMA data transfers between two endpoints, effecting deterministic DMA data transfers through a DMA controller and the deterministic data communications network; and executing through the PAMI, with no FENCE accounting for DMA data transfers, an active FENCE instruction, the FENCE instruction completing execution only after completion of all DMA instructions initiated prior to execution of the FENCE instruction for DMA data transfers between the two endpoints.

Claims

1. A parallel computer that fences direct memory access (DMA) data transfers in a parallel active messaging interface (‘

PAMI’

), the parallel computer comprising a plurality of compute nodes that execute a parallel application, the compute node comprising;

computer processors; and

computer memory coupled to the computer processors having disposed within it computer program instructions that, when executed by the computer processors, cause the parallel computer to function by;

receiving, at an origin endpoint in the PAMI, a DMA PUT instruction from an origin application, wherein the DMA PUT instruction specifies a data transfer between the origin endpoint and a target endpoint in the PAMI, wherein the DMA PUT instruction, upon execution, transmits data from the origin endpoint directly into a target random access memory (RAM) for the target endpoint, wherein each endpoint specifies a client, a context, and a task;

receiving, at an origin endpoint, a FENCE instruction corresponding the data transfer between the origin endpoint and the target endpoint, wherein the FENCE instruction advises the application when the transfers between the origin endpoint and the target endpoint are complete, and

wherein the transfers between the origin endpoint and the target endpoint occur across a deterministic network connecting the origin endpoint and the target endpoint;

posting the DMA PUT instructions and the FENCE instruction to a work queue of the context for the origin endpoint, wherein the FENCE instruction is posted at a location in the ordered sequence such that the FENCE instruction is processed after the plurality of DMA PUT instructions is processed; and

executing, through the PAMI, by the origin endpoint, an advance function that executes instructions in the work queue strictly in the ordered sequence, wherein execution of the FENCE instruction indicates to the origin application that the execution of the DMA PUT instruction is complete.

2. The parallel computer of claim 1 wherein:

each client comprises a collection of data communications resources dedicated to the exclusive use of an application-level data processing entity;

each context comprises a subset of the collection of data processing resources of a client, context functions, and a work queue of data transfer instructions to be performed by use of the subset through the context functions operated by an assigned thread of execution; and

each task represents a process of execution of the parallel application.

3. The parallel computer of claim 1 wherein each context carries out, through post and advance functions, data communications for the parallel application on data communications resources in the exclusive possession of that context.

4. The parallel computer of claim 1 wherein each context carries out data communications operations independently and in parallel with other contexts.

5. The parallel computer of claim 1 wherein the DMA instructions comprise DMA PUT instructions, and executing a FENCE instruction further comprises:

receiving in the origin endpoint the FENCE instruction, the FENCE instruction specifying a FENCE done callback function;

transmitting, after completion of all PUT data transfers previously initiated between the two endpoints, the FENCE instruction through the deterministic network from the origin endpoint to the target endpoint;

transmitting from the target endpoint to the origin endpoint a FENCE acknowledgment message; and

executing by the origin endpoint the FENCE done callback function.

6. A computer program product for fencing direct memory access (DMA) data transfers in a parallel active messaging interface (‘

PAMI’

), the computer program product comprising computer code residing on a computer readable storage device that is not a signal, the computer code, when executed by a computer processor, causes the PAMI to function by;

receiving, at an origin endpoint in the PAMI, a DMA PUT instruction from an origin application, wherein the DMA PUT instruction specifies a data transfer between the origin endpoint and a target endpoint in the PAMI, wherein the DMA PUT instruction, upon execution, transmits data from the origin endpoint directly into a target random access memory (RAM) for the target endpoint, wherein each endpoint specifies a client, a context, and a task;

receiving, at an origin endpoint, a FENCE instruction corresponding the data transfer between the origin endpoint and the target endpoint, wherein the FENCE instruction advises the application when the transfers between the origin endpoint and the target endpoint are complete, and

wherein the transfers between the origin endpoint and the target endpoint occur across a deterministic network connecting the origin endpoint and the target endpoint;

posting the DMA PUT instructions and the FENCE instruction to a work queue of the context for the origin endpoint, wherein the FENCE instruction is posted at a location in the ordered sequence such that the FENCE instruction is processed after the plurality of DMA PUT instructions is processed; and

executing, through the PAMI, by the origin endpoint, an advance function that executes instructions in the work queue strictly in the ordered sequence, wherein execution of the FENCE instruction indicates to the origin application that the execution of the DMA PUT instruction is complete.

7. The computer program product of claim 6 wherein:

each client comprises a collection of data communications resources dedicated to the exclusive use of an application-level data processing entity;

each context comprises a subset of the collection of data processing resources of a client, context functions, and a work queue of data transfer instructions to be performed by use of the subset through the context functions operated by an assigned thread of execution; and

each task represents a process of execution of the parallel application.

8. The computer program product of claim 6 wherein each context carries out, through post and advance functions, data communications for the parallel application on data communications resources in the exclusive possession of that context.

9. The computer program product of claim 6 wherein each context carries out data communications operations independently and in parallel with other contexts.

10. The computer program product of claim 6 wherein the DMA instructions comprise DMA PUT instructions, and executing a FENCE instruction further comprises:

receiving in the origin endpoint the FENCE instruction, the FENCE instruction specifying a FENCE done callback function;

transmitting, after completion of all PUT data transfers previously initiated between the two endpoints, the FENCE instruction through the deterministic network from the origin endpoint to the target endpoint;

transmitting from the target endpoint to the origin endpoint a FENCE acknowledgment message; and

executing by the origin endpoint the FENCE done callback function.