Endpoint-based parallel data processing in a parallel active messaging interface of a parallel computer

• US 8,804,488 B2
• Filed: 11/09/2012
• Issued: 08/12/2014
• Est. Priority Date: 12/09/2010
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Archer, Charles J., Blocksome, Michael A., Ratterman, Joseph D., Smith, Brian E.
• Primary Examiner(s): Mesfin, Yemane
• Assistant Examiner(s): Banthrongsack, Jeff
• Application Number:
  US13/673,188
• Publication Number:
  US 20130067206A1
• Time in Patent Office: 641 Days
• Field of Search: 370/217
• US Class Current: 370/217
• CPC Class Codes: G06F 9/541 via adapters, e.g. between incompatible applications

Abstract

Endpoint-based parallel data processing in a parallel active messaging interface (‘PAMI’) of a parallel computer, the PAMI composed of data communications endpoints, each endpoint including a specification of data communications parameters for a thread of execution on a compute node, including specifications of a client, a context, and a task, the compute nodes coupled for data communications through the PAMI, including establishing a data communications geometry, the geometry specifying, for tasks representing processes of execution of the parallel application, a set of endpoints that are used in collective operations of the PAMI including a plurality of endpoints for one of the tasks; receiving in endpoints of the geometry an instruction for a collective operation; and executing the instruction for a collective operation through the endpoints in dependence upon the geometry, including dividing data communications operations among the plurality of endpoints for one of the tasks.

Claims

1. A method of endpoint-based parallel data processing in a parallel active messaging interface ('"'"'PAMI'"'"') of a parallel computer, the parallel computer comprising a plurality of compute nodes that execute a parallel application, the PAMI comprising data communications endpoints, each endpoint comprising a specification of data communications parameters for a thread of execution on a compute node, including specifications of a client, a context, and a task, the endpoints coupled for data communications through the PAMI, the method comprising:

establishing by an application-level entity, for collective operations of the PAMI, a data communications geometry, the geometry specifying, for tasks representing processes of execution of the parallel application, a set of endpoints that are used in collective operations of the PAMI, including a plurality of endpoints for one of the tasks;

receiving in one or more endpoints of the geometry an instruction for a collective operation, the instruction specifying communications of transfer data among the endpoints of the geometry; and

executing the instruction for a collective operation through the endpoints in dependence upon the geometry, including dividing data communications operations among the plurality of endpoints for one of the tasks.

2. The method of claim 1 wherein establishing a data communications geometry further comprises associating with the geometry a list of collective algorithms valid for use with the endpoints of the geometry.

3. The method of claim 1 wherein dividing data communications operations further comprises:

assigning to each of the plurality of endpoints for one of the tasks a separate subset of the endpoints in the geometry; and

transferring, in parallel operations by each of the plurality of endpoints for one of the tasks, the transfer data to each of the endpoints in the subset assigned to that endpoint.

4. The method of claim 1 wherein dividing data communications operations further comprises:

assigning portions of the transfer data to each of the plurality of endpoints for one of the tasks, the number of portions being equal to the number of tasks of endpoints of the geometry, each portion of the transfer data corresponding to a task of the geometry; and

transferring, in parallel operations by the plurality of endpoints for one of the tasks to one endpoint for each task of the geometry, a corresponding portion of the transfer data.

5. The method of claim 1 wherein dividing data communications operations further comprises:

assigning to each of the plurality of endpoints for one of the tasks a separate subset of the tasks in the geometry;

transferring, by one endpoint of each task in the geometry, a different portion of the transfer data to one of the plurality of endpoints for one of the tasks; and

gathering, in parallel operations by the plurality of endpoints for one task, all of the transfer data into a single receive buffer, the receive buffer divided into segments each of which corresponds to a portion of the transfer data transferred from an endpoint of a task in the geometry.

6. The method of claim 1 wherein:

each client comprises a collection of data communications resources dedicated to the exclusive use of an application-level data processing entity;

each context comprises a subset of the collection of data processing resources of a client, context functions, and a work queue of data transfer instructions to be performed by use of the subset through the context functions operated by an assigned thread of execution; and

each task represents a process of execution of the parallel application.

7. The method of claim 1 wherein each context carries out, through post and advance operations, data communications for the application on data communications resources in the exclusive possession of that context.

8. The method of claim 1 wherein each context carries out data communications operations independently and in parallel with other contexts.