Direct Memory Access In A Hybrid Computing Environment

• US 20100274868A1
• Filed: 04/23/2009
• Published: 10/28/2010
• Est. Priority Date: 04/23/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Schimke, Timothy J., Arroyo, Jesse P.
• Granted Patent: US 8,037,217 B2
• Time in Patent Office: Days
• US Class Current: 709/212
• CPC Class Codes: G06F 9/52 Program synchronisation; Mutual exclusion, e.g. by means of semaphores

Abstract

DMA in a computing environment that includes several computers and DMA engines, the computers adapted to one another for data communications by an data communications fabric, each computer executing an application, where DMA includes pinning, by a first application, a memory region, including providing, to all applications, information describing the memory region; effecting, by a second application in dependence upon the information describing the memory region, DMA transfers related to the memory region, including issuing DMA requests to a particular DMA engine for processing; and unpinning, by the first application, the memory region, including insuring, prior to unpinning, that no additional DMA requests related to the memory region are issued, that all outstanding DMA requests related to the memory region are provided to a DMA engine, and that processing of all outstanding DMA requests related to the memory region and provided to a DMA engine has been completed.

Claims

1. A method of direct memory access (‘

DMA’

) in a computing environment, the computing environment comprising a plurality of computers and one or more DMA engines, the computers adapted to one another for data communications by an data communications fabric, each computer executing an application capable of issuing DMA requests to a DMA engine, the method comprising;

pinning, by a first application executing on a first computer, a memory region for DMA transfer of data, including providing, to all applications executing in the computing environment, information describing the memory region;

effecting, by a second application executing on a second computer in dependence upon the information describing the memory region, DMA transfers related to the memory region, including issuing DMA requests to a particular DMA engine for processing; and

unpinning, by the first application, the memory region, including insuring, prior to unpinning, that no additional DMA requests related to the memory region are issued, that all outstanding DMA requests related to the memory region are provided to a DMA engine, and that processing of all outstanding DMA requests related to the memory region and provided to a DMA engine has been completed.

2. The method of claim 1 wherein pinning a memory region for DMA transfer of data further comprises:

translating virtual memory addresses of the memory region into DMA addresses; and

inserting, into a memory region control block for the memory region, as the information describing the memory region, a valid flag indicating whether the memory region is available for DMA transfers, the virtual memory addresses, and the DMA addresses of the memory region.

3. The method of claim 1 wherein effecting DMA transfers related to the memory region further comprises:

incrementing an entrance count in a DMA injection control block associated with the second application;

determining, in dependence upon a valid flag in a memory region control block associated with the memory region, whether memory region is available for DMA transfers; and

if the memory region is available for DMA transfers;

incrementing an issued count in a DMA execution control block associated with the particular DMA engine;

inserting in a DMA request a command to write, as a value of a completed sequence count in the DMA execution control block, the value of the incremented issue count upon completion by the particular DMA engine of the DMA request;

providing the DMA request to the particular DMA engine to process; and

incrementing an exit count in the DMA injection control block associated with the second application.

4. The method of claim 3 further comprising processing, by the particular DMA engine, the DMA request including writing, as the value of the completed sequence count in the DMA execution control block, the value of the incremented issue count upon completion of the DMA request.

5. The method of claim 1 wherein unpinning the memory region further comprises:

setting a value of a valid flag in a memory region control block to indicate that the memory region is not available for DMA transfers; and

unpinning only if;

an entrance count in a DMA injection control block associated with each application executing in the computing environment is greater than or equal to an exit count in the DMA injection control block; and

a completed sequence count in a DMA execution control block associated with each DMA engine in the computing environment is greater than or equal to a corresponding issued count in the DMA execution control block.

6. The method of claim 1 wherein:

effecting DMA transfers related to the memory region further comprises;

incrementing an entrance count in a DMA injection control block associated with the second application;

determining, in dependence upon a valid flag in a memory region control block associated with the memory region, whether the memory region is available for DMA transfers; and

if the memory region is available for DMA transfers;

incrementing an issued count in a DMA execution control block associated with the particular DMA engine;

inserting in a DMA request a command to write, as a value of a completed sequence count for the particular DMA engine in a memory region control block associated with the memory region, the value of the incremented issue count upon completion by the particular DMA engine of the DMA request;

providing the DMA request to the particular DMA engine to process;

incrementing an exit count in the DMA injection control block associated with the second application; and

the method further comprises processing the DMA request by the particular DMA engine including writing, as the value of the completed sequence count for the particular DMA engine in the memory region control block associated with the memory region, the value of the incremented issue count upon completion of the DMA request.

7. The method of claim 1 wherein unpinning the memory region further comprises:

setting a value of a valid flag in a memory region control block associated with the memory region to indicate that the memory region is not available for DMA transfers; and

unpinning only if;

an entrance count in a DMA injection control block associated with each application executing in the computing environment is greater than or equal to an exit count in the DMA injection control block; and

a completed sequence count, for each DMA engine in the computing environment, in the memory region control block associated with the memory region is greater than or equal to a corresponding issued count in a DMA execution control block associated with the DMA engine.

8. The method of claim 1 wherein the computing environment comprises a hybrid computing environment, and the computers of the hybrid computing environment comprise host computers having a host computer architecture and accelerators having an accelerator architecture, the accelerator architecture optimized, with respect to the host computer architecture, for speed of execution of a particular class of computing functions, the host computers adapted to the accelerators for data communications by a system level message passing module.

9. An apparatus for direct memory access (‘

DMA’

) in a computing environment, the computing environment comprising a plurality of computers and one or more DMA engines, the computers adapted to one another for data communications by an data communications fabric, each computer executing an application capable of issuing DMA requests to a DMA engine, the apparatus comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions capable of;

pinning, by a first application executing on a first computer, a memory region for DMA transfer of data, including providing, to all applications executing in the computing environment, information describing the memory region;

effecting, by a second application executing on a second computer in dependence upon the information describing the memory region, DMA transfers related to the memory region, including issuing DMA requests to a particular DMA engine for processing; and

unpinning, by the first application, the memory region, including insuring, prior to unpinning, that no additional DMA requests related to the memory region are issued, that all outstanding DMA requests related to the memory region are provided to a DMA engine, and that processing of all outstanding DMA requests related to the memory region and provided to a DMA engine has been completed.

10. The apparatus of claim 9 wherein pinning a memory region for DMA transfer of data further comprises:

translating virtual memory addresses of the memory region into DMA addresses; and

inserting, into a memory region control block for the memory region, as the information describing the memory region, a valid flag indicating whether the memory region is available for DMA transfers, the virtual memory addresses, and the DMA addresses of the memory region.

11. The apparatus of claim 9 wherein effecting DMA transfers related to the memory region further comprises:

incrementing an entrance count in a DMA injection control block associated with the second application;

determining, in dependence upon a valid flag in a memory region control block associated with the memory region, whether memory region is available for DMA transfers; and

if the memory region is available for DMA transfers;

incrementing an issued count in a DMA execution control block associated with the particular DMA engine;

inserting in a DMA request a command to write, as a value of a completed sequence count in the DMA execution control block, the value of the incremented issue count upon completion by the particular DMA engine of the DMA request;

providing the DMA request to the particular DMA engine to process; and

incrementing an exit count in the DMA injection control block associated with the second application.

12. The apparatus of claim 11 further comprising computer program instructions capable of processing, by the particular DMA engine, the DMA request including writing, as the value of the completed sequence count in the DMA execution control block, the value of the incremented issue count upon completion of the DMA request.

13. The apparatus of claim 9 wherein unpinning the memory region further comprises:

setting a value of a valid flag in a memory region control block to indicate that the memory region is not available for DMA transfers; and

unpinning only if;

an entrance count in a DMA injection control block associated with each application executing in the computing environment is greater than or equal to an exit count in the DMA injection control block; and

a completed sequence count in a DMA execution control block associated with each DMA engine in the computing environment is greater than or equal to a corresponding issued count in the DMA execution control block.

14. The apparatus of claim 9 wherein:

effecting DMA transfers related to the memory region further comprises;

incrementing an entrance count in a DMA injection control block associated with the second application;

determining, in dependence upon a valid flag in a memory region control block associated with the memory region, whether the memory region is available for DMA transfers; and

if the memory region is available for DMA transfers;

incrementing an issued count in a DMA execution control block associated with the particular DMA engine;

inserting in a DMA request a command to write, as a value of a completed sequence count for the particular DMA engine in a memory region control block associated with the memory region, the value of the incremented issue count upon completion by the particular DMA engine of the DMA request;

providing the DMA request to the particular DMA engine to process;

incrementing an exit count in the DMA injection control block associated with the second application; and

the apparatus further comprises computer program instructions capable of processing the DMA request by the particular DMA engine including writing, as the value of the completed sequence count for the particular DMA engine in the memory region control block associated with the memory region, the value of the incremented issue count upon completion of the DMA request.

15. The apparatus of claim 9 wherein unpinning the memory region further comprises:

setting a value of a valid flag in a memory region control block associated with the memory region to indicate that the memory region is not available for DMA transfers; and

unpinning only if;

an entrance count in a DMA injection control block associated with each application executing in the computing environment is greater than or equal to an exit count in the DMA injection control block; and

a completed sequence count, for each DMA engine in the computing environment, in the memory region control block associated with the memory region is greater than or equal to a corresponding issued count in a DMA execution control block associated with the DMA engine.

16. The apparatus of claim 9 wherein the computing environment comprises a hybrid computing environment, and the computers of the hybrid computing environment comprise host computers having a host computer architecture and accelerators having an accelerator architecture, the accelerator architecture optimized, with respect to the host computer architecture, for speed of execution of a particular class of computing functions, the host computers adapted to the accelerators for data communications by a system level message passing module.

17. A computer program product for direct memory access (‘

DMA’

) in a computing environment, the computing environment comprising a plurality of computers and one or more DMA engines, the computers adapted to one another for data communications by an data communications fabric, each computer executing an application capable of issuing DMA requests to a DMA engine, the computer program product disposed in a computer readable recording medium, the computer program product comprising computer program instructions capable of;

pinning, by a first application executing on a first computer, a memory region for DMA transfer of data, including providing, to all applications executing in the computing environment, information describing the memory region;

effecting, by a second application executing on a second computer in dependence upon the information describing the memory region, DMA transfers related to the memory region, including issuing DMA requests to a particular DMA engine for processing; and

unpinning, by the first application, the memory region, including insuring, prior to unpinning, that no additional DMA requests related to the memory region are issued, that all outstanding DMA requests related to the memory region are provided to a DMA engine, and that processing of all outstanding DMA requests related to the memory region and provided to a DMA engine has been completed.

18. The computer program product of claim 17 wherein pinning a memory region for DMA transfer of data further comprises:

translating virtual memory addresses of the memory region into DMA addresses; and

inserting, into a memory region control block for the memory region, as the information describing the memory region, a valid flag indicating whether the memory region is available for DMA transfers, the virtual memory addresses, and the DMA addresses of the memory region.

19. The computer program product of claim 17 wherein effecting DMA transfers related to the memory region further comprises:

incrementing an entrance count in a DMA injection control block associated with the second application;

determining, in dependence upon a valid flag in a memory region control block associated with the memory region, whether memory region is available for DMA transfers; and

if the memory region is available for DMA transfers;

incrementing an issued count in a DMA execution control block associated with the particular DMA engine;

inserting in a DMA request a command to write, as a value of a completed sequence count in the DMA execution control block, the value of the incremented issue count upon completion by the particular DMA engine of the DMA request;

providing the DMA request to the particular DMA engine to process; and

incrementing an exit count in the DMA injection control block associated with the second application.

20. The computer program product of claim 17 wherein unpinning the memory region further comprises:

setting a value of a valid flag in a memory region control block to indicate that the memory region is not available for DMA transfers; and

unpinning only if;

an entrance count in a DMA injection control block associated with each application executing in the computing environment is greater than or equal to an exit count in the DMA injection control block; and

a completed sequence count in a DMA execution control block associated with each DMA engine in the computing environment is greater than or equal to a corresponding issued count in the DMA execution control block.