Method and system for efficient memory region deallocation

• US 9,244,829 B2
• Filed: 12/20/2012
• Issued: 01/26/2016
• Est. Priority Date: 12/20/2012
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Manula, Brian Edward, Bugge, Haakon Ording, Wittosch, Robert W.
• Primary Examiner(s): Li, Aimee
• Assistant Examiner(s): TA, TRANG KHANH
• Application Number:
  US13/721,875
• Publication Number:
  US 20140181454A1
• Time in Patent Office: 1,132 Days
• Field of Search: 711/206, 711/207
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0223 User address space allocation, e.g. contiguous or non contiguous base addressing
• CPC Class Codes: G06F 12/1027 using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB]
• CPC Class Codes: G06F 12/1081 for peripheral access to main memory, e.g. direct memory access [DMA]
• CPC Class Codes: G06F 12/1475 in a virtual system, e.g. with translation means
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 2212/683 Invalidation

Abstract

A method for deallocation of a memory region involving transmitting, by a host channel adapter (HCA), a first invalidation command for invalidating at least one key associated with the memory region, transmitting, by the HCA, a second invalidation command for invalidating a translation lookaside buffer (TLB) entry for the memory region, invalidate the at least one key associated with the memory region, determining whether all memory access requests to the memory region have been processed by the HCA, stalling processing of the second invalidation command when outstanding memory access requests to the memory region are present, and processing the outstanding memory access requests for the memory region by the HCA before executing the second invalidation command invalidating the TLB entry for the memory region.

Claims

1. A method for deallocation of a memory region, comprising:

transmitting, by a host channel adapter (HCA), a first invalidation command for invalidating at least one key associated with the memory region, wherein the HCA is an Infiniband HCA that communicates via the Infiniband network;

transmitting, by the HCA, a second invalidation command for invalidating a translation lookaside buffer (TLB) entry for the memory region, wherein the first and second invalidation commands are transmitted via a send queue of a queue pair (QP) associated with the Infiniband HCA;

invalidating the at least one key associated with the memory region by removing a corresponding DMA validation entry for the memory region from a key validation table stored in the DMA module of the Infiniband HCA, wherein the at least one key is an index into the key validation table;

determining whether all memory access requests to the memory region have been processed by the Infiniband HCA;

stalling processing of the second invalidation command when outstanding memory access requests to the memory region are present; and

processing the outstanding memory access requests for the memory region by the Infiniband HCA before executing the second invalidation command invalidating the TLB entry for the memory region.

2. The method of claim 1, further comprising:

receiving, by the Infiniband HCA, a request to access the memory region, wherein the request to access the memory region comprises a key for authenticating the request to access the memory region;

incrementing a memory access counter when the request to access the memory region is processed by the Infiniband HCA; and

attempting to validate the key by the Infiniband HCA,

wherein when the key is validated by the Infiniband HCA, ensuring that a corresponding address translation for the memory region is completed before deallocating the memory region.

3. The method of claim 2, wherein ensuring that a corresponding address translation for the memory region is completed before deallocating the memory region comprises stalling processing of the second invalidation command until the request to access the memory region completes address translation using the TLB entry for the memory region.

4. The method of claim 2, wherein when the validation of the key is unsuccessful, blocking access to the memory region.

5. The method of claim 2, further comprising:

granting access to the memory region, when the key is validated by the Infiniband HCA.

6. The method of claim 2, wherein determining whether all memory access requests to the memory region have been processed comprises:

inspecting the memory access counter to determine whether the memory access counter has a value greater than zero.

7. The method of claim 6, wherein when the memory access counter has a value of zero, completing processing of the second invalidation command, resulting in deallocation of the memory region.

8. The method of claim 1, wherein the first and second invalidation commands are issued as a single work request by software executing on a host system operatively connected to the Infiniband HCA.

9. The method of claim 8, wherein stalling processing of the second invalidation command is performed by the software.

10. The method of claim 1, wherein invaliding the at least one key for the memory region comprises invalidating a DMA validation entry in the Infiniband HCA corresponding to the memory region.

11. The method of claim 1, wherein a completion is not written for the execution of the first invalidation command until the second invalidation command is successfully processed by the Infiniband HCA.

12. A system, comprising:

a host in an Infiniband network executing a software application, wherein the software application is configured to issue a first invalidation command and a second invalidation command to deallocate a memory region in the host, wherein the first invalidation command is for invalidating at least one key for access to the memory region, and the second invalidation command is for invalidating a translation lookaside buffer (TLB) entry for the memory region; and

an Infiniband host communication adapter (HCA) operatively connected to the host and configured to communicate via the Infiniband network to;

receive the first and second invalidation commands, wherein the first and second invalidation commands are transmitted and received via a queue pair (QP) associated with the Infiniband HCA;

invalidate the at least one key associated with the memory region by removing a corresponding DMA validation entry for the memory region from a key validation table stored in the DMA module of the Infiniband HCA, wherein the at least one key is an index into the key validation table;

determine whether all memory access requests to the memory region have been processed by the HCA;

stall processing of the second invalidation command, when outstanding memory access requests to the memory region are present; and

process the outstanding memory access requests for the memory region before executing the second invalidation command invalidating the TLB entry for the memory region.

13. The system of claim 12, wherein the Infiniband HCA is further configured to:

receive a request to access the memory region, wherein the request to access the memory region comprises a key for access to the memory region;

increment a memory access counter when the request to access the memory region is processed by the Infiniband HCA; and

attempt to validate the key,

wherein when the key is validated by the Infiniband HCA, ensuring that a corresponding address translation for the memory region is completed before deallocating the memory region.

14. The system of claim 12, wherein ensuring that a corresponding address translation for the memory region is completed before deallocating the memory region comprises stalling processing of the second invalidation command until the request to access the memory region completes address translation using the TLB entry for the memory region.

15. The system of claim 12, wherein determining whether all memory access requests to the memory region have been processed comprises:

inspecting the memory access counter to determine whether the memory access counter has a value greater than zero.

16. The system of claim 12, wherein when the memory access counter has a value of zero, completing processing of the second invalidation command, resulting in deallocation of the memory region.

17. The system of claim 16, wherein a completion is written for successful execution of the first invalidation command only after execution of the second invalidation command takes place.

18. The system of claim 12, wherein the first and second invalidation commands are issued by the software and processed by the Infiniband HCA as two separate work requests.

19. The system of claim 12, wherein invaliding the at least one key for the memory region comprises invalidating a DMA validation entry corresponding to the memory region in the Infiniband HCA.

20. An Infiniband host channel adapter (HCA) configured to communicate via the Infiniband network, comprising:

a direct memory access (DMA) validation module as part of the Infiniband HCA for storing a DMA validation table and a plurality of memory access keys, each of which is a credential to authenticate a memory access request, wherein the DMA validation module is configured to;

authenticate the memory access request using a corresponding key; and

process a command to invalidate a memory access key by invalidating a corresponding DMA validation entry in the DMA validation table wherein the memory access key is an index into the DMA validation table; and

a memory management module (MMU) for storing a translation lookaside buffer (TLB) comprising a plurality of memory address translations for a plurality of memory regions,

wherein the Infiniband HCA is configured to;

receive and process a request to deallocate a memory region of the plurality of memory regions, wherein the request to deallocate the memory region comprises a first invalidation command for invalidating at least one key associated with the memory region and a second invalidation command for invalidating a translation lookaside buffer (TLB) entry for the memory region;

invalidate at least one key associated with the memory region;

determine whether all memory access requests to the memory region have completed address translation in the MMU;

stall processing of the second invalidation command, when outstanding memory access requests to the memory region are present; and

process the outstanding memory access requests for the memory region before executing the second invalidation command.