METHOD AND SYSTEM FOR CONDITIONAL REMOTE DIRECT MEMORY ACCESS WRITE

• US 20130054726A1
• Filed: 08/31/2011
• Published: 02/28/2013
• Est. Priority Date: 08/31/2011
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Bugge, Haakon Ording
• Granted Patent: US 8,832,216 B2
• Time in Patent Office: Days
• US Class Current: 709/212
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
• CPC Class Codes: G06F 9/52 Program synchronisation; Mutual exclusion, e.g. by means of semaphores

Abstract

A method for conditional execution of a remote direct memory access (RDMA) write includes a host channel adapter receiving at least one message that includes an atomic operation and the RDMA write. The host channel adapter obtains a descriptor corresponding to the RDMA write, and determines, from the descriptor, that the RDMA write is a conditional RDMA write conditioned on a successful execution of the atomic operation. Based on determining that the RDMA write is the conditional RDMA write, the conditional RDMA write is queued to be conditionally executed based on a success indicator of the atomic operation. After queuing the RDMA write, the atomic operation is executed successfully. In response to the successful execution, the host channel adapter executes the conditional RDMA write to write to the memory location on the host.

Claims

1. A method for conditional execution of a remote direct memory access (RDMA) write, the method comprising:

receiving, by a host channel adapter, at least one message comprising an atomic operation and the RDMA write from a requester system, wherein the RDMA write is to write to a memory location on a host connected to the host channel adapter;

obtaining, by the host channel adapter, a descriptor corresponding to the RDMA write;

determining, from the descriptor, that the RDMA write is a conditional RDMA write conditioned on a successful execution of the atomic operation;

queuing, based on determining that the RDMA write is the conditional RDMA write, the conditional RDMA write to be conditionally executed based on a success indicator of the atomic operation;

executing, after queuing the conditional RDMA write, the atomic operation, wherein execution of the atomic operation is successful;

maintaining, by the host channel adapter, a state of the atomic operation by setting the success indicator to indicate the successful execution of the atomic operation when execution of the atomic operation is successfully completed; and

in response to setting the success indicator to indicate the successful execution, executing, by the host channel adapter, the conditional RDMA write to write to the memory location on the host.

2. The method of claim 1, wherein the RDMA write is directed to a queue pair, and wherein queuing the conditional RDMA write comprises queuing the RDMA write to execute only after all outstanding atomic operations for the queue pair execute.

3. The method of claim 1, further comprising:

obtaining, by the host channel adapter, the descriptor from the host.

4. The method of claim 1, wherein the atomic operation is a compare-and-swap operation, wherein successful execution is when (1) a compare portion of the compare-and-swap operation determines that a compare value of the compare-and-swap operation is equal to a first version number and (2) a swap portion of the compare-and-swap operation is executed to change the first version number to a new version number specified in the compare-and-swap operation.

5. The method of claim 4, further comprising:

performing a first RDMA read of the memory location to confirm that a consistent copy of the memory location is obtained; and

performing a second RDMA read of the memory location to confirm that the atomic operation resulted in the consistent copy of the memory location,

wherein the atomic operation is executed in response to determining that the consistent copy of the memory location is obtained.

6. The method of claim 5, wherein the consistent copy is confirmed when the first version number preceding the memory location is equal to a second version number following the memory location.

7. The method of claim 1, wherein the atomic operation is included in a first packet and the RDMA write is included in a second packet, and wherein the first packet is queued on a receive queue of a queue pair before the second packet.

8. A responder communication adapter for conditional execution of a remote direct memory access (RDMA) write comprising:

a network port configured to;

receive at least one message comprising a request to perform an atomic operation and the RDMA write from a requester system, wherein the RDMA write is to write to a memory location on a host connected to the responder communication adapter;

a responder processing logic connected to the network port and configured to;

obtain a descriptor corresponding to the RDMA write;

determine, from the descriptor, that the RDMA write is a conditional RDMA write conditioned on a successful execution of the atomic operation;

queue, based on determining that the RDMA write is the conditional RDMA write, the conditional RDMA write to be conditionally executed based on a success indicator of the atomic operation;

execute, after queuing the conditional RDMA write, the atomic operation resulting in the successful execution;

maintain a state of the atomic operation by setting the success indicator to indicate the successful execution of the atomic operation when execution of the atomic operation is successfully completed; and

in response to setting the success indicator to indicate the successful execution, execute the conditional RDMA write to write to the memory location on the host.

9. The responder communication adapter of claim 8, wherein the responder processing logic comprises:

a descriptor fetch module for obtaining the descriptor.

10. The responder communication adapter of claim 8, wherein the responder processing logic comprises:

a DMA validation module configured to;

determine the state of the atomic operation by accessing the success indicator; and

execute the conditional RDMA write only after determining that the state of the atomic operation is successfully completed.

11. The responder communication adapter of claim 8, wherein the responder communication adapter is a host channel adapter.

12. The responder communication adapter of claim 8, wherein the success indicator is a success bit, and wherein the success indicator is set to indicate successful completion when the success bit is set to true.

13. The responder communication adapter of claim 12, wherein the atomic operation is a compare-and-swap operation.

14. The responder communication adapter of claim 13, wherein the success bit is set to true when (1) a compare portion of the compare-and-swap operation determines that a compare value of the compare-and-swap operation is equal to a first version number and (2) a swap portion of the compare-and-swap operation is executed to change the first version number to a new version number specified in the compare-and-swap operation.

15. The responder communication adapter of claim 8, wherein the atomic operation is included in a first packet and the RDMA write is included in a second packet, and wherein the first packet is queued on a receive queue of a queue pair before the second packet.

16. A non-transitory computer readable medium storing instructions for conditional execution of a remote direct memory access (RDMA) write, the instructions comprising functionality for causing a computer system to:

receive at least one message comprising an atomic operation and the RDMA write from a requester system, wherein the RDMA write is to write to a memory location on a host connected to a host channel adapter;

obtain a descriptor corresponding to the RDMA write;

determine, from the descriptor, that the RDMA write is a conditional RDMA write conditioned on a successful execution of the atomic operation;

queue, based on determining that the RDMA write is the conditional RDMA write, the conditional RDMA write to be conditionally executed based on a success indicator of the atomic operation;

execute, after queuing the conditional RDMA write, the atomic operation resulting in the successful execution;

maintain a state of the atomic operation by setting the success indicator to indicate the successful execution of the atomic operation when execution of the atomic operation is successfully completed; and

in response to setting the success indicator to indicate the successful execution, execute the conditional RDMA write to write to the memory location on the host.

17. The non-transitory computer readable medium of claim 16, wherein the instructions further comprise functionality to:

perform a first RDMA read of the memory location to confirm that a consistent copy of the memory location is obtained; and

perform a second RDMA read of the memory location to confirm that the atomic operation resulted in the consistent copy of the memory location,

wherein the atomic operation is executed in response to determining that the consistent copy of the memory location is obtained.

18. The non-transitory computer readable medium of claim 17, wherein the consistent copy is confirmed when a first version number preceding the memory location is equal to a second version number following the memory location.

19. The non-transitory computer readable medium of claim 18, wherein the atomic operation is a compare-and-swap operation, wherein successfully completing execution is when (1) a compare portion of the compare-and-swap operation determines that a compare value of the compare-and-swap operation is equal to the first version number and (2) a swap portion of the compare-and-swap operation is executed to change the first version number to a new version number specified in the compare-and-swap operation.

20. The non-transitory computer readable medium of claim 16, wherein the instructions further comprise functionality to:

obtain the descriptor from the host.