Delaying acknowledgment of an operation until operation completion confirmed by local adapter read operation

• US 8,589,603 B2
• Filed: 08/30/2010
• Issued: 11/19/2013
• Est. Priority Date: 08/30/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Craddock, David, Gregg, Thomas A.
• Primary Examiner(s): Alrobaye, Idriss N
• Assistant Examiner(s): Yimer, Getente A
• Application Number:
  US12/871,532
• Publication Number:
  US 20120054381A1
• Time in Patent Office: 1,177 Days
• Field of Search: 710/28, 710/1, 710/22, 710/100, 710/105, 710/106
• US Class Current: 710/28
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence

Abstract

A request to perform an operation, such as a remote direct memory access (RDMA) write operation or a send operation that writes to memory, is sent from a sending input/output (I/O) adapter (e.g., an RDMA-capable adapter) to a receiving I/O adapter. The receiving I/O adapter receives the request and initiates performance of the operation, but delays sending an acknowledgment for the operation. The acknowledgment is delayed until the operation is complete (i.e., until the memory is updated and the data is visible to the remote processor), as determined by a read operation initiated and performed by the receiving I/O adapter transparent to the sending I/O adapter.

Claims

1. A computer program product for facilitating processing in a computing environment, the computer program product comprising a non-transitory computer readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method, the method comprising:

receiving at a receiving adapter of the computing environment from a sending adapter of the computing environment a write operation to be performed by the receiving adapter;

performing the write operation by the receiving adapter to write data to memory of the computing environment;

determining by the receiving adapter whether the write operation has completed, wherein the determining comprises performing by the receiving adapter a read operation to confirm the write operation has completed in that the data has been written to the memory, the read operation initiated locally by the receiving adapter and reading only a portion of the data that has been written by the write operation to determine the write operation has completed, and wherein the read operation is absent a comparing of the written data and the read data; and

based on determining via the read operation that the write operation has completed, sending an acknowledgment to the sending adapter.

2. The computer program product of claim 1, wherein acknowledgment of the write operation is delayed until after the read operation confirms completion of the write operation.

3. The computer program product of claim 2, wherein acknowledgment of the write operation prior to the read operation confirming completion of the write operation is avoided.

4. The computer program product of claim 1, wherein the memory is coupled to the receiving adapter via an interconnect that facilitates the write operation, the interconnect having a configuration in which indication of completion of the write operation to the sending adapter is absent.

5. The computer program product of claim 4, wherein the interconnect is a peripheral component interconnect (PCI).

6. The computer program product of claim 1, wherein the read operation reads one or more locations in memory written to by the write operation, and wherein the write operation is performed via an interconnect coupling the receiving adapter and the memory, the interconnect having a configuration with ordering rules, the ordering rules indicating that the read will not complete successfully until after the data is written to the memory by the write operation.

7. The computer program product of claim 1, wherein the method further comprises checking by the receiving adapter whether the sending of the acknowledgment is to be delayed, and performing the determining based on the checking indicating that the sending of the acknowledgment is to be delayed.

8. The computer program product of claim 1, wherein the method further comprises generating based on the acknowledgment a completion queue element used to indicate completion of the write operation.

9. The computer program product of claim 1, wherein the read operation is transparent to the sending adapter.

10. A computer system for facilitating processing in a computing environment, the computer system comprising:

a receiving adapter configured to perform a method, said method comprising;

receiving at the receiving adapter from a sending adapter a write operation to be performed by the receiving adapter;

performing the write operation by the receiving adapter to write data to memory of the computing environment;

determining by the receiving adapter whether the write operation has completed, wherein the determining comprises performing by the receiving adapter a read operation to confirm the write operation has completed in that the data has been written to the memory, the read operation initiated locally by the receiving adapter and reading only a portion of the data that has been written by the write operation to determine the write operation has completed, and wherein the read operation is absent a comparing of the written data and the read data; and

based on determining via the read operation that the write operation has completed, sending an acknowledgment to the sending adapter.

11. The system of claim 10, wherein acknowledgment of the write operation is delayed until after the read operation confirms completion of the write operation.

12. The system of claim 11, wherein acknowledgment of the write operation prior to the read operation confirming completion of the write operation is avoided.

13. The system of claim 10, wherein the memory is coupled to the receiving adapter via an interconnect that facilitates the write operation, the interconnect having a configuration in which indication of completion of the write operation to the sending adapter is absent.

14. The system of claim 10, wherein the read operation reads one or more locations in memory written to by the write operation, and wherein the write operation is performed via an interconnect coupling the receiving adapter and the memory, the interconnect having a configuration with ordering rules, the ordering rules indicating that the read will not complete successfully until after the data is written to the memory by the operation.

15. The system of claim 10, wherein the method further comprises checking by the receiving adapter whether the sending of the acknowledgment is to be delayed, and performing the determining based on the checking indicating that the sending of the acknowledgment is to be delayed.

16. A method of facilitating processing in a computing environment, said method comprising:

receiving at a receiving adapter of the computing environment from a sending adapter of the computing environment a write operation to be performed by the receiving adapter;

performing the write operation by the receiving adapter to write data to memory of the computing environment;

determining by the receiving adapter whether the write operation has completed, wherein the determining comprises performing by the receiving adapter a read operation to confirm the write operation has completed in that the data has been written to the memory, the read operation initiated locally by the receiving adapter and reading only a portion of the data that has been written by the write operation to determine the write operation has completed, and wherein the read operation is absent a comparing of the written data and the read data; and

based on determining via the read operation that the write operation has completed, sending an acknowledgment to the sending adapter.

17. The method of claim 16, wherein acknowledgment of the write operation is delayed until after the read operation confirms completion of the write operation, and wherein acknowledgment of the write operation prior to the read operation confirming completion of the write operation is avoided.

18. The method of claim 16, further comprising checking by the receiving adapter whether the sending of the acknowledgment is to be delayed, and performing the determining based on the checking indicating that the sending of the acknowledgment is to be delayed.

19. A computer program product for facilitating processing in a computing environment, the computer program product comprising a non-transitory computer readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method, the method comprising:

receiving at a receiving adapter of the computing environment from a sending adapter of the computing environment a first operation to be performed by the receiving adapter, the first operation being associated with a first acknowledgment type;

performing the first operation by the receiving adapter;

based on the first acknowledgment type indicating non-delayed acknowledgment, sending a receipt acknowledgment to the sending adapter indicating receipt of the first operation, the receipt acknowledgment not indicating completion of the first operation;

receiving at the receiving adapter from the sending adapter a second operation to be performed by the receiving adapter, the second operation being associated with a second acknowledgment type;

performing the second operation by the receiving adapter, the second operation being a write operation to write data to memory of the computing environment, the computing environment having ordering rules that permit a read operation to complete only based on pending write operations to memory having completed;

based on the second acknowledgment type indicating delayed acknowledgment, determining by the receiving adapter whether the second operation has completed in that the data is written to memory, wherein the determining comprises performing by the receiving adapter a read operation to confirm the second operation has completed, the read operation initiated locally by the receiving adapter and completing, based on the ordering rules, only if the write operation has completed; and

based on determining via completion of the read operation that the second operation has completed, sending a completion acknowledgment to the sending adapter.