Apparatus and Method for Communicating with an I/O Adapter Using Cached Address Translations

• US 20080091855A1
• Filed: 10/17/2006
• Published: 04/17/2008
• Est. Priority Date: 10/17/2006
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Recio, Renato J., Thurber, Steven M., Salzberg, Claudia A., Moertl, Daniel F.
• Granted Patent: US 7,506,084 B2
• Time in Patent Office: Days
• US Class Current: 710/52
• CPC Class Codes: G06F 12/1081 for peripheral access to main memory, e.g. direct memory access [DMA]
• CPC Class Codes: G06F 12/1475 in a virtual system, e.g. with translation means
• CPC Class Codes: G06F 13/122 where hardware performs an I/O function other than control of data transfer
• CPC Class Codes: G06F 2212/683 Invalidation

Abstract

An apparatus and method for communicating with an input/output (I/O) adapter configured to communicate with a locally attached I/O device are provided using cached address translations. With the apparatus and method, in response to receiving a storage transaction request, a queue element is created in a command queue specifying an untranslated buffer address. The queue element may be retrieved by the I/O adapter and a determination may be made as to whether the queue element contains a read operation command. If so, a translation request may be sent from the I/O adapter to a root complex at substantially a same time as the read operation command is sent to a locally attached external I/O device. The translated address corresponding to the untranslated address of the queue element may be returned and stored in the I/O adapter prior to receiving the data read from the external I/O device.

Claims

1. A method, in a data processing system, for performing an operation between an system image and a locally attached external storage device, comprising:

receiving a storage transaction request from the system image;

creating a command queue element (CQE) in a command queue data structure, the CQE specifying an untranslated address of a buffer data structure and containing a data transfer operation command;

retrieving, by an input/output (I/O) adapter for communicating with the locally attached external storage device, the CQE from the command queue data structure;

sending the data transfer operation command to the locally attached external storage device coupled to the I/O adapter;

issuing, by the I/O adapter, a request to a root complex of the data processing system for one or more translated addresses, corresponding to the one or more untranslated addresses of the buffer data structure, at substantially a same time as the sending of the data transfer operation command to the locally attached external storage device; and

storing, by the I/O adapter, in a local address translation cache of the I/O adapter, the one or more translated addresses corresponding to the one or more untranslated addresses of the buffer data structure.

2. The method of claim 1, wherein the data transfer operation is one of a write to the locally attached external storage device or a read from the locally attached external storage device.

3. The method of claim 1, further comprising:

utilizing, by the I/O adapter, the one or more translated addresses stored in the local address translation cache to perform a direct memory access (DMA) transaction between the I/O adapter and the buffer data structure to place data read from the locally attached external storage device into the buffer data structure.

4. The method of claim 3, wherein the DMA transaction has a translation bit set indicating that the address used by the DMA transaction is a translated address, and wherein the root complex passes the DMA operation directly to the buffer data structure without performing an address translation operation in response to detecting that the translation bit is set.

5. The method of claim 1, further comprising:

storing the address translations associated with the data transfer operation command in a local copy of the CQE in the I/O adapter.

6. The method of claim 1, further comprising:

writing, in response to creating the CQE, a value to a doorbell storage device associated with the I/O adapter indicating that the CQE is available for processing by the I/O adapter, wherein the I/O adapter retrieves the CQE from the command queue data structure in response to the writing of the value to the doorbell storage device.

7. The method of claim 1, further comprising:

invoking, by a device driver of the data processing system, device driver services for initializing one or more address translation entries in an address translation data structure associated with the root complex for the command queue data structure; and

providing the one or more untranslated addresses associated with the command queue data structure to the I/O adapter, wherein the one or more address translation entries specify a translation of one or more untranslated addresses associated with the command queue data structure to one or more translated addresses.

8. The method of claim 1, wherein the I/O adapter is one of a small component system interface (SCSI) adapter, serially attached SCSI (SAS) adapter, Serial Advanced Technology Attachment (SATA), or Advanced Technology Attachment (ATA) adapter.

9. The method of claim 1, further comprising:

reading a value provided in a header of the CQE; and

determining if the CQE contains a read data transfer operation command based on the value provided in the header of the CQE, wherein issuing the request to the root complex for one or more translated addresses and storing the one or more translated addresses is performed in response to a determination that the CQE contains a read data transfer operation command.

10. The method of claim 1, further comprising:

sending a response queue entry (RQE) from the I/O adapter to a response queue data structure associated with the system image in response to completion of the data transfer operation command of the CQE.

11. The method of claim 10, further comprising:

retrieving, by a device driver, the RQE from the response queue data structure; and

invoking, by the device driver, device driver services to unpin host memory associated with the buffer data structure, release the one or more untranslated addresses associated with the buffer data structure, and clear address translation entries in an address translation data structure corresponding to the buffer data structure.

12. A computer program product comprising a computer useable medium having a computer readable program, wherein the computer readable program, when executed on a computing device, causes the computing device to:

receive a storage transaction request from the system image;

create a command queue element (CQE) in a command queue data structure, the CQE specifying an untranslated address of a buffer data structure and containing a data transfer operation command;

retrieve, by an input/output (I/O) adapter for communicating with the locally attached external storage device, the CQE from the command queue data structure;

send the data transfer operation command to the locally attached external storage device coupled to the I/O adapter;

issue, by the I/O adapter, a request to a root complex of the data processing system for one or more translated addresses, corresponding to the one or more untranslated addresses of the buffer data structure, at substantially a same time as the sending of the data transfer operation command to the locally attached external storage device; and

store, by the I/O adapter, in a local address translation cache of the I/O adapter, the one or more translated addresses corresponding to the one or more untranslated addresses of the buffer data structure.

13. The computer program product of claim 12, wherein the data transfer operation is one of a write to the locally attached external storage device or a read from the locally attached external storage device.

14. The computer program product of claim 12, wherein the computer readable program further causes the computing device to:

utilize, by the I/O adapter, the one or more translated addresses stored in the local address translation cache to perform a direct memory access (DMA) transaction between the I/O adapter and the buffer data structure to place data read from the locally attached external storage device into the buffer data structure.

15. The computer program product of claim 14, wherein the DMA transaction has a translation bit set indicating that the address used by the DMA transaction is a translated address, and wherein the root complex passes the DMA operation directly to the buffer data structure without performing an address translation operation in response to detecting that the translation bit is set.

16. The computer program product of claim 12, wherein the computer readable program further causes the computing device to:

store the address translations associated with the data transfer operation command in a local copy of the CQE in the I/O adapter.

17. The computer program product of claim 12, wherein the computer readable program further causes the computing device to:

write, in response to creating the CQE, a value to a doorbell storage device associated with the I/O adapter indicating that the CQE is available for processing by the I/O adapter, wherein the I/O adapter retrieves the CQE from the command queue data structure in response to the writing of the value to the doorbell storage device.

18. The computer program product of claim 12, wherein the computer readable program further causes the computing device to:

invoke, by a device driver of the data processing system, device driver services for initializing one or more address translation entries in an address translation data structure associated with the root complex for the command queue data structure; and

provide the one or more untranslated addresses associated with the command queue data structure to the I/O adapter, wherein the one or more address translation entries specify a translation of one or more untranslated addresses associated with the command queue data structure to one or more translated addresses.

19. The computer program product of claim 12, wherein the computer readable program further causes the computing device to:

read a value provided in a header of the CQE; and

determine if the CQE contains a read data transfer operation command based on the value provided in the header of the CQE, wherein issuing the request to the root complex for one or more translated addresses and storing the one or more translated addresses is performed in response to a determination that the CQE contains a read data transfer operation command.

20. An apparatus, comprising:

a processor; and

an input/output (I/O) adapter coupled to the processor, the I/O adapter being configured for communicating with a locally attached external storage device, wherein the processor;

receives a storage transaction request from the system image; and

creates a command queue element (CQE) in a command queue data structure, the CQE specifying an untranslated address of a buffer data structure and containing a data transfer operation command, and wherein the I/O adapter

retrieves the CQE from the command queue data structure;

sends the data transfer operation command to the locally attached external storage device coupled to the I/O adapter;

issues a request to a root complex of the data processing system for one or more translated addresses, corresponding to the one or more untranslated addresses of the buffer data structure, at substantially a same time as the sending of the data transfer operation command to the locally attached external storage I/O device; and

stores, in a local address translation cache of the I/O adapter, the one or more translated addresses corresponding to the one or more untranslated addresses of the buffer data structure.