Method and apparatus for employing commit-signals and prefetching to maintain inter-reference ordering in a high-performance I/O processor

• US 6,085,263 A
• Filed: 10/24/1997
• Issued: 07/04/2000
• Est. Priority Date: 10/24/1997
• Status: Expired due to Term

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Compaq Computer Corporation (HP Inc.)
• Inventors: Pawlowski, Chester, Steely, Simon C. Jr., Van Doren, Stephen R., Gharachorloo, Kourosh, Sharma, Madhumitra
• Primary Examiner(s): Lee, Thomas C.
• Assistant Examiner(s): PEYTON, TAMMARA R
• Application Number:
  US08/956,861
• Time in Patent Office: 984 Days
• Field of Search: 711/167, 711/146, 711/147, 711/150, 711/137, 711/203, 711/239, 710/56, 710/59, 710/119, 712/23, 712/218
• US Class Current: 710/56
• CPC Class Codes: G06F 12/082 Associative directories G06F12/0822 takes precedence
• CPC Class Codes: G06F 12/0862 with prefetch
• CPC Class Codes: G06F 13/124 where hardware is a sequential transfer control unit, e.g. microprocessor, peripheral processor or state-machine
• CPC Class Codes: G06F 2212/621 Coherency control relating to peripheral accessing, e.g. from DMA or I/O device

Abstract

An improved I/O processor (IOP) delivers high I/O performance while maintaining inter-reference ordering among memory reference operations issued by an I/O device as specified by a consistency model in a shared memory multiprocessor system. The IOP comprises a retire controller which imposes inter-reference ordering among the operations based on receipt of a commit signal for each operation, wherein the commit signal for a memory reference operation indicates the apparent completion of the operation rather than actual completion of the operation. In addition, the IOP comprises a prefetch controller coupled to an I/O cache for prefetching data into cache without any ordering constraints (or out-of-order). The ordered retirement functions of the IOP are separated from its prefetching operations, which enables the latter operations to be performed in an arbitrary manner so as to improve the overall performance of the system.

Claims

1. An I/O processor (IOP) for delivering I/O performance while maintaining inter-reference ordering among memory reference operations issued by an I/O device to a shared memory multiprocessor system, the IOP comprising:

an I/O cache having a plurality of cache entries for storing data relating to the memory reference operations issued by the I/O device; and

a retire controller coupled to the I/O cache and configured to one of (i) retrieve data from an entry of the cache and provide the retrieved data to the I/O device and (ii) update the entry with data provided by the I/O device, the retire controller imposing inter-reference ordering among the memory reference operations including operations issued to the system for data not present in the cache, the inter-reference ordering based on receipt of a commit signal for each operation issued to the system, the commit signal generated by an ordering point of the system and transmitted to the IOP in response to total ordering of each issued operation at the ordering point, wherein the commit signal for a memory reference operation indicates the apparent completion of the operation rather than actual completion of the operation.

2. The IOP of claim 1 further comprising a retire queue coupled to the retire controller for storing previous and subsequent memory reference operations issued by the I/O device.

3. The IOP of claim 2 further comprising a prefetch controller coupled to the I/O cache for prefetching data into the cache without any ordering constraints.

4. The IOP of claim 3 further comprising an input request queue for receiving the memory reference operations issued by the I/O device and providing the operations to the prefetch controller.

5. The IOP of claim 4 further comprising an output request queue for receiving the memory reference operations provided to the prefetch controller as prefetch operations and providing the prefetch operations to the system.

6. The IOP of claim 5 further comprising an input response queue coupled to the I/O cache for receiving data requested by the prefetch operations and providing the data to appropriate cache entries of the cache.

7. The IOP of claim 6 further comprising, wherein commitment of the previous operation is associated with receipt of a commit signal returned by the system to the IOP, a probe queue coupled to the I/O cache for receiving the commit signal and providing the signal to the appropriate cache entry of the cache.

8. The IOP of claim 7 further comprising a probe and response handler coupled to the I/O cache for accessing the appropriate cache entries to apply the data and commit signal provided by the input response and probe queues.

9. The IOP of claim 8 further comprising an output response queue coupled to the I/O cache for returning data to the I/O device.

10. The IOP of claim 1 wherein a programming interface to the distributed shared memory is sequential consistency.

11. The IOP of claim 8 wherein a programming interface is weak-ordering and wherein inter-reference ordering is imposed by an I/O memory barrier (MB) operation issued by the I/O device between sets of the memory reference operations stored in the retire queue, the sets of memory reference operations issued to the IOP as pre-I/OMB memory reference operations and post-I/OMB memory reference operations.

12. The IOP of claim 11 wherein ordering of each prefetch operation constitutes a commit-event for the operation and wherein the commit signal is transmitted to the IOP upon the occurrence of, or after, the commit-event.

13. The IOP of claim 12 wherein receipt of the transmitted commit signal by the IOP constitutes commitment of the prefetch operation and wherein the retire controller may pass the I/OMB operation in the retire queue once commitment of all pre-I/OMB memory reference operations have been received by the IOP.

14. The IOP of claim 13 further comprising a counter of the IOP coupled to the prefetch controller, the retire controller and the probe and response handler, the prefetch controller incrementing the counter upon loading the pre-I/OMB instruction into the retire queue, the probe and response handler decrementing the counter upon receipt of each commit signal responsive to the pre-I/OMB operations, and the retire controller monitoring the counter for value zero in order to proceed past the I/OMB operation.

15. A method for delivering input/output (I/O) performance while maintaining inter-reference ordering among memory reference operations issued by an I/O device to an I/O processor (IOP) for data in a shared memory multiprocessor system, the method comprising the steps of:

issuing a first memory reference operation from the I/O device to a prefetch controller of the IOP;

transferring the first memory reference operation to the system as a prefetch operation for data requested by the first memory reference operation in response to the requested data not being present in a cache of the IOP;

loading the first memory reference operation into a retire queue of the IOP;

issuing a second memory reference operation from the I/O device to the prefetch controller of the IOP and loading the second memory reference operation into the retire queue behind the first memory reference operation;

in response to the first memory reference operation propagating to a head of the retire queue, determining whether the requested data is present in the cache;

if the requested data is not present in the cache, keeping the first memory reference operation at the head of the queue until the requested data is provided to the cache;

if the requested data is present in the cache, delaying retirement of the second memory reference operation until a commit signal corresponding to the prefetch operation has been returned to the cache, the commit signal generated by an ordering point of the system and transmitted to the cache in response to total ordering of the prefetch operation at the ordering point, thereby maintaining inter-reference ordering among the operations issued by the I/O device.

16. The method of claim 15 wherein the step of transferring comprises the steps of:

forwarding the ordered prefetch operation to an owner of the requested data; and

providing the requested data from the owner to the cache of the IOP.

17. The method of claim 16 wherein the step of transferring further comprises the steps of:

returning the corresponding commit signal to the IOP; and

updating the cache to reflect return of the corresponding commit signal.

18. The method of claim 17 further comprising the step of retiring the first memory reference operation at a retire controller of the IOP when the requested data is present in the cache and the cache is updated to reflect return of the corresponding commit signal, wherein the step of retiring of the memory reference operation comprises the step of delivering the requested data to the I/O device.