Fast path memory read request processing in a multi-level memory architecture

• US 20070113019A1
• Filed: 11/17/2005
• Published: 05/17/2007
• Est. Priority Date: 11/17/2005
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Yifrach, Shaul, Bar-Joshua, Michael, Beukema, Bruce, Mesh, Alexander
• Granted Patent: US 7,500,062 B2
• Time in Patent Office: Days
• US Class Current: 711/137
• CPC Class Codes: G06F 12/0897 with two or more cache hierarchy levels with multilevel cache hierarchies G06F12/0811
• CPC Class Codes: G06F 12/126 with special data handling, e.g. priority of data or instructions, handling errors or pinning
• CPC Class Codes: G06F 9/3834 Maintaining memory consistency

Abstract

A circuit arrangement and method selectively reorder speculatively issued memory read requests being communicated to a lower memory level in a multi-level memory architecture. In particular, a memory read request that has been speculatively issued to a lower memory level prior to completion of a cache lookup operation initiated in a cache memory in a higher memory level may be reordered ahead of at least one previously received and pending request awaiting communication to the lower memory level. By doing so, the latency associated with the memory read request is reduced when the request results in a cache miss in the higher level memory, and as a result, system performance is improved.

Claims

1. A method of processing a memory read request in a multi-level memory architecture of the type including a cache memory and a main memory, the method comprising, in response to receiving the memory read request:

initiating a cache lookup operation to determine if the memory read request is directed to a cache line currently cached in the cache memory;

speculatively issuing the memory read request to the main memory prior to completion of the cache lookup operation by;

receiving the memory read request;

determining whether the memory read request is directed to the same cache line as any memory write request stored in a first queue that stores requests awaiting a cache lookup response;

bypassing the first queue in response to determining that the memory read request is not directed to the same cache line as any memory write request stored in the first queue;

determining whether the memory read request is directed to the same cache line as any memory write request stored in a second queue that stores requests for which cache lookup responses have been received and that are awaiting communication to the main memory; and

bypassing the second queue in response to determining that the memory read request is not directed to the same cache line as any memory write request stored in the second queue.

2. A method of processing a memory read request in a multi-level memory architecture of the type including first and second memory levels, wherein the first memory level comprises a cache memory, the method comprising, in response to receiving the memory read request:

initiating a cache lookup operation to determine if the memory read request is directed to a cache line currently cached in the cache memory in the first memory level; and

speculatively issuing the memory read request to the second memory level prior to completion of the cache lookup operation, including reordering the memory read request ahead of at least one previously received and pending request awaiting communication to the second memory level.

3. The method of claim 2, wherein reordering the memory read request comprises:

determining if another request directed to the same cache line as that to which the memory read request is directed is currently pending in a request queue; and

placing the memory read request at a top of the request queue if no other request directed to the same cache line as that to which the memory read request is directed is currently pending in the request queue.

4. The method of claim 3, wherein determining if another request directed to the same cache line as that to which the memory read request is directed is currently pending in the request queue includes determining if a memory write request directed to the same cache line as that to which the memory read request is directed is currently pending in the request queue.

5. The method of claim 3, wherein placing the memory read request at the top of the request queue comprises storing the memory read request in a bypass buffer.

6. The method of claim 3, wherein the request queue comprises a lookup pending queue that stores requests awaiting a cache lookup response from the first memory level.

7. The method of claim 6, wherein reordering the memory read request further comprises:

determining if another request directed to the same cache line as that to which the memory read request is directed is currently pending in a lookup complete queue; and

placing the memory read request at a top of the lookup complete queue if no other request directed to the same cache line as that to which the memory read request is directed is currently pending in the lookup complete queue.

8. The method of claim 3, wherein the request queue comprises a lookup complete queue that stores requests for which cache lookup responses have been received and that are awaiting transmission to the second memory level.

9. The method of claim 2, wherein speculatively issuing the memory read request to the second memory level prior to completion of the cache lookup operation comprises issuing the memory read request to the second memory level concurrently with initiating the cache lookup operation.

10. The method of claim 2, wherein the second memory level comprises a second cache memory.

11. The method of claim 2, wherein the second memory level comprises a main memory.

12. A circuit arrangement, comprising:

first control logic configured to process a received memory read request by initiating a cache lookup operation of a cache memory disposed in a first memory level among first and second memory levels in a multi-level memory architecture to determine if the memory read request is directed to a cache line currently cached in the cache memory in the first memory level, and speculatively the memory read request to the second memory level prior to completion of the cache lookup operation; and

second control logic coupled to the first control logic and configured to selectively reorder the memory read request ahead of at least one previously received and pending request awaiting communication to the second memory level in connection with speculatively issuing the memory read request.

13. The circuit arrangement of claim 12, further comprising a request queue, wherein the second control logic is configured to reorder the memory read request by determining if another request directed to the same cache line as that to which the memory read request is directed is currently pending in the request queue, and placing the memory read request at a top of the request queue if no other request directed to the same cache line as that to which the memory read request is directed is currently pending in the request queue.

14. The circuit arrangement of claim 13, wherein the second control logic is configured to determine if a memory write request directed to the same cache line as that to which the memory read request is directed is currently pending in the request queue.

15. The circuit arrangement of claim 13, further comprising a bypass buffer and address collision logic configured to store the memory read request in the bypass buffer in response to an address associated with the memory read request colliding with an address associated with another request stored in the request queue.

16. The circuit arrangement of claim 15, wherein the request queue comprises a lookup pending queue, wherein the circuit arrangement further comprises a lookup complete queue, a second bypass buffer and second address collision logic configured to store the memory read request in the second bypass buffer in response to an address associated with the memory read request colliding with an address associated with another request stored in the lookup complete queue.

17. The circuit arrangement of claim 13, wherein the request queue comprises a lookup pending queue that stores requests awaiting a cache lookup response from the first memory level.

18. The circuit arrangement of claim 17, further comprising a lookup complete queue that stores requests for which cache lookup responses have been received and that are awaiting transmission to the second memory level, and wherein the second control logic is further configured to reorder the memory read request by determining if another request directed to the same cache line as that to which the memory read request is directed is currently pending in the lookup complete queue, and placing the memory read request at a top of the lookup complete queue if no other request directed to the same cache line as that to which the memory read request is directed is currently pending in the lookup complete queue.

19. The circuit arrangement of claim 12, wherein the first control logic is configured to speculatively issue the memory read request to the second memory level prior to completion of the cache lookup operation by issuing the memory read request to the second memory level concurrently with initiating the cache lookup operation.

20. The circuit arrangement of claim 12, wherein the second control logic is configured to reorder the memory read request ahead of at least one previously received and pending request after receiving the memory read request over a first bus and prior to communicating the memory read request to the second memory level over a second bus.

21. The circuit arrangement of claim 12, wherein the second memory level comprises a second cache memory.

22. The circuit arrangement of claim 12, wherein the second memory level comprises a main memory.

23. An apparatus comprising the circuit arrangement of claim 12.

24. An integrated circuit device comprising the circuit arrangement of claim 12.

25. A program product, comprising a hardware definition program that defines the circuit arrangement of claim 12;

and a computer readable medium bearing the hardware definition program.