Handling cache misses by selectively flushing the pipeline

• US 7,509,484 B1
• Filed: 06/30/2004
• Issued: 03/24/2009
• Est. Priority Date: 06/30/2004
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Golla, Robert T., Luttrell, Mark A.
• Primary Examiner(s): Kindred; Alford W
• Assistant Examiner(s): Moll; Jesse R
• Application Number:
  US10/882,807
• Time in Patent Office: 1,728 Days
• Field of Search: 712/225, 712/229, 712/203
• US Class Current: 712/225
• CPC Class Codes: G06F 9/3802 Instruction prefetching
• CPC Class Codes: G06F 9/3824 Operand accessing
• CPC Class Codes: G06F 9/3826 Bypassing or forwarding of data results, e.g. locally between pipeline stages or within a pipeline stage
• CPC Class Codes: G06F 9/3834 Maintaining memory consistency
• CPC Class Codes: G06F 9/3851 from multiple instruction streams, e.g. multistreaming
• CPC Class Codes: G06F 9/3861 Recovery, e.g. branch miss-prediction, exception handling error detection or correction G06F11/00
• CPC Class Codes: G06F 9/3867 using instruction pipelines

Abstract

An apparatus and method for efficiently managing data cache load misses is described in connection with a multithreaded, pipelined multiprocessor chip. A CMT processor keeps track of load misses for each thread by issuing a load miss signal each time a load instruction to the data cache misses. A detection logic functionality in the IFU responds the load miss signal to determine if a valid instruction from the thread is at the one of the pipeline stages. If no instructions from the thread are detected in the pipeline, then no flush is required and the thread is placed in a wait state until the requested data is returned from higher order memory. If any instruction from the thread is detected in the pipeline, the thread is flushed and the instruction is re-fetched.

Claims

1. A single chip multithreaded processor comprising:

at least one processor core comprising a plurality of resources for forming a pipeline that generates one or more load instructions, said processor core comprising;

an instruction fetch unit for providing instructions to said pipeline, said instruction fetch unit comprising thread detection logic for monitoring the status of each thread in the pipeline; and

a cache unit for servicing load instructions from the pipeline;

wherein there are a plurality of threads in the pipeline simultaneously,

wherein said pipeline is arranged to switch from a first thread of the plurality of threads containing a first load instruction that misses the cache unit to a second thread, and, only when the thread detection logic detects a follow-on instruction in the pipeline from the first thread to the first load instruction, flush the first thread, and

wherein said instruction fetch unit places the first thread in a wait state without flushing the first thread when the first load instruction misses the cache unit and the thread detection logic detects no instructions in the pipeline that are from the first thread and subsequent to the first load instruction.

2. The single chip multithreaded processor of claim 1, where the cache unit issues a load miss signal to the instruction fetch unit that is used to trigger the thread detection logic.

3. The single chip multithreaded processor of claim 1, wherein the instruction fetch unit comprises a pick unit for tracking the progress of thread instructions through the pipeline.

4. The single chip multithreaded processor of claim 1, wherein the pipeline comprises a decode stage, an execute stage, a memory stage and a bypass stage, where the bypass stage determines whether a hit or miss occurred in the cache unit.

5. The single chip multithreaded processor of claim 1, wherein the pipeline comprises a plurality of pipeline stages, and where the thread detection logic keeps track, for each thread, of whether a valid instruction is at any of the plurality of pipeline stages.

6. The single chip multithreaded processor of claim 1, wherein the cache unit comprises a first level data cache.

7. The single chip multithreaded processor of claim 1, wherein said instruction fetch unit places the first thread in a ready state from the wait state once the first load instruction is serviced by higher order memory.

8. The single chip multithreaded processor of claim 1, wherein said pipeline flushes any thread containing a load instruction that misses the cache unit without flushing any other threads from the pipeline.

18. The processor of claim 1, wherein the pipeline is arranged so that if an other load instruction that is in the pipeline and contained by another thread different from the first thread misses the cache unit, the pipeline will switch from the other thread and, only if the thread detection logic detects an instruction that is contained in the other thread and in the pipeline and that is subsequent in the other thread to the other load instruction, flush all instructions of the other thread that are in the pipeline and subsequent to the other load instruction.

19. The processor of claim 1, wherein the pipeline is arranged so that if an other load instruction that is in the pipeline and contained by another thread different from the first thread misses the cache unit, the pipeline will switch from the other thread and, only if the thread detection logic detects an instruction that is contained in the other thread and in the pipeline and that is subsequent to the other load instruction, flush from the pipeline all instructions that are both contained in the other thread and located in the pipeline between the other load instruction and the instruction fetch unit.

9. A processor system comprising at least one pipelined processing element and a cache memory, said pipelined processing element comprising:

a plurality of pipeline stages that generates load instructions to the cache memory for at least a first thread;

a memory controller for generating a cache miss signal when a load instruction in the first thread misses the cache memory; and

means for switching from the first thread to a second thread in response to the cache miss signal and, only if an instruction after the load instruction in the first thread is at any of the plurality of pipeline stages, flushing instructions of the first thread from the plurality of pipeline stages,

wherein there are a plurality of threads in the plurality of pipeline stages simultaneously, and

wherein the means for flushing suppresses the flushing of the first thread from the plurality of stages in response to the cache miss signal if no instruction after the load instruction in the first thread is detected at any of the plurality of pipeline stages.

10. The processor system of claim 9, wherein said processing element is constructed on a single integrated circuit.

11. The processor system of claim 9, wherein the pipelined processing element comprises:

a plurality of pipeline stages that generates load instructions to the cache memory for a plurality of threads;

a memory controller for generating a cache miss signal when a load instruction in a first thread misses the cache memory; and

means for flushing only the first thread from the plurality of pipeline stages in response to the cache miss signal only if an instruction after the load instruction in the first thread is detected at any of the plurality of pipeline stages.

12. The processor system of claim 9, where the cache memory comprises a first level data cache memory.

13. The processor system of claim 9, wherein the memory controller comprises a load store unit and the means for flushing comprises an instruction fetch unit.

14. The processor system of claim 13, wherein the instruction fetch unit stalls instructions from the first thread when a load instruction in the first thread misses the cache memory until load data is returned to the processing element.

15. The processor system of claim 13, wherein the instruction fetch unit comprises:

a fetch unit; and

a pick unit comprising switch logic for detecting if an instruction after the load instruction in the first thread is present at any of the plurality of pipeline stages.

20. The system of claim 9, wherein the flushing instructions of the first thread comprises flushing from the pipeline all instructions of the first thread that are in the pipeline and subsequent to the load instruction.

16. In a multithreaded processor comprising a cache memory and an instruction fetch unit for issuing instructions to a pipeline, a method for handling cache misses, comprising:

processing a plurality of threads with a pipelined processor core comprising a plurality of pipeline stages;

issuing a memory request from a first thread to the cache memory;

issuing a cache miss signal if the memory request from the first thread misses the cache memory;

in response to the cache miss signal, switching to a second thread and, only when one or more of the plurality of pipeline stages contains an instruction from the first thread that is subsequent to the memory request, flushing from the plurality of pipeline stages all instructions in the first thread that are subsequent to the memory request; and

entering the first thread into a wait state and suppressing the flushing of the first thread from the plurality of pipeline stages, when none of the plurality of pipeline stages contains an instruction from the first thread subsequent to the memory request,

wherein there are a plurality of threads in the plurality of pipeline stages simultaneously.

17. The method of claim 16, where the cache memory comprises a first level data cache memory and wherein the cache miss signal indicates that the memory request missed the first level data cache memory.

21. The method of claim 16, comprising:

in response to the cache miss signal and only if one or more of the plurality of pipeline stages contains an instruction from the first thread that is subsequent to the memory request, flushing from the plurality of pipeline stages all instructions in the first thread that are between the instruction fetch unit and the memory request.