Low latency thread context caching

• US 10,127,076 B1
• Filed: 06/06/2016
• Issued: 11/13/2018
• Est. Priority Date: 10/21/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Barroso, Luiz Andre, Laudon, James, Marty, Michael R.
• Primary Examiner(s): Sough, S.
• Assistant Examiner(s): Nguyen, Mongbao
• Application Number:
  US15/174,319
• Time in Patent Office: 890 Days
• CPC Class Codes: G06F 12/0815 Cache consistency protocols
• CPC Class Codes: G06F 12/0875 with dedicated cache, e.g. instruction or stack
• CPC Class Codes: G06F 2212/45 Caching of specific data in cache memory
• CPC Class Codes: G06F 2212/451 Stack data
• CPC Class Codes: G06F 9/3009 Thread control instructions
• CPC Class Codes: G06F 9/30123 according to context, e.g. thread buffers
• CPC Class Codes: G06F 9/3851 from multiple instruction streams, e.g. multistreaming
• CPC Class Codes: G06F 9/461 Saving or restoring of program or task context
• CPC Class Codes: G06F 9/4825 Interrupt from clock, e.g. time of day

Abstract

A method includes performing one or more operations as requested by a thread executing on a processor, the thread having a thread context; receiving a park request from the thread, the park request received following a request from the thread for a low latency resource, wherein the cache response time is less than or equal to a resource response threshold so as to allow the thread context to be stored and retrieved from the cache in less time than the portion of time it takes to complete the request for the low latency resource; storing the thread context in the cache; detecting that the resume condition has occurred; retrieving the thread context from the cache; and resuming execution of the thread.

Claims

1. A computer-implemented method executed by a processor, the method comprising:

performing, by a processor core of the processor, one or more operations as requested by a thread executing on the processor, the thread having a thread context including information used by the thread during execution;

receiving, by the processor core, a park request from the thread indicating that execution of the thread should be suspended and the thread context should be stored in a cache included in the processor core and having a cache response time, the park request including a resume condition indicating when the thread should be resumed, the park request received following a request from the thread for a low latency resource to perform an operation for which the cache response time is less than or equal to a response time threshold so as to allow the thread context to be stored and retrieved from the cache in less time than a time it takes the low latency resource to complete the operation;

storing the thread context in the cache;

detecting that the resume condition has occurred, wherein detecting that the resume condition has occurred includes detecting that a value of a resume register has been updated;

retrieving the thread context from the cache; and

resuming execution of the thread on the processor core using the retrieved thread context.

2. The method of claim 1, wherein the request from the thread for the operation includes at least one of:

a store operation, a load operation, or a mathematical operation.

3. The method of claim 2, wherein the operation is performed using one of:

flash memory, random access memory (RAM), a co-processor, a graphics processing unit (GPU), a network interface controller (NIC), or a local-area network (LAN).

4. The method of claim 1, wherein detecting that the resume condition has occurred includes detecting that a memory location has been updated.

5. The method of claim 1, wherein storing the thread context in the cache includes storing at least one of:

a stack location associated with the thread, a current instruction pointer associated with the thread, an execution register state associated with the thread, a pointer to a set of higher level cache entries associated with the thread, or a pointer to a set of translation lookaside buffer entries associated with the thread.

6. The method of claim 1, wherein the response time for the operation is between 100 nanoseconds and 100 microseconds.

7. A system comprising:

memory for storing data; and

a processor operable to perform operations comprising;

performing, by a processor core of the processor, one or more operations as requested by a thread executing on the processor, the thread having a thread context including information used by the thread during execution;

receiving, by the processor core, a park request from the thread indicating that execution of the thread should be suspended and the thread context should be stored in a cache included in the processor core and having a cache response time, the park request including a resume condition indicating when the thread should be resumed, the park request received following a request from the thread for a low latency resource to perform an operation for which the cache response time is less than or equal to a response time threshold so as to allow the thread context to be stored and retrieved from the cache in less time than a time it takes the low latency resource to complete the operation;

storing the thread context in the cache;

detecting that the resume condition has occurred, wherein detecting that the resume condition has occurred includes detecting that a value of a resume register has been updated;

retrieving the thread context from the cache; and

resuming execution of the thread on the processor core using the retrieved thread context.

8. The system of claim 7, wherein the request from the thread for the operation includes at least one of:

a store operation, a load operation, or a mathematical operation.

9. The system of claim 8, wherein the operation is performed using one of:

flash memory, random access memory (RAM), a co-processor, a graphics processing unit (GPU), a network interface controller (NIC), or a local-area network (LAN).

10. The system of claim 7, wherein detecting that the resume condition has occurred includes detecting that a memory location has been updated.

11. The system of claim 7, wherein storing the thread context in the cache includes storing at least one of:

a stack location associated with the thread, a current instruction pointer associated with the thread, an execution register state associated with the thread, a pointer to a set of higher level cache entries associated with the thread, or a pointer to a set of translation lookaside buffer entries associated with the thread.

12. The system of claim 7, wherein the response time for the operation is between 100 nanoseconds and 100 microseconds.

13. A non-transitory, computer-readable medium storing instructions operable when executed to cause a processor to perform operations comprising:

performing, by a processor core of the processor, one or more operations as requested by a thread executing on the processor, the thread having a thread context including information used by the thread during execution;

receiving, by the processor core, a park request from the thread indicating that execution of the thread should be suspended and the thread context should be stored in a cache included in the processor core and having a cache response time, the park request including a resume condition indicating when the thread should be resumed, the park request received following a request from the thread for a low latency resource to perform an operation for which the cache response time is less than or equal to a response time threshold so as to allow the thread context to be stored and retrieved from the cache in less time than a time it takes the low latency resource to complete the operation;

storing the thread context in the cache;

detecting that the resume condition has occurred, wherein detecting that the resume condition has occurred includes detecting that a value of a resume register has been updated;

retrieving the thread context from the cache; and

resuming execution of the thread on the processor core using the retrieved thread context.

14. The non-transitory, computer-readable medium of claim 13, wherein the request from the thread for the operation includes at least one of:

a store operation, a load operation, or a mathematical operation.

15. The non-transitory, computer-readable medium of claim 14, wherein the operation is performed using one of:

flash memory, random access memory (RAM), a co-processor, a graphics processing unit (GPU), a network interface controller (NIC), or a local-area network (LAN).

16. The non-transitory, computer-readable medium of claim 13, wherein detecting that the resume condition has occurred includes detecting that a memory location has been updated.

17. The non-transitory, computer-readable medium of claim 13, wherein storing the thread context in the cache includes storing at least one of:

a stack location associated with the thread, a current instruction pointer associated with the thread, an execution register state associated with the thread, a pointer to a set of higher level cache entries associated with the thread, or a pointer to a set of translation lookaside buffer entries associated with the thread.