System and method for implementing NUMA-aware reader-writer locks

• US 8,966,491 B2
• Filed: 04/27/2012
• Issued: 02/24/2015
• Est. Priority Date: 04/27/2012
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Calciu, Irina, Dice, David, Luchangco, Victor M., Marathe, Virendra J., Shavit, Nir N., Lev, Yosef
• Primary Examiner(s): An, Meng
• Assistant Examiner(s): NAM, DAREAUM A
• Application Number:
  US13/458,868
• Publication Number:
  US 20130290967A1
• Time in Patent Office: 1,033 Days
• Field of Search: None
• US Class Current: 718/102
• CPC Class Codes: G06F 2209/523 Mode
• CPC Class Codes: G06F 9/526 Mutual exclusion algorithms

Abstract

NUMA-aware reader-writer locks may leverage lock cohorting techniques to band together writer requests from a single NUMA node. The locks may relax the order in which the lock schedules the execution of critical sections of code by reader threads and writer threads, allowing lock ownership to remain resident on a single NUMA node for long periods, while also taking advantage of parallelism between reader threads. Threads may contend on node-level structures to get permission to acquire a globally shared reader-writer lock. Writer threads may follow a lock cohorting strategy of passing ownership of the lock in write mode from one thread to a cohort writer thread without releasing the shared lock, while reader threads from multiple NUMA nodes may simultaneously acquire the shared lock in read mode. The reader-writer lock may follow a writer-preference policy, a reader-preference policy or a hybrid policy.

Claims

1. A method, comprising:

performing, by a computer;

beginning execution of a multithreaded application that comprises one or more requests to acquire a shared lock, wherein the shared lock controls write access to a critical section of code or a shared resource by concurrently executing threads of the multithreaded application, wherein only one thread of the multithreaded application can hold the shared lock for writing at a time, wherein each of the concurrently executing threads of the multithreaded application executes on one of a plurality of processor cores in a cluster of processor cores that is one of a plurality of clusters of processor cores on which the concurrently executing threads of the multithreaded application are executing, and wherein the plurality of processor cores in each of the clusters of processor cores share a memory;

a reader thread of the multithreaded application;

determining that no thread holds the shared lock for writing; and

in response to determining that no thread holds the shared lock for writing;

accessing the critical section of code or shared resource in read-only mode;

subsequent to the reader thread of the multithreaded application accessing the critical section of code or shared resource in read-only mode, a writer thread of the multithreaded application acquiring ownership of a cluster specific lock for writing, and in response to the writer thread of the multithreaded application acquiring ownership of the cluster-specific lock for writing, acquiring the shared lock for writing;

in response to acquiring the shared lock for writing, the writer thread of the multithreaded application;

accessing the critical section of code or shared resource for writing; and

subsequent to said accessing the critical section of code or shared resource for writing;

determining whether at least one other thread of the multithreaded application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread of the multithreaded application is executing is waiting to acquire the shared lock for writing; and

in response to determining that at least one other thread of the multithreaded application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread of the multithreaded application is executing is waiting to acquire the shared lock for writing, passing ownership of the cluster-specific lock for writing to one of the at least one other threads of the multithreaded application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread of the multithreaded application is executing that is waiting to acquire the shared lock for writing without releasing the shared lock.

2. The method of claim 1,

wherein the shared lock comprises a reader-writer lock that further controls access to the critical section of code or shared resource in read-only mode;

and

wherein the method further comprises;

prior to the reader thread of the multithreaded application accessing the critical section of code or shared resource, the reader thread of the multithreaded application acquiring the shared lock in read-only mode.

3. The method of claim 2,

wherein the method further comprises, prior to the reader thread of the multithreaded application acquiring the shared lock in read-only mode, the reader thread of the multithreaded application acquiring ownership of the cluster-specific lock in read-only mode or acquiring ownership of a different cluster-specific lock in read-only mode;

and

wherein said acquiring the shared lock in read-only mode is performed in response to the thread acquiring ownership of the cluster-specific lock in read-only mode or acquiring ownership of a different cluster-specific lock in read-only mode.

4. The method of claim 2,

wherein the reader thread of the multithreaded application is one of a plurality of reader threads of the multithreaded application executing on a plurality of processor cores in a given one of the plurality of clusters of processor cores, wherein the plurality of processor cores in the given cluster of processor cores share a memory;

and

wherein said acquiring the shared lock in read-only mode comprises acquiring the shared lock in read-only mode on behalf of the plurality of reader threads of the multithreaded application.

5. The method of claim 1,

wherein the shared lock comprises a reader-writer lock that further controls access to the critical section of code or shared resource in read-only mode;

and

wherein the method further comprises;

prior to the writer thread of the multithreaded application acquiring the shared lock for writing, another reader thread of the multithreaded application accessing the critical section of code or shared resource in read-only mode while the reader thread of the multithreaded application holds the shared lock in read-only mode.

6. The method of claim 5, further comprising:

prior to the other reader thread of the multithreaded application accessing the critical section of code or shared resource in read-only mode, the other reader thread of the multithreaded application acquiring the shared lock in read-only mode while the reader thread of the multithreaded application holds the shared lock in read-only mode.

7. The method of claim 5, wherein the other reader thread of the multithreaded application is a reader thread of the multithreaded application that is executing on a processor core in one of the plurality of clusters of processor cores other than the cluster of processor cores that comprises the processor core on which the reader thread of the multithreaded application is executing.

8. The method of claim 1, further comprising, subsequent to said passing, the thread to which ownership of the cluster-specific lock for writing was passed performing one or more of:

accessing the critical section of code or shared resource;

releasing the cluster-specific lock;

or

releasing the shared lock.

9. The method of claim 1, wherein said acquiring the shared lock for writing comprises waiting for the reader thread of the multithreaded application to release the shared lock.

10. The method of claim 1, further comprising:

prior to the reader thread of the multithreaded application accessing the critical section of code or shared resource in read-only mode, the reader thread of the multithreaded application determining whether there are any other writer threads of the multithreaded application waiting to acquire the shared lock for writing; and

wherein said accessing the critical section of code or shared resource in read-only mode is performed in further response to determining that there are no other writer threads of the multithreaded application waiting to acquire the shared lock for writing.

11. The method of claim 1, wherein said acquiring the shared lock for writing comprises waiting for one or more currently executing reader threads of the multithreaded application to complete one or more accesses to the critical section of code or shared resource in read-only mode.

12. The method of claim 1, wherein the cluster-specific lock comprises a reader-writer lock.

13. The method of claim 1, wherein the shared lock and one or more cluster-specific locks comprise non-NUMA-aware locks that collectively implement a NUMA-aware composite reader-writer lock usable to manage access to the critical section of code or shared resource.

14. A system, comprising:

a plurality of processor core clusters, each of which comprises two or more processor cores that support multithreading and that share a local memory;

a system memory coupled to the plurality of processor core clusters;

wherein the system memory stores program instructions that when executed on one or more processor cores in the plurality of processor core clusters cause the one or more processor cores to perform;

beginning execution of a multithreaded application that comprises one or more requests to acquire a shared lock, wherein the shared lock controls write access to a critical section of code or a shared resource by concurrently executing threads of the multithreaded application, wherein only one thread of the multithreaded application can hold the shared lock for writing at a time, and wherein each of the concurrently executing threads of the multithreaded application executes on one of the plurality of processor cores in the processor core clusters;

a reader thread of the multithreaded application;

determining that no thread holds the shared lock for writing; and

in response to determining that no thread holds the shared lock for writing;

accessing the critical section of code or shared resource in read-only mode;

subsequent to the reader thread of the multithreaded application accessing the critical section of code or shared resource in read-only mode, a writer thread of the multithreaded application acquiring ownership of a cluster specific lock for writing, and in response to the writer thread of the multithreaded application acquiring ownership of the cluster-specific lock for writing, acquiring the shared lock for writing;

in response to acquiring the shared lock for writing, the writer thread of the multithreaded application;

accessing the critical section of code or shared resource for writing; and

subsequent to said accessing the critical section of code or shared resource for writing;

determining whether at least one other thread of the multithreaded application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread of the multithreaded application is executing is waiting to acquire the shared lock for writing; and

in response to determining that at least one other thread of the multithreaded application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread of the multithreaded application is executing is waiting to acquire the shared lock for writing, passing ownership of the cluster-specific lock for writing to one of the at least one other threads of the multithreaded application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread of the multithreaded application is executing that is waiting to acquire the shared lock for writing without releasing the shared lock.

15. The system of claim 14,

wherein the shared lock comprises a reader-writer lock that further controls access to the critical section of code or shared resource in read-only mode;

and

wherein when executed on the one or more processor cores in the plurality of processor core clusters, the program instructions further cause the one or more processor cores to perform;

prior to the writer thread of the multithreaded application acquiring the shared lock for writing, another reader thread of the multithreaded application accessing the critical section of code or shared resource in read-only mode while the reader thread of the multithreaded application holds the shared lock in read-only mode.

16. The system of claim 14, wherein when executed on the one or more processor cores in the plurality of processor core clusters, the program instructions further cause the one or more processor cores to perform:

subsequent to said passing, the thread to which ownership of the cluster-specific lock for writing was passed performing one or more of;

accessing the critical section of code or shared resource;

releasing the cluster-specific lock;

or

releasing the shared lock.

17. A non-transitory, computer-readable storage medium storing program instructions that when executed on one or more computers cause the one or more computers to perform:

beginning execution of a multithreaded application that comprises one or more requests to acquire a shared lock, wherein the shared lock controls write access to a critical section of code or a shared resource by concurrently executing threads of the multithreaded application, wherein only one thread of the multithreaded application can hold the shared lock for writing at a time, wherein each of the concurrently executing threads of the multithreaded application executes on one of a plurality of processor cores in a cluster of processor cores that is one of a plurality of clusters of processor cores on which the concurrently executing threads of the multithreaded application are executing, and wherein the plurality of processor cores in each of the clusters of processor cores share a memory;

a reader thread of the multithreaded application;

determining that no thread holds the shared lock for writing; and

in response to determining that no thread holds the shared lock for writing;

accessing the critical section of code or shared resource in read-only mode;

subsequent to the reader thread of the multithreaded application accessing the critical section of code or shared resource in read-only mode, a writer thread of the multithreaded application acquiring ownership of a cluster specific lock for writing, and in response to the writer thread of the multithreaded application acquiring ownership of the cluster-specific lock for writing, acquiring the shared lock for writing;

in response to acquiring the shared lock for writing, the writer thread of the multithreaded application;

accessing the critical section of code or shared resource for writing; and

subsequent to said accessing the critical section of code or shared resource for writing;

determining whether at least one other thread of the multithreaded application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread of the multithreaded application is executing is waiting to acquire the shared lock for writing; and

in response to determining that at least one other thread of the multithreaded application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread of the multithreaded application is executing is waiting to acquire the shared lock for writing, passing ownership of the cluster-specific lock for writing to one of the at least one other threads of the multithreaded application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread of the multithreaded application is executing that is waiting to acquire the shared lock for writing without releasing the shared lock.

18. The non-transitory, computer-readable storage medium of claim 17,

wherein the shared lock comprises a reader-writer lock that further controls access to the critical section of code or shared resource in read-only mode;

and

wherein when executed on the one or more computers, the program instructions further cause the one or more computers to perform;

prior to the writer thread of the multithreaded application acquiring the shared lock for writing, another reader thread of the multithreaded application accessing the critical section of code or shared resource in read-only mode while the reader thread of the multithreaded application holds the shared lock in read-only mode.

19. The non-transitory, computer-readable storage medium of claim 17, wherein when executed on the one or more computers, the program instructions further cause the one or more computers to perform:

subsequent to said passing, the thread to which ownership of the cluster-specific lock for writing was passed performing one or more of;

accessing the critical section of code or shared resource;

releasing the cluster-specific lock;

or

releasing the shared lock.