System and Method for Implementing NUMA-Aware Reader-Writer Locks

• US 20130290967A1
• Filed: 04/27/2012
• Published: 10/31/2013
• Est. Priority Date: 04/27/2012
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Dice, David, Luchangco, Victor M., Marathe, Virendra J., Shavit, Nir N., Lev, Yosef, Calciu, Irina
• Granted Patent: US 8,966,491 B2
• Time in Patent Office: Days
• US Class Current: 718/102
• CPC Class Codes: G06F 2209/523 Mode
• CPC Class Codes: G06F 9/526 Mutual exclusion algorithms

Abstract

NUMA-aware reader-writer locks may leverage lock cohorting techniques to band together writer requests from a single NUMA node. The locks may relax the order in which the lock schedules the execution of critical sections of code by reader threads and writer threads, allowing lock ownership to remain resident on a single NUMA node for long periods, while also taking advantage of parallelism between reader threads. Threads may contend on node-level structures to get permission to acquire a globally shared reader-writer lock. Writer threads may follow a lock cohorting strategy of passing ownership of the lock in write mode from one thread to a cohort writer thread without releasing the shared lock, while reader threads from multiple NUMA nodes may simultaneously acquire the shared lock in read mode. The reader-writer lock may follow a writer-preference policy, a reader-preference policy or a hybrid policy.

Claims

1. A method, comprising:

performing by a computer;

beginning execution of a multithreaded application that comprises one or more requests to acquire a shared lock, wherein the shared lock controls write access to a critical section of code or a shared resource by concurrently executing threads of the application, wherein only one thread can hold the shared lock for writing at a time, and wherein each of the concurrently executing threads of the application executes on one of a plurality of processor cores in a cluster of processor cores that share a memory, and wherein the cluster of processor cores is one of a plurality of clusters of processor cores on which threads of the multithreaded application are executing;

a reader thread of the application;

determining that no thread holds the shared lock for writing; and

in response to determining that no thread holds the shared lock for writing;

accessing the critical section of code or shared resource in read-only mode;

subsequent to the reader thread accessing the critical section of code or shared resource in read-only mode, a writer thread of the application acquiring the shared lock for writing;

in response to acquiring the shared lock for writing, the writer thread;

accessing the critical section of code or shared resource for writing; and

subsequent to said accessing the critical section of code or shared resource for writing;

determining whether any other threads of the application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread is executing are waiting to acquire the shared lock for writing; and

in response to determining that at least one other thread of the application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread is executing is waiting to acquire the shared lock for writing, passing ownership of a cluster-specific lock for writing to another thread of the application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread is executing that is waiting to acquire the shared lock for writing without releasing the shared lock.

2. The method of claim 1,

wherein the shared lock comprises a reader-writer lock that further controls access to the critical section of code or shared resource in read-only mode;

and

wherein the method further comprises;

prior to the reader thread accessing the critical section of code or shared resource, the reader thread acquiring the shared lock in read-only mode.

3. The method of claim 2,

wherein the method further comprises, prior to the reader thread acquiring the shared lock in read-only mode, the reader thread acquiring ownership of a cluster-specific lock in read-only mode;

and

wherein said acquiring the shared lock in read-only mode is performed in response to the thread acquiring ownership of a cluster-specific lock in read-only mode.

4. The method of claim 2,

wherein the reader thread is one of a plurality of reader threads executing on a plurality of processor cores in a given cluster of processor cores that share a memory;

and

wherein said acquiring the shared lock in read-only mode comprises acquiring the shared lock in read-only mode on behalf of the plurality of reader threads.

5. The method of claim 1,

wherein the shared lock comprises a reader-writer lock that further controls access to the critical section of code or shared resource in read-only mode;

and

wherein the method further comprises;

prior to the writer thread acquiring the shared lock for writing, another reader thread accessing the critical section of code or shared resource in read-only mode while the reader thread holds the shared lock in read-only mode.

6. The method of claim 5, further comprising:

prior to the other reader thread accessing the critical section of code or shared resource in read-only mode, the other reader thread acquiring the shared lock in read mode while the reader thread holds the shared lock in read-only mode.

7. The method of claim 5, wherein the other reader thread is a reader thread that is executing on a processor core in a cluster of processor cores other than the cluster of processor cores that comprises the processor core on which the reader thread is executing.

8. The method of claim 1,

wherein the method further comprises, prior to the writer thread acquiring the shared lock for writing, the writer thread acquiring ownership of the cluster-specific lock for writing;

and

wherein said acquiring the shared lock for writing is performed in response to the writer thread acquiring ownership of the cluster-specific lock for writing.

9. The method of claim 1, further comprising, subsequent to said passing, the other thread performing one or more of:

accessing the critical section of code or shared resource;

releasing the cluster-specific lock;

or

releasing the shared lock.

10. The method of claim 1, wherein said acquiring the shared lock for writing comprises waiting for the reader thread to release the shared lock.

11. The method of claim 1, further comprising:

prior to the reader thread accessing the critical section of code or shared resource in read-only mode, the reader thread determining whether there are any other writer threads waiting to acquire the shared lock for writing; and

wherein said accessing the critical section of code or shared resource in read-only mode is performed in further response to determining that there are no other writer threads waiting to acquire the shared lock for writing.

12. The method of claim 1, wherein said acquiring the shared lock for writing comprises waiting for one or more currently executing reader threads to complete one or more accesses to the critical section of code or shared resource in read-only mode.

13. The method of claim 1, wherein the cluster-specific lock comprises a reader-writer lock.

14. The method of claim 1, wherein the shared lock and one or more cluster-specific locks comprise non-NUMA-aware locks that collectively implement a NUMA-aware composite reader-writer lock usable to manage access to the critical section of code or shared resource.

15. A system, comprising:

a plurality of processor core clusters, each of which comprises two or more processor cores that support multithreading and that share a local memory;

a system memory coupled to the plurality of processor core clusters;

wherein the system memory stores program instructions that when executed on one or more processor cores in the plurality of processor core clusters cause the one or more processor cores to perform;

beginning execution of a multithreaded application that comprises one or more requests to acquire a shared lock, wherein the shared lock controls write access to a critical section of code or a shared resource by concurrently executing threads of the application, wherein only one thread can hold the shared lock for writing at a time, and wherein the each of the concurrently executing threads of the application executes on one of the plurality of processor cores in the processor core clusters;

a reader thread of the application;

determining that no thread holds the shared lock for writing; and

in response to determining that no thread holds the shared lock for writing;

accessing the critical section of code or shared resource in read-only mode;

subsequent to the reader thread accessing the critical section of code or shared resource in read-only mode, a writer thread of the application acquiring the shared lock for writing;

in response to acquiring the shared lock for writing, the writer thread;

accessing the critical section of code or shared resource for writing; and

subsequent to said accessing the critical section of code or shared resource for writing;

determining whether any other threads of the application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread is executing are waiting to acquire the shared lock for writing; and

in response to determining that at least one other thread of the application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread is executing is waiting to acquire the shared lock for writing, passing ownership of a cluster-specific lock for writing to another thread of the application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread is executing that is waiting to acquire the shared lock for writing without releasing the shared lock.

16. The system of claim 15,

wherein the shared lock comprises a reader-writer lock that further controls access to the critical section of code or shared resource in read-only mode;

and

wherein when executed on the one or more processor cores in the plurality of processor core clusters, the program instructions further cause the one or more processor cores to perform;

prior to the writer thread acquiring the shared lock for writing, another reader thread accessing the critical section of code or shared resource in read-only mode while the reader thread holds the shared lock in read-only mode.

17. The system of claim 15, wherein when executed on the one or more processor cores in the plurality of processor core clusters, the program instructions further cause the one or more processor cores to perform:

prior to the writer thread acquiring the shared lock for writing, the writer thread acquiring ownership of the cluster-specific lock for writing; and

subsequent to said passing, the other thread performing one or more of;

accessing the critical section of code or shared resource;

releasing the cluster-specific lock;

or

releasing the shared lock.

18. A non-transitory, computer-readable storage medium storing program instructions that when executed on one or more computers cause the one or more computers to perform:

beginning execution of a multithreaded application that comprises one or more requests to acquire a shared lock, wherein the shared lock controls write access to a critical section of code or a shared resource by concurrently executing threads of the application, wherein only one thread can hold the shared lock for writing at a time, and wherein the each of the concurrently executing threads of the application executes on one of a plurality of processor cores in a cluster of processor cores that share a memory, and wherein the cluster of processor cores is one of a plurality of clusters of processor cores on which threads of the multithreaded application are executing;

a reader thread of the application;

determining that no thread holds the shared lock for writing; and

in response to determining that no thread holds the shared lock for writing;

accessing the critical section of code or shared resource in read-only mode;

subsequent to the reader thread accessing the critical section of code or shared resource in read-only mode, a writer thread of the application acquiring the shared lock for writing;

in response to acquiring the shared lock for writing, the writer thread;

accessing the critical section of code or shared resource for writing; and

subsequent to said accessing the critical section of code or shared resource for writing;

determining whether any other threads of the application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread is executing are waiting to acquire the shared lock for writing; and

in response to determining that at least one other thread of the application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread is executing is waiting to acquire the shared lock for writing, passing ownership of a cluster-specific lock for writing to another thread of the application executing on a processor core in the cluster of processor cores that comprises the processor core on which the writer thread is executing that is waiting to acquire the shared lock for writing without releasing the shared lock.

19. The non-transitory, computer-readable storage medium of claim 18,

wherein the shared lock comprises a reader-writer lock that further controls access to the critical section of code or shared resource in read-only mode;

and

wherein when executed on the one or more computers, the program instructions further cause the one or more computers to perform;

prior to the writer thread acquiring the shared lock for writing, another reader thread accessing the critical section of code or shared resource in read-only mode while the reader thread holds the shared lock in read-only mode.

20. The non-transitory, computer-readable storage medium of claim 18, wherein when executed on the one or more computers, the program instructions further cause the one or more computers to perform:

prior to the writer thread acquiring the shared lock for writing, the writer thread acquiring ownership of the cluster-specific lock for writing; and

subsequent to said passing, the other thread performing one or more of;

accessing the critical section of code or shared resource;

releasing the cluster-specific lock;

or

releasing the shared lock.