Scalable RCU callback offloading

• US 9,886,329 B2
• Filed: 06/25/2015
• Issued: 02/06/2018
• Est. Priority Date: 06/25/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: McKenney, Paul E.
• Primary Examiner(s): Tsai, Henry
• Assistant Examiner(s): Daley, Christopher A
• Application Number:
  US14/750,340
• Publication Number:
  US 20160378573A1
• Time in Patent Office: 957 Days
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/526 Mutual exclusion algorithms

Abstract

In order to scale Read-Copy Update (RCU) callback offloading from no-callbacks (No-CBs) CPUs, a set of RCU callback offload kernel threads (rcuo kthreads) may be spawned and each may be assigned to one of the No-CBs CPUs to invoke RCU callbacks generated by workloads running on the No-CBs CPUs at CPUs that are not No-CBs CPUs. Groups of the rcuo kthreads may be established, with each rcuo kthread group having one leader kthread and one or more follower rcuo kthreads. The leader rcuo kthreads may be periodically awakened without waking up the follower kthreads when an RCU grace period ends and an RCU callback needs to be invoked, or when a new RCU callback arrives and a new RCU grace period needs to be started. The leader rcuo kthreads may periodically awaken their associated follower rcuo kthreads for which the leader rcuo kthreads have sole responsibility to wake.

Claims

1. A method for scaling Read-Copy Update (RCU) callback offloading, comprising:

designating a set of CPUs in a computer system as no-callbacks (No-CBs) CPUs that do not perform RCU callback processing;

spawning a set of RCU callback offload kernel threads (rcuo kthreads) that are each assigned to one of said No-CBs CPUs to invoke RCU callbacks generated by workloads running on said No-CBs CPUs at CPUs that are not No-CBs CPUs;

establishing groups of said rcuo kthreads, each rcuo kthread group having one leader rcuo kthread and one or more follower rcuo kthreads;

periodically waking up said leader rcuo kthreads without waking up said follower rcuo kthreads when an RCU grace period ends and an RCU callback needs to be invoked, or when a new RCU callback arrives and a new RCU grace period needs to be started; and

said leader rcuo kthreads periodically waking up their associated follower rcuo kthreads for which said leader rcuo kthreads have sole responsibility to wake.

2. The method of claim 1, wherein the number of said rcuo kthread groups is substantially the square root of the number of said CPUs.

3. The method of claim 1, wherein each of said No-CBs CPUs has a set of associated callback queues that accumulate RCU callbacks to be processed at different times, and wherein each of said leader rcuo kthreads advances said RCU callbacks through said callback queues on behalf of each rcuo kthread in its group until said RCU callbacks are ready for invocation.

4. The method of claim 3, wherein said rcuo kthreads each have an associated callback queue for said RCU callbacks that are actively being invoked, said leader rcuo kthreads and said follower rcuo kthreads each managing their own said associated callback queue.

5. The method of claim 3, wherein said leader rcuo kthreads wake up their follower rcuo kthreads when said follower rcuo kthreads have RCU callbacks that are ready for invocation so that said follower rcuo kthreads can invoke their RCU callbacks.

6. The method of claim 4, wherein said leader rcuo kthreads perform any necessary RCU grace period waiting on behalf of their follower rcuo kthreads so that said follower rcuo kthreads do not need to separately wait for RCU grace periods.

7. The method of claim 1, wherein said leader rcuo kthreads use kthread status flags and memory barriers to ensure that callback handoffs to said follower rcuo kthreads avoid races with new postings of said RCU callbacks.

8. A system, comprising:

a plurality of CPUs;

a memory coupled to said CPUs, said memory including a computer readable storage medium tangibly embodying at least one program of instructions executable by said CPUs to perform operations for scaling Read-Copy Update (RCU) callback offloading, said operations comprising;

designating a set of said CPUs in said system as no-callbacks (No-CBs) CPUs that do not perform RCU callback processing;

spawning a set of RCU callback offload kernel threads (rcuo kthreads) that are each assigned to one of said No-CBs CPUs to invoke RCU callbacks generated by workloads running on said No-CBs CPUs at CPUs that are not No-CBs CPUs;

establishing groups of said rcuo kthreads, each rcuo kthread group having one leader rcuo kthread and one or more follower rcuo kthreads;

periodically waking up said leader rcuo kthreads without waking up said follower rcuo kthreads when an RCU grace period ends and an RCU callback needs to be invoked, or when a new RCU callback arrives and a new RCU grace period needs to be started; and

said leader rcuo kthreads periodically waking up their associated follower rcuo kthreads for which said leader rcuo kthreads have sole responsibility to wake.

9. The system of claim 8, wherein the number of said rcuo kthread groups is substantially the square root of the number of said CPUs.

10. The system of claim 8, wherein each of said No-CBs CPUs has a set of associated callback queues that accumulate RCU callbacks to be processed at different times, and wherein each of said leader rcuo kthreads advances said RCU callbacks through said callback queues on behalf of each rcuo kthread in its group until said RCU callbacks are ready for invocation.

11. The system of claim 10, wherein said rcuo kthreads each have an associated callback queue for said RCU callbacks that are actively being invoked, said leader rcuo kthreads and said follower rcuo kthreads each managing their own said associated callback queue.

12. The system of claim 10, wherein said leader rcuo kthreads wake up their follower rcuo kthreads when said follower rcuo kthreads have RCU callbacks that are ready for invocation so that said follower rcuo kthreads can invoke their RCU callbacks.

13. The method of claim 12, wherein said leader rcuo kthreads perform any necessary RCU grace period waiting on behalf of their follower rcuo kthreads so that said follower rcuo kthreads do not need to separately wait for RCU grace periods.

14. The system of claim 8, wherein said leader rcuo kthreads use kthread status flags and memory barriers to ensure that callback handoffs to said follower rcuo kthreads avoid races with new postings of said RCU callbacks.

15. A computer program product, comprising:

one or more computer readable data storage media;

program instructions stored on said one or more computer readable data storage media for programming a data processing platform having a plurality of CPUs to perform operations for scaling Read-Copy Update (RCU) callback offloading, said operations comprising;

designating a set of said CPUs on said data processing platform as no-callbacks (No-CBs) CPUs that do not perform RCU callback processing;

spawning a set of RCU callback offload kernel threads (rcuo kthreads) that are each assigned to one of said No-CBs CPUs to invoke RCU callbacks generated by workloads running on said No-CBs CPUs at CPUs that are not No-CBs CPUs;

establishing groups of said rcuo kthreads, each rcuo kthread group having one leader rcuo kthread and one or more follower rcuo kthreads;

periodically waking up said leader rcuo kthreads without waking up said follower rcuo kthreads when an RCU grace period ends and an RCU callback needs to be invoked, or when a new RCU callback arrives and a new RCU grace period needs to be started; and

said leader rcuo kthreads periodically waking up their associated follower rcuo kthreads for which said leader rcuo kthreads have sole responsibility to wake.

16. The computer program product of claim 15, wherein the number of said rcuo kthread groups is substantially the square root of the number of said CPUs.

17. The computer program product of claim 15, wherein each of said No-CBs CPUs has a set of associated callback queues that accumulate RCU callbacks to be processed at different times, and wherein each of said leader rcuo kthreads advances said RCU callbacks through said callback queues on behalf of each rcuo kthread in its group until said RCU callbacks are ready for invocation.

18. The computer program product of claim 17, wherein said rcuo kthreads each have an associated callback queue for said RCU callbacks that are actively being invoked, said leader rcuo kthreads and said follower rcuo kthreads each managing their own said associated callback queue.

19. The computer program product of claim 17, wherein said leader rcuo kthreads wake up their follower rcuo kthreads when said follower rcuo kthreads have RCU callbacks that are ready for invocation so that said follower rcuo kthreads can invoke their RCU callbacks, and wherein said leader rcuo kthreads perform any necessary RCU grace period waiting on behalf of their follower rcuo kthreads so that said follower rcuo kthreads do not need to separately wait for RCU grace periods.

20. The computer program product of claim 15, wherein said leader rcuo kthreads use kthread status flags and memory barriers to ensure that callback handoffs to said follower rcuo kthreads avoid races with new postings of said RCU callbacks.