Mechanism for enabling the distribution of operating system resources in a multi-node computer system

• US 7,574,708 B2
• Filed: 03/04/2004
• Issued: 08/11/2009
• Est. Priority Date: 03/04/2004
• Status: Active Grant

• Current Assignee: ServiceNow Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Vossen, Kenneth Charles, Francois, Christopher, Kirkman, Richard Karl, Chung, Lynn Keat, McCarthy, Patrick Joseph, Reed, Don Darrell
• Primary Examiner(s): Nguyen; Van H
• Assistant Examiner(s): WAI, ERIC CHARLES
• Application Number:
  US10/793,476
• Publication Number:
  US 20050210470A1
• Time in Patent Office: 1,986 Days
• Field of Search: 709/227, 718104-105
• US Class Current: 718/104
• CPC Class Codes: G06F 2209/5019 Workload prediction
• CPC Class Codes: G06F 9/505 considering the load
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system

Abstract

Disclosed is an apparatus, method, and program product that enables distribution of operating system resources on a nodal basis in the same proportions as the expected system workload. The preferred embodiment of the present invention accomplishes this by assigning various types of weights to each node to represent their proportion of the overall balance within the system. Target Weights represent the desired distribution of the workload based on the existing proportions of processor and memory resources on each node. The actual workload balance on the system is represented by Current Weights, which the operating system strives to keep as close to the Target Weights as possible, on an ongoing basis. When the system is started, operating system services distribute their resources nodally in the same proportions as the Target Weights, and can request to be notified if the Target Weights ever change. If processors and/or memory are subsequently added or removed, new Target Weights are calculated at that time, and all services which requested notification are notified so they can redistribute their resources according to the new Target Weights or a stepwise refinement thereof.

Claims

1. A computer-implemented method, said method comprising the steps of:

ascertaining a respective at least one workload measure corresponding to a first processor and to a second processor of a computer apparatus, said first processor having first memory associated therewith and said second processor having second memory associated therewith, wherein said first processor and first memory form a first node of said computer apparatus and wherein said second processor and said second memory form a second node of said computer apparatus, wherein said respective at least one workload measure is derived from a respective first quantitative value representing a respective desired workload for the node containing the corresponding processor and a respective second quantitative value representing a respective current workload experienced by the node containing the corresponding processor, said respective current workloads representing the actual distribution of work within said computer apparatus; and

notifying a service provider of said respective workload, said service provider providing a plurality of kernel services of an operating system of said computer apparatus, said plurality of kernel services performed by a plurality of service provider threads, each said service provider thread being executable on any of said first processor and said second processor and being associated with a respective at least one processor, each said service provider thread being preferentially dispatched for execution to a processor with which the service provider thread is associated, said notifying a service provider permitting said service provider to adjust service provider resources by changing a processor association of at least one service provider thread of said plurality of service provider threads according to said respective workload, said service provider providing said plurality of kernel services to entities that execute on said first and second processors.

2. The method of claim 1 wherein said respective desired work load is represented by a first target weight for said first node and a second target weight for said second node.

3. The method of claim 1 wherein wherein said respective current work load is represented by a current weight for said first node and a second current weight for said second node.

4. The method of claim 1 wherein wherein said respective desired work load is represented by a first target weight on said first node and a second target weight on said second node, and wherein said respective current work load is represented by a current weight for said first node and a second current weight for said second node.

5. A computer-implemented method, said method comprising the steps of:

ascertaining a respective desired workload corresponding to a first processor and to a second processor of a computer apparatus, wherein said first processor and a first memory form a first node of said computer apparatus and wherein said second processor and a second memory form a second node of said computer apparatus, said desired workload being a first quantitative value representing a respective desired workload for the node containing the corresponding processor;

ascertaining a respective current workload experienced by the node containing said first processor and by the node containing said second processor, said respective current workloads being second quantitative values representing the actual distribution of work within said computer apparatus;

deriving a respective intermediate workload from said respective desired workload and from said respective current workload; and

notifying a service provider of said respective intermediate workload, said service provider providing a plurality of kernel services of an operating system of said computer apparatus, said plurality of kernel services performed by a plurality of service provider threads, each said service provider thread being executable on any of said first processor and said second processor and being associated with a respective at least one processor, each said service provider thread being preferentially dispatched for execution to a processor with which the service provider thread is associated, said notifying a service provider permitting said service provider to adjust service provider resources by changing a processor association of at least one service provider thread of said plurality of service provider threads according to said respective intermediate workload, said service provider providing said plurality of kernel services to entities that execute on said first and second processors.

6. The method of claim 5 wherein said respective current workload is adjusted over time such that said respective current workload more closely matches said respective desired workload and wherein said respective intermediate workload is periodically updated to reflect adjustments to said respective current workload.

7. The method of claim 5 wherein said respective desired workload is represented by a target weight for said first node and a target weight for said second node, and wherein said respective current workload is represented by a first current weight for said first node and a second current weight for said second node, and wherein said respective intermediate workload is represented by a first transition weight for said first node and by a second transition weight for said second node.

8. A computer-implemented method, said method comprising the steps of:

ascertaining a respective at least one workload measure corresponding to a first processor and to a second processor of a computer apparatus, said first processor having first memory associated therewith and said second processor having second memory associated therewith, wherein said respective at least one workload measure is derived from a respective desired workload for the corresponding processor and a respective current workload experienced by the corresponding processor; and

notifying a service provider of said respective at least one workload measure, said service provider providing a plurality of kernel services of an operating system of said computer apparatus, said plurality of kernel services performed by a plurality of service provider threads, each said service provider thread being executable on any of said first processor and said second processor and being associated with a respective at least one processor, each said service provider thread being preferentially dispatched for execution to a processor with which the service provider thread is associated; and

responsive to notifying a service provider of said respective at least one workload measure, adjusting service provider resources by changing a processor association of at least one service provider thread of said plurality of service provider threads according to said respective at least one workload measure, said service provider providing said plurality of kernel services to entities that execute on said first and second processors;

wherein said first processor and first memory form a first node and wherein said second processor and said second memory form a second node and wherein said respective desired work load is represented by a first target weight on said first node and a second target weight on said second node, and wherein said respective current workload is represented by a current weight for said first node and a second current weight for said second node.

9. A computer apparatus, said computer apparatus comprising:

a plurality of processors;

a service provider for providing a plurality of kernel services of an operating system of said computer apparatus to entities that execute on said plurality of processors, said plurality of kernel services being performed by plurality of service provider threads, each said service provider thread being executable of any of said processors and being associated with a respective processor subset of a plurality of disjoint subsets of said plurality of processors, each said processor subset containing at least one respective processor;

wherein a first processor subset and a first subset of memory of said computer apparatus form a first node of said computer apparatus and wherein a second processor subset and a second subset of memory of said computer apparatus form a second node of said computer apparatus, said first and second subsets of memory being disjoint;

a workload management mechanism that ascertains respective current workload experienced by each said subset of processors, said respective current workloads being first quantitative values representing the actual distribution of work within said computer apparatus, said workload management mechanism notifying said service provider of a respective at least workload measure associated with each said node, said respective at least one workload measure being derived from a respective at least one second quantitative value representing a respective desired workload for the corresponding node and each said respective current workload;

wherein, responsive to notification of said respective at least one workload measure by said workload management mechanism, said service provider adjusts service provider resources by changing a processor subset association of at least one service provider thread of said plurality of service provider threads according to said respective workload; and

a dispatch mechanism which preferentially dispatches said service provider threads to execute on processors within the respective processor subset with which each said service provider thread is associated.

10. The computer apparatus of claim 9 wherein said service provider service provider adjusts said service provider resources according to a difference between said respective current workload and said respective desired workload.

11. The computer apparatus of claim 10 wherein said respective desired workload is represented by a first target weight for said first node and a second target weight for said second node.

12. The computer apparatus of claim 11 wherein said respective current workload is represented by a first current weight for said first node and a second current weight for said second node.

13. The computer apparatus of claim 9, wherein said workload management mechanism derives a respective intermediate workload for each said processor subset from said respective desired workload and from said respective current workload, said workload management mechanism notifying said service provider of said respective intermediate workload, said service provider adjusting service provider resources according to said respective intermediate workload.

14. The computer apparatus of claim 13 wherein said respective current workload is adjusted over time such that said respective current workload more closely matches said respective desired workload and wherein said respective intermediate workload is periodically updated to reflect adjustments to said respective current workload.

15. The computer apparatus of claim 14 wherein said respective desired workload is represented by a first target weight for said first node and a second target weight for said second node, and wherein said respective current workload is represented by a first current weight for said first node and a second current weight for said second node, and wherein said respective intermediate workload is represented by a first transition weight for said first node and by a second transition weight for said second node.

16. A program product, said program product comprising:

tangible computer-readable storage media, and

a workload management mechanism disposed on said tangible computer-readable storage media, said workload management mechanism ascertaining a respective current workload experienced by each processor subset of a plurality of disjoint processor subsets of a plurality of processors of a computer apparatus, said current workloads being first quantitative values representing the actual distribution of work within said computer apparatus, each said processor subset containing at least one respective processor, wherein a first processor subset and a first subset of memory of a computer apparatus form a first node of said computer apparatus and wherein a second processor subset and a second subset of memory of said computer apparatus form a second node of said computer apparatus, said first and second subsets of memory being disjoint, said workload management mechanism notifying a service provider of at least one respective workload measure associated with each said processor subset, said respective at least one workload measure being derived from a respective at least one second quantitative value representing a respective desired share of work for the corresponding node and each said respective current workload, thereby permitting said service provider to adjust service provider resources by changing a processor subset association of at least one service provider thread of a plurality of service provider threads according to said respective at least one workload measure;

wherein said service provider provides a plurality of kernel services of an operating system of said computer apparatus to entities that execute on said plurality of processors, said plurality of kernel services being performed by said plurality of service provider threads, each said service provider thread being executable on any of said processors and being associated with a respective processor subset of said plurality of disjoint processor subsets, each said service provider thread being preferentially dispatched for execution to the at least one processor within the respective subset of processors with which the service provider thread is associated.

17. The program product of claim 16 wherein said respective desired workload is represented by a first target weight for said first node and a second target weight for said second node.

18. The program product of claim 17 wherein said respective current workload is represented by a current weight for said first node and a second current weight for said second node.

19. The program product of claim 16, wherein said workload management mechanism derives a respective intermediate workload associated with each said processor subset from the respective desired workload associated with the processor subset and the respective current workload experienced by the processor subset, said workload management mechanism notifying a service provider of said respective intermediate workload thereby permitting said service provider to adjust service provider resources according to said respective intermediate workload.

20. The program product of claim 19 wherein said respective current workload is adjusted over time such that said respective current workload more closely matches said respective desired workload and wherein said respective intermediate workload is periodically updated to reflect adjustments to said respective current workload.

21. The program product of claim 19 wherein said respective desired workload is represented by a first target weight for said first node and a second target weight for said second node, and wherein said respective current workload is represented by a first current weight for said first node and a second current weight for said second node, and wherein said respective intermediate workload is represented by a first transition weight for said first node and by a second transition weight for said second node.