Dynamic allocation and partitioning of compute nodes in hierarchical job scheduling

• US 8,381,212 B2
• Filed: 10/09/2007
• Issued: 02/19/2013
• Est. Priority Date: 10/09/2007
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brelsford, David P., Chan, Waiman, Hughes, Stephen C., Marthi, Kailash N., Sure, Ravindra R.
• Primary Examiner(s): To, Jennifer
• Assistant Examiner(s): WU, BENJAMIN C
• Application Number:
  US11/869,133
• Publication Number:
  US 20090094605A1
• Time in Patent Office: 1,960 Days
• Field of Search: None
• US Class Current: 718/100
• CPC Class Codes: G06F 9/4843 by program, e.g. task dispatcher, supervisor, operating system

Abstract

The present invention employs a master node for each job to be scheduled and in turn the master node distributes job start information and executable tasks to a plurality of nodes configured in a hierarchical node tree of a multinode job scheduling system. The status of the various tasks executing at the leaf nodes and other nodes of the tree report status back up the same hierarchical tree structure used to start the job, not to a scheduling agent but rather to the master node which has been established by the scheduling agent as the focal point, not only for job starting, but also for the reporting of status information from the leaf and other nodes in the tree.

Claims

1. A method of reporting job status comprising:

obtaining from a parent node, by a plurality of compute nodes, one or more portions of a job to be executed by the plurality of compute nodes, together with information indicating a plurality of child nodes that are children to the plurality of compute nodes that are available to accept at least one portion of the job, the plurality of compute nodes and the plurality of child nodes being in a hierarchical relationship, wherein the plurality of compute nodes are higher in the hierarchical relationship than the plurality of child nodes and wherein the parent node is higher in the hierarchical relationship than the plurality of compute nodes;

executing by the plurality of compute nodes at least one portion of the job;

forwarding by the plurality of compute nodes to the plurality of child nodes at least one portion of the job to be executed by the plurality of child nodes;

receiving by a node of the plurality of compute nodes from at least one child node of the plurality of child nodes status information regarding the at least one portion of the job executed by the at least one child node;

consolidating by the node the status information received from the at least one child node with status information regarding the at least one portion of the job executed by the node to provide consolidated status information, the consolidated status information including status information from the at least one child node and from the node itself; and

providing to the parent node the consolidated status information.

2. The method of claim 1, wherein the node includes an agent for receiving the status information.

3. The method of claim 2, wherein the agent is able to concurrently process status information from at least two jobs.

4. The method of claim 1, wherein the node, during execution of the job, retains an indication of the plurality of child nodes of the node.

5. The method of claim 1, wherein the status information includes at least one of node readiness, job start, job completion and resource consumption.

6. The method of claim 1, wherein the hierarchical relationship includes a tree structure.

7. The method of claim 6, wherein the tree structure is a binary tree.

8. The method of claim 1, further including providing by the parent node a consolidated status report to a scheduling agent.

9. The method of claim 1, further comprising:

consolidating by the parent node the consolidated status information with status information received from at least one other compute node in the plurality of compute nodes to provide further consolidated status information; and

providing the further consolidated status information to a given node higher in the hierarchical relationship than the parent node.

10. The method of claim 1, further comprising:

forwarding by the plurality of compute nodes to the plurality of child nodes together with the at least one portion of the job to be executed by the plurality of child nodes, information indicating a plurality of grandchild nodes that are children to the plurality of child nodes and that are available to accept at least one portion of the job.

11. A computer system for reporting job status comprising:

a memory; and

a processor in communications with the memory, wherein the computer system is configured to perform a method comprising;

obtaining from a parent node, by a plurality of compute nodes, one or more portions of a job to be executed by the plurality of compute nodes, together with information indicating a plurality of child nodes that are children to the plurality of compute nodes that are available to accept at least one portion of the job, the plurality of compute nodes and the plurality of child nodes being in a hierarchical relationship, wherein the plurality of compute nodes are higher in the hierarchical relationship than the plurality of child nodes and wherein the parent node is higher in the hierarchical relationship than the plurality of compute nodes;

executing by the plurality of compute nodes at least one portion of the job;

forwarding by the plurality of compute nodes to the plurality of child nodes at least one portion of the job to be executed by the plurality of child nodes;

receiving by a node of the plurality of compute nodes from at least one child node of the plurality of child nodes status information regarding the at least one portion of the job executed by the at least one child node;

consolidating by the node the status information received from the at least one child node with status information regarding the at least one portion of the job executed by the node to provide consolidated status information, the consolidated status information including status information from the at least one child node and from the node itself; and

providing to the parent node the consolidated status information.

12. The computer system of claim 11, wherein the status information includes at least one of node readiness, job start, job completion and resource consumption.

13. The computer system of claim 11, wherein the hierarchical relationship includes a tree structure, and wherein the tree structure is a binary tree.

14. The computer system of claim 11, wherein the method further comprises providing by the parent node a consolidated status report to a scheduling agent.

15. The computer system of claim 11, wherein the node includes an agent for receiving the status information, and wherein the agent is able to concurrently process status information from at least two jobs.

16. A computer program product for reporting job status comprising:

a non-transitory storage medium readable by a processor and storing instructions for execution by the processor for performing a method comprising;

obtaining from a parent node, by a plurality of compute nodes, one or more portions of a job to be executed by the plurality of compute nodes, together with information indicating a plurality of child nodes that are children to the plurality of compute nodes that are available to accept at least one portion of the job, the plurality of compute nodes and the plurality of child nodes being in a hierarchical relationship, wherein the plurality of compute nodes are higher in the hierarchical relationship than the plurality of child nodes and wherein the parent node is higher in the hierarchical relationship than the plurality of compute nodes;

executing by the plurality of compute nodes at least one portion of the job;

forwarding by the plurality of compute nodes to the plurality of child nodes at least one portion of the job to be executed by the plurality of child nodes;

receiving by a node of the plurality of compute nodes from at least one child node of the plurality of child nodes status information regarding the at least one portion of the job executed by the at least one child node;

consolidating by the node the status information received from the at least one child node with status information regarding the at least one portion of the job executed by the node to provide consolidated status information, the consolidated status information including status information from the at least one child node and from the node itself; and

providing to the parent node the consolidated status information.

17. The computer program product of claim 16, wherein the status information includes at least one of node readiness, job start, job completion and resource consumption.

18. The computer program product of claim 16, wherein the hierarchical relationship includes a tree structure, and wherein the tree structure is a binary tree.

19. The computer program product of claim 16, wherein the method further comprises providing by the parent node a consolidated status report to a scheduling agent.

20. The computer program product of claim 16, wherein the node includes an agent for receiving the status information, and wherein the agent is able to concurrently process status information from at least two jobs.

21. The computer program product of claim 16, wherein the method further comprises:

consolidating by the parent node the consolidated status information with status information received from at least one other compute node in the plurality of compute nodes to provide further consolidated status information; and

providing the further consolidated status information to a given node higher in the hierarchical relationship than the parent node.