System and method for cluster management based on HPC architecture

• US 9,178,784 B2
• Filed: 04/15/2004
• Issued: 11/03/2015
• Est. Priority Date: 04/15/2004
• Status: Active Grant

• Current Assignee: Raytheon Company (Rtx Corporation)
• Original Assignee: Raytheon Company (Rtx Corporation)
• Inventors: Ballew, James D., Davidson, Shannon V., Richoux, Anthony N.
• Primary Examiner(s): GREENE, JOSEPH L
• Application Number:
  US10/825,539
• Publication Number:
  US 20050251567A1
• Time in Patent Office: 4,219 Days
• Field of Search: 709/223, 709/220
• US Class Current: 1/1
• CPC Class Codes: G06F 2209/505 Clust
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 63/101 Access control lists [ACL]
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

Cluster management software comprises a plurality of cluster agents, with each cluster agent associated with an HPC node including an integrated fabric and the cluster agent operable to determine a status of the associated HPC node. The software further includes a cluster management engine communicably coupled with the plurality of the HPC nodes and operable to execute an HPC job using a dynamically allocated subset of the plurality of HPC nodes based on the determined status of the plurality of HPC nodes.

Claims

1. A method, comprising:

determining, by a cluster management engine, available space in a virtual cluster of a plurality of communicatively coupled nodes included in a computing environment, each node including a cluster agent in communication with the cluster management engine, the virtual cluster associated with a group of users that submit similar jobs, and comprising a logical grouping of nodes configured to process related jobs;

prior to job execution, determining an optimum job that is compatible with the available space in the virtual cluster of nodes; and

following the determining, executing the optimum job in the available space in the virtual cluster of nodes;

wherein determining the optimum job that is compatible with the available space in the virtual cluster of nodes further comprises;

determining a number of available nodes in the virtual cluster;

selecting a first job from a job queue;

dynamically determining an optimum shape of the first job;

determining whether the number of available nodes is enough to execute the first job, based on the optimum shape thereof; and

dynamically allocating one or more of the available nodes for the first job, in the event that the determined number of available nodes is enough to execute the first job;

wherein the optimum shape comprises one or more of;

a best fit cube in which the one or more available nodes are allocated in a cubic volume so as to allow cooperating tasks to exchange data with any other tasks by minimizing the distance between any two nodes; and

a best fit sphere in which the one or more available nodes are allocated in a spherical volume such that a first task is placed in a center node of the sphere with remaining tasks placed on nodes surrounding the center node so as to minimize the distance between the first task and the remaining tasks, wherein the remaining tasks communicate with the first task, but not with each other.

2. The method of claim 1, further comprising:

responsive to dynamically allocating one or more of the available nodes for the first job, recalculating the number of available nodes in the virtual cluster; and

executing the first job on the dynamically allocated one or more nodes.

3. The method of claim 2, further comprising:

in the event that either the determined number of available nodes is not enough to execute the first job, or following execution of the first job, selecting a next job in the job queue and repeating the dynamically determining an optimum shape of the next job, determining whether the number of available nodes is enough to execute the next job, based on the optimum shape thereof, and dynamically allocating one or more of the available nodes for the next job, in the event that the determined number of available nodes is enough to execute the next job.

4. The method of claim 3, wherein the job queue is initially sorted based on a job priority.

5. The method of claim 1, wherein the optimum shape is based job dimensions, including one or more of length, width and height.

6. The method of claim 1, wherein the plurality of communicatively coupled nodes included in the computing environment is arranged in a three dimensional node structure.

7. A system comprising:

a plurality of communicatively coupled nodes of a computing environment, each node comprising a processing device and including a cluster agent in communication with a cluster management engine; and

the cluster management engine configured to;

determine available space in a virtual cluster of a plurality of the communicatively coupled nodes, the virtual cluster associated with a group of users that submit similar jobs, and comprising a logical grouping of nodes configured to process related jobs;

prior to job execution, determine an optimum job that is compatible with the available space in the virtual cluster of nodes; and

following the determining, execute the optimum job in the available space in the virtual cluster of nodes

wherein the cluster management engine is further configured to;

determine a number of available nodes in the virtual cluster;

select a first job from a job queue;

dynamically determine an optimum shape of the first job;

determine whether the number of available nodes is enough to execute the first job, based on the optimum shape thereof; and

dynamically allocate one or more of the available nodes for the first job, in the event that the determined number of available nodes is enough to execute the first job;

wherein the optimum shape comprises one or more of;

a best fit cube in which the one or more available nodes are allocated in a cubic volume so as to allow cooperating tasks to exchange data with any other tasks by minimizing the distance between any two nodes; and

a best fit sphere in which the one or more available nodes are allocated in a spherical volume such that a first task is placed in a center node of the sphere with remaining tasks placed on nodes surrounding the center node so as to minimize the distance between the first task and the remaining tasks, wherein the remaining tasks communicate with the first task, but not with each other.

8. The system of claim 7, wherein the cluster management engine is further configured to:

responsive to dynamically allocating one or more of the available nodes for the first job, recalculate the number of available nodes in the virtual cluster; and

execute the first job on the dynamically allocated one or more nodes.

9. The system of claim 8, wherein the cluster management engine is further configured to:

in the event that either the determined number of available nodes is not enough to execute the first job, or following execution of the first job, select a next job in the job queue and repeat the dynamically determining an optimum shape of the next job, determine whether the number of available nodes is enough to execute the next job, based on the optimum shape thereof, and dynamically allocate one or more of the available nodes for the next job, in the event that the determined number of available nodes is enough to execute the next job.

10. The system of claim 9, wherein the job queue is initially sorted based on a job priority.

11. The system of claim 7, wherein the optimum shape is based job dimensions, including one or more of length, width and height.

12. The system of claim 7, wherein the plurality of communicatively coupled nodes included in the computing environment is arranged in a three dimensional node structure.

13. A non-transitory, computer readable storage medium having computer readable instructions stored thereon that, when executed by a computer, implement a method, the method comprising:

determining, by a cluster management engine, available space in a virtual cluster of a plurality of communicatively coupled nodes included in a computing environment, each node including a cluster agent in communication with the cluster management engine, the virtual cluster associated with a group of users that submit similar jobs, and comprising a logical grouping of nodes configured to process related jobs;

prior to job execution, determining an optimum job that is compatible with the available space in the virtual cluster of nodes; and

following the determining, executing the optimum job in the available space in the virtual cluster of nodes;

wherein determining the optimum job that is compatible with the available space in the virtual cluster of nodes further comprises;

determining a number of available nodes in the virtual cluster;

selecting a first job from a job queue;

dynamically determining an optimum shape of the first job;

determining whether the number of available nodes is enough to execute the first job, based on the optimum shape thereof; and

dynamically allocating one or more of the available nodes for the first job, in the event that the determined number of available nodes is enough to execute the first job;

wherein the optimum shape comprises one or more of;

a best fit cube in which the one or more available nodes are allocated in a cubic volume so as to allow cooperating tasks to exchange data with any other tasks by minimizing the distance between any two nodes; and

a best fit sphere in which the one or more available nodes are allocated in a spherical volume such that a first task is placed in a center node of the sphere with remaining tasks placed on nodes surrounding the center node so as to minimize the distance between the first task and the remaining tasks, wherein the remaining tasks communicate with the first task, but not with each other.

14. The non-transitory, computer readable storage medium of claim 13, wherein the method further comprises:

responsive to dynamically allocating one or more of the available nodes for the first job, recalculating the number of available nodes in the virtual cluster; and

executing the first job on the dynamically allocated one or more nodes.

15. The non-transitory, computer readable storage medium of claim 14, wherein the method further comprises:

in the event that either the determined number of available nodes is not enough to execute the first job, or following execution of the first job, selecting a next job in the job queue and repeating the dynamically determining an optimum shape of the next job, determining whether the number of available nodes is enough to execute the next job, based on the optimum shape thereof, and dynamically allocating one or more of the available nodes for the next job, in the event that the determined number of available nodes is enough to execute the next job.

16. The non-transitory, computer readable storage medium of claim 13, wherein the job queue is initially sorted based on a job priority.

17. The non-transitory, computer readable storage medium of claim 13, wherein the optimum shape is based job dimensions, including one or more of length, width and height.

18. The non-transitory, computer readable storage medium of claim 13, wherein the plurality of communicatively coupled nodes included in the computing environment is arranged in a three dimensional node structure.