EVENT-BASED DYNAMIC RESOURCE PROVISIONING

• US 20100269119A1
• Filed: 04/16/2009
• Published: 10/21/2010
• Est. Priority Date: 04/16/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chaudhary, Piyush, Arimilli, Ravi Kumar
• Granted Patent: US 8,977,752 B2
• Time in Patent Office: Days
• US Class Current: 719/313
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources

Abstract

Disclosed are a method, a system and a computer program product for automatically allocating and de-allocating resources for jobs executed or processed by one or more supercomputer systems. In one or more embodiments, a supercomputing system can process multiple jobs with respective supercomputing resources. A global resource manager can automatically allocate additional resources to a first job and de-allocate resources from a second job. In one or more embodiments, the global resource manager can provide the de-allocated resources to the first job as additional supercomputing resources. In one or more embodiments, the first job can use the additional supercomputing resources to perform data analysis at a higher resolution, and the additional resources can compensate for an amount of time the higher resolution analysis would take using originally allocated supercomputing resources.

Claims

1. A computer-implemented method for operating a supercomputing system, comprising:

processing a first supercomputing job with a first amount of resources of the supercomputing system;

determining that a first event occurred while processing a data set of the first supercomputing job;

in response to determining that the first event occurred;

determining a first amount of additional resources of the supercomputing system based on a first resolution, a second resolution, a size of the data set, and a target completion time for the first supercomputing job;

allocating the first amount of additional resources of the supercomputing system;

distributing at least a portion of the data set to the first additional computing resources; and

processing the first supercomputing job with the first amount of resources of the supercomputing system and the first amount of additional resources of the supercomputing system.

2. The computer-implemented method of claim 1, further comprising:

determining the first event from analysis of the data set; and

transmitting, to a resource manager, a message that indicates that the first event occurred.

3. The computer-implemented method of claim 1, further comprising:

processing a second supercomputing job with a second amount of resources of the supercomputing system; and

de-allocating a portion of the second amount of resources of the supercomputing system of the second supercomputing job.

4. The computer-implemented method of claim 3, further comprising:

processing the second supercomputing job with the second amount of resources of the supercomputing system less the de-allocated portion;

wherein said allocating the first amount of additional resources of the supercomputing system includes allocating the de-allocated portion of the second amount of resources of the supercomputing system.

5. The computer-implemented method of claim 4, further comprising:

de-allocating the first amount of additional resources of the supercomputing system;

processing the first supercomputing job with the first amount of resources of the supercomputing system;

allocating the de-allocated portion of the second amount of resources of the supercomputing system for the second supercomputing job; and

processing the second supercomputing job with the second amount of resources of the supercomputing system.

6. The computer-implemented method of claim 1, wherein the supercomputing system is coupled to a second supercomputing system through a network;

the method further comprising;

allocating a first amount of additional resources of the second supercomputing system; and

processing the first supercomputing job with the first amount of resources of the supercomputing system, the first amount of additional resources of the supercomputing system, and the first amount of additional resources of the second supercomputing system.

7. The computer-implemented method of claim 1, further comprising:

during said processing the first supercomputing job with the first amount of resources of the supercomputing system and the first amount of additional resources of the supercomputing system, determining that the first supercomputing job is processing anomalous data; and

de-allocating the first amount of additional resources of the supercomputing system; and

processing the first supercomputing job with the first amount of resources of the supercomputing system.

8. A supercomputing system, comprising:

a plurality of compute nodes, wherein each of the plurality of compute nodes is coupled to another of the plurality of compute nodes;

a first memory medium coupled to at least a first compute node of the plurality of compute nodes, wherein the first memory medium includes instructions that when executed on the first compute node provides logic for performing the functions of

processing a first supercomputing job with a first amount of resources of the supercomputing system;

determining that a first event occurred while processing a data set of the first supercomputing job;

in response to determining that the first event occurred;

determining a first amount of additional resources of the supercomputing system based on a first resolution, a second resolution, a size of the data set, and a target completion time for the first supercomputing job;

allocating the first amount of additional resources of the supercomputing system;

distributing at least a portion of the data set to the first additional computing resources; and

processing the first supercomputing job with the first amount of resources of the supercomputing system and the first amount of additional resources of the supercomputing system.

9. The supercomputing system of claim 8, further comprising:

a second memory medium coupled to at least a second compute node of the plurality of compute nodes, wherein the second memory medium includes instructions that when executed on the second compute node provides logic for performing the functions of;

determining the first event from analysis of the data set; and

transmitting, to the first compute node, a message that indicates that the first event occurred.

10. The supercomputing system of claim 8, wherein the first memory medium further includes instructions that when executed on the first compute node provides logic for performing the functions of:

processing a second supercomputing job with a second amount of resources of the supercomputing system; and

de-allocating a portion of the second amount of resources of the supercomputing system of the second supercomputing job.

11. The supercomputing system of claim 10, wherein the first memory medium further includes instructions that when executed on the first compute node provides logic for performing the function of:

processing the second supercomputing job with the second amount of resources of the supercomputing system less the de-allocated portion;

wherein said allocating the first amount of additional resources of the supercomputing system includes allocating the de-allocated portion of the second amount of resources of the supercomputing system.

12. The supercomputing system of claim 11, wherein the first memory medium further includes instructions that when executed on the first compute node provides logic for performing the functions of:

de-allocating the first amount of additional resources of the supercomputing system;

processing the first supercomputing job with the first amount of resources of the supercomputing system;

allocating the de-allocated portion of the second amount of resources of the supercomputing system for the second supercomputing job; and

processing the second supercomputing job with the second amount of resources of the supercomputing system.

13. The supercomputing system of claim 8, wherein the supercomputing system is coupled to a second supercomputing system through a network;

wherein the first memory medium further includes instructions that when executed on the first compute node provides logic for performing the functions of;

allocating a first amount of additional resources of the second supercomputing system; and

processing the first supercomputing job with the first amount of resources of the supercomputing system, the first amount of additional resources of the supercomputing system, and the first amount of additional resources of the second supercomputing system.

14. The supercomputing system of claim 8, wherein the first memory medium further includes instructions that when executed on the first compute node provides logic for performing the functions of:

during said processing the first supercomputing job with the first amount of resources of the supercomputing system and the first amount of additional resources of the supercomputing system, determining that the first supercomputing job is processing anomalous data; and

de-allocating the first amount of additional resources of the supercomputing system; and

processing the first supercomputing job with the first amount of resources of the supercomputing system.

15. A computer readable memory medium comprising instructions, which when executed on a processing system of a supercomputing system, cause the supercomputing system to perform:

processing a first supercomputing job with a first amount of resources of the supercomputing system;

determining that a first event occurred while processing a data set of the first supercomputing job;

in response to determining that the first event occurred;

determining a first amount of additional resources of the supercomputing system based on a first resolution, a second resolution, a size of the data set, and a target completion time for the first supercomputing job;

allocating the first amount of additional resources of the supercomputing system;

distributing at least a portion of the data set to the first additional computing resources; and

processing the first supercomputing job with the first amount of resources of the supercomputing system and the first amount of additional resources of the supercomputing system.

16. The computer readable memory medium of claim 15, wherein the instructions, when executed on the processing system of the supercomputing system, cause the supercomputing system to perform:

processing a second supercomputing job with a second amount of resources of the supercomputing system; and

de-allocating a portion of the second amount of resources of the supercomputing system of the second supercomputing job.

17. The computer readable memory medium of claim 15, wherein the instructions, when executed on the processing system of the supercomputing system, cause the supercomputing system to perform:

processing the second supercomputing job with the second amount of resources of the supercomputing system less the de-allocated portion;

wherein said allocating the first amount of additional resources of the supercomputing system includes allocating the de-allocated portion of the second amount of resources of the supercomputing system.

18. The computer readable memory medium of claim 17, wherein the instructions, when executed on the processing system of the supercomputing system, cause the supercomputing system to perform:

de-allocating the first amount of additional resources of the supercomputing system;

processing the first supercomputing job with the first amount of resources of the supercomputing system;

allocating the de-allocated portion of the second amount of resources of the supercomputing system for the second supercomputing job; and

processing the second supercomputing job with the second amount of resources of the supercomputing system.

19. The computer readable memory medium of claim 15, wherein the instructions, when executed on the processing system of the supercomputing system, cause the supercomputing system to perform:

allocating a first amount of additional resources of a second supercomputing system coupled to the supercomputing system via a network; and

processing the first supercomputing job with the first amount of resources of the supercomputing system, the first amount of additional resources of the supercomputing system, and the first amount of additional resources of the second supercomputing system.

20. The computer readable memory medium of claim 15, wherein the instructions, when executed on the processing system of the supercomputing system, cause the supercomputing system to perform:

during said processing the first supercomputing job with the first amount of resources of the supercomputing system and the first amount of additional resources of the supercomputing system, determining that the first supercomputing job is processing anomalous data; and

de-allocating the first amount of additional resources of the supercomputing system; and

processing the first supercomputing job with the first amount of resources of the supercomputing system.