Monitoring real-time computing resources for predicted resource deficiency

• US 8,875,150 B2
• Filed: 04/20/2012
• Issued: 10/28/2014
• Est. Priority Date: 10/29/2010
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ferdous, Rashed, Meyer, Arthur J III, Sosa, Carlos P
• Primary Examiner(s): Puente, Emerson
• Assistant Examiner(s): WU, BENJAMIN C
• Application Number:
  US13/452,662
• Publication Number:
  US 20120222032A1
• Time in Patent Office: 921 Days
• Field of Search: None
• US Class Current: 718/104
• CPC Class Codes: G06F 11/004 Error avoidance G06F11/07 and subgroups take precedence
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3442 for planning or managing the needed capacity
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 2209/503 Resource availability
• CPC Class Codes: G06F 2209/508 Monitor
• CPC Class Codes: G06F 9/5044 considering hardware capabilities
• CPC Class Codes: G06F 9/505 considering the load

Abstract

Techniques used to enhance the execution of long-running or complex software application instances and jobs on computing systems. In one embodiment, inadequate system resources and failure of a job execution on the computing system may be predicted. A determination may be made as to whether inadequate resources exist prior to execution of the job, and resource requirements may be monitored to detect in real time if inadequate resources will be encountered during the job execution for cases where, for example, resource availability has unexpectedly decreased. If a resource deficiency is predicted on the executing computer system, the job may be paused and corrective action may be taken or a user may be alerted. The job may resume after the resource deficiency is met. Additional embodiments may integrate resource monitoring with the adaptive selection of a computer system or application execution environment based on resource capability predictions and benchmarks.

Claims

1. A method for real time job resource monitoring in a computing system, comprising:

analyzing input data for a job, the job provided to the computing system for a job execution with one or more application instances on the computing system, the job execution consuming resources of the computing system;

estimating total resource usage requirements for the job execution on the computing system, and a total amount of time required to execute the job on the computing system, based on the analysis of the input data, further comprising;

obtaining a series of application-specific performance benchmarks for the computing system;

determining the estimated total resource usage requirements and a resource consumption rate of the job based on the application-specific performance benchmarks; and

using the determined resource consumption rate to estimate the total amount of time required in view of the input data;

monitoring usage of the resources throughout the job execution on the computing system by repeatedly verifying, at a predefined interval that correlates to one or both of the input data and the estimated total resource usage requirements, whether future resource availability is sufficient for the job execution, comprising;

measuring cumulative resource consumption for the job and currently available resource levels in the computing system;

computing an estimated remaining execution time for the job;

computing estimated future resource requirements for the execution of the job by subtracting, from the total resource usage requirements of the job, the cumulative resource consumption for the job;

comparing the future resource usage requirements for the execution of the job to the currently available resource levels in the computing system, in view of the estimated remaining execution time for the job; and

predicting a resource deficiency if the comparing determines that the future resource usage requirements exceed the currently available resource levels in the computing system; and

implementing one or more actions to prevent failure of the job execution on the computing system responsive to prediction of the resource deficiency.

2. The method of claim 1, wherein the one or more actions include pausing the job.

3. The method of claim 1, wherein the one or more actions include alerting a user of the predicted resource deficiency.

4. The method of claim 1, further comprising automatically selecting the computing system from a plurality of computing systems for execution of the job based on the input data and the total resource usage requirements of the job in view of currently available resource levels in the plurality of computing systems at a time when the computing system is automatically selected.

5. The method of claim 4, further comprising factoring in resource usage of currently executing or scheduled jobs on each of the plurality of computing systems and resource usage thresholds of each of the plurality of computing systems when automatically selecting the computing system.

6. The method of claim 1, wherein the resources include one or more of disk space, memory, and CPU cycles.

7. The method of claim 1, wherein the computing system comprises a massively parallel processing computer architecture.

8. The method of claim 1, wherein the computing system comprises a single-processor computer system.

9. The method of claim 1, wherein the job is a batch job and contains a plurality of jobs for execution in a plurality of the application instances.

10. A method for real time job resource monitoring in a computing environment comprising a plurality of computing systems, comprising:

calculating job processing capabilities of each of the plurality of computing systems;

analyzing input data of a job to be executed on the plurality of computing systems;

estimating total resource usage requirements and a total amount of time required to execute the job on each of the plurality of computing systems, based on the analysis of the input data, further comprising;

obtaining a series of application-specific performance benchmarks for each of the plurality of computing systems;

determining a resource consumption rate of the job on each of the plurality of computing systems, based on the respective application-specific performance benchmarks; and

using the determined resource consumption rate to estimate the total amount of time required to execute the job on each of the plurality of computing systems, in view of the input data;

selecting one or more computing systems from the plurality of computing systems to execute the job, based on the estimated total resource usage requirements, the estimated total amount of time required, and available resources on each of the plurality of computing systems in view of any currently executing or scheduled jobs;

providing the job, for execution, to the selected computing systems;

monitoring resource usage of the job execution on the selected computing systems throughout the job execution, comprising repeatedly verifying, at a predefined interval that correlates to one or both of the input data and the estimated total resource usage requirements on the selected computing systems, whether future resource availability is sufficient for continued execution of the job on the selected computing systems, thereby providing a prediction of any resource deficiency; and

responsive to prediction of a resource deficiency, pausing the job execution on the selected computing systems until sufficient resources become available on or until sufficient additional resources are allocated to the selected computing systems to continue the job execution.

11. The method of claim 10, wherein monitoring resource usage of the job execution further comprises:

measuring cumulative resource consumption for the job on each of the selected computing systems and currently available resource levels in the selected computing systems;

estimating future resource usage requirements of the execution of the job on each of the selected computing systems based on the total resource usage requirements of the job on the each selected computing system and the cumulative resource consumption for the job on the each selected computing system;

comparing the estimated future resource usage requirements of the execution of the job on the selected computing systems to the currently available resource levels in the selected computing systems; and

predicting the resource deficiency if the comparing determines that the estimated future resource usage requirements on any of the selected computing systems exceed the currently available resource levels in the any computing system.

12. A computer program product for real-time self-predicting job resource monitoring, the computer program product comprising:

a non-transitory computer readable storage medium having computer-readable program code embodied therein, the computer readable program code configured for;

obtaining, from a user interface, an identification of a job to be executed on a computing system and an identification of input data for use during the execution of the job;

selecting, from a plurality of available computing systems, a particular one of the available computing systems as the computing system on which the job will execute, further comprising;

estimating total resource usage requirements and a total amount of time required to execute the job on each of the plurality of available computing systems, based on an analysis of the identified input data, comprising;

obtaining a series of application-specific performance benchmarks for each of the plurality of available computing systems;

determining a resource consumption rate of the job on each of the plurality of available computing systems, based on the respective application-specific performance benchmarks, and

estimating a total amount of time required to execute the job on each of the plurality of computing systems, in view of the input data and the determined resource consumption rate;

determining resource availability of each of the available computing systems for executing the job;

displaying, on the user interface, an identification of each of the available computing systems for which the determined resource availability meets or exceeds the estimated total resource usage requirements of the job, in view of the estimated total amount of time required to execute the job; and

receiving, from the user interface, an indication of which of the displayed identified available computing systems is to be selected to execute the job;

executing the job on the selected computing system;

periodically calculating remaining resource usage requirements of the job, during the execution of the job and at a predefined interval that correlates to one or both of the input data and the estimated total resource usage requirements on the selected computing system, to determine whether a resource deficiency is predicted to occur prior to completion of the execution of the job;

responsive to predicting the resource deficiency, pausing the execution of the job and notifying the selected computing system to free up resources to correct the resource deficiency for the execution of the job; and

responsive to the correction of the resource deficiency, continuing the execution of the job.

13. The computer program product according to claim 12, wherein:

the computing system is a back-end computing system that is not adapted for directly communicating with web browser software; and

the user interface is a web interface that communicates with the back-end computing system and the web browser software.

14. The computer program product according to claim 12, wherein the periodically calculating remaining resource usage requirements further comprises:

determining particular resources that are no longer needed for the execution; and

removing the particular resources from consideration in the periodic calculation of the remaining resource usage requirements.