Dynamic job processing based on estimated completion time and specified tolerance time

• US 9,122,529 B2
• Filed: 08/07/2013
• Issued: 09/01/2015
• Est. Priority Date: 07/20/2001
• Status: Expired due to Term

• Current Assignee: Callidus Software, Inc. (SAP SE)
• Original Assignee: Searchlite Advances LLC (Innovation Management Sciences, LLC)
• Inventors: Tymes, Adrian, Prael, Charles
• Primary Examiner(s): Lee, Tammy
• Application Number:
  US13/961,881
• Publication Number:
  US 20140040911A1
• Time in Patent Office: 755 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 2209/5014 Reservation
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5038 considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration scheduling strategies G06F9/4881 and subgroups
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455

Abstract

The invention provides a system and method for managing clusters of parallel processors for use by groups and individuals requiring supercomputer level computational power. A Beowulf cluster provides supercomputer level processing power. Unlike a traditional Beowulf cluster; however, cluster size in not singular or static. As jobs are received from users/customers, a Resource Management System (RMS) dynamically configures and reconfigures the available nodes in the system into clusters of the appropriate sizes to process the jobs. Depending on the overall size of the system, many users may have simultaneous access to supercomputer level computational processing. Users are preferably billed based on the time for completion with faster times demanding higher fees.

Claims

1. A computer-implemented method, comprising:

receiving a first job to be processed by a first dynamic cluster of processing resources, the first job having a predetermined completion time requirement;

receiving processing information from a pool of processing resources, said processing information including at least completion times for each processing resource that is currently executing a job;

determining the first dynamic cluster of processing resources from the pool of processing resources to execute the first job in the future in response to the predetermined completion time requirement and the completion times of the processing resources that are currently executing a job;

receiving a second job to be processed by a second dynamic cluster of processing resources, the second job requiring different overall processing resources relative to the first job;

determining the second dynamic cluster of processing resources to complete execution of the second job, the second dynamic cluster of processing resources differing in composition from the first dynamic cluster of processing resources while at least one of the processing resources of the second dynamic cluster of processing resources is also reserved for the first job in the first dynamic cluster of processing resources; and

enqueueing the first job to complete execution prior to starting execution of the second job, the second job completing execution prior to a predetermined completion time requirement for the second job.

2. The method of claim 1 further comprising:

configuring the first and second dynamic cluster of processing resources by transmitting a configuration file to each processing resource, wherein each processing resource dynamically configures into a cluster.

3. The method of claim 1 wherein said enqueueing includes:

maintaining one or more queues of jobs to be processed;

maintaining processing times for each job in said queue of jobs;

adding jobs to the queue of jobs in response to said processing time and a completion time requirement for each added job.

4. The method of claim 3 further including:

reordering at least one of queues of jobs wherein jobs are enqueued to meet the completion time requirement.

5. The method of claim 1 wherein said enqueueing includes:

maintaining a plurality of queues of jobs to be processed wherein each queue operates at a different job priority.

6. The method of claim 1 further including:

receiving a tolerance time for each job to be processed;

wherein said enqueueing is in response to said tolerance time.

7. The method of claim 1, further comprising:

adjusting a duration of the predetermined completion time requirement and the first dynamic cluster of processing resources in accordance with a set of fees.

8. The method of claim 1, wherein at least one of the first and second job comprises a graphics rendering job.

9. One or more non-transitory processor readable storage devices having non-volatile and non-transitory processor readable code embodied on said storage devices, said processor readable code for programming one or more processors to perform a method comprising:

receiving a first job to be processed by a first dynamic cluster of processing resources, the first job having a predetermined completion time requirement;

receiving processing information from a pool of processing resources, said processing information including at least completion times for each processing resource that is currently executing a job;

determining the first dynamic cluster of processing resources from the pool of processing resources to execute the first job in the future in response to the predetermined completion time requirement and the completion times of the processing resources that are currently executing a job;

receiving a second job to be processed by a second dynamic cluster of processing resources, the second job requiring different overall processing resources relative to the first job;

determining the second dynamic cluster of processing resources to complete execution of the second job, the second dynamic cluster of processing resources differing in composition from the first dynamic cluster of processing resources while at least one of the processing resources of the second dynamic cluster of processing resources is also reserved for the first job in the first dynamic cluster of processing resources; and

enqueueing the first job to complete execution prior to starting execution of the second job, the second job completing execution prior to a predetermined completion time requirement for the second job.

10. The device of claim 9 wherein the method further comprises:

configuring the first and second dynamic cluster of processing resources by transmitting a configuration file to each processing resource, wherein each processing resource dynamically configures into a cluster.

11. The device of claim 9 wherein said enqueueing includes:

maintaining one or more queues of jobs to be processed;

maintaining processing times for each job in said queue of jobs;

adding jobs to the queue of jobs in response to said processing time and a completion time requirement for each added job.

12. The device of claim 11 wherein said method further includes:

reordering at least one of said queues of jobs wherein jobs are enqueued to meet the completion time requirement.

13. The device of claim 9 wherein said enqueueing includes:

maintaining a plurality of queues of jobs to be processed wherein each queue operates at a different job priority.

14. The device of claim 9 wherein the method further includes:

receiving a tolerance time for each job to be processed;

wherein said enqueueing is in response to said tolerance time.

15. The method of claim 9, further comprising:

adjusting a duration of the predetermined completion time requirement and the first dynamic cluster of processing resources in accordance with a set of fees.

16. The method of claim 9, wherein at least one of the first and second job comprises a graphics rendering job.

17. A hardware device, with a resource management system, the hardware device comprising:

a processor; and

a memory, storing;

a queue to receive a first job to be processed by a first dynamic cluster of processing resources, the first job having a predetermined completion time requirement;

a resource manager to receive processing information from a pool of processing resources, said processing information including at least completion times for each processing resource that is currently executing a job, the resource manager to determine the first dynamic cluster of processing resources from the pool of processing resources to execute the first job in the future in response to the predetermined completion time requirement and the completion times of the processing resources that are currently executing a job,

wherein the queue receives a second job to be processed by a second dynamic cluster of processing resources, the second job requiring different overall processing resources relative to the first job,

wherein the resource manager determines the second dynamic cluster of processing resources to complete execution of the second job, the second dynamic cluster of processing resources differing in composition from the first dynamic cluster of processing resources while at least one of the processing resources of the second dynamic cluster of processing resources is also reserved for the first job in the first dynamic cluster of processing resources; and

a resource scheduler to queue the first job to complete execution prior to starting execution of the second job, the second job completing execution prior to a predetermined completion time requirement for the second job.