Mechanism for managing parallel execution of processes in a distributed computing environment

• US 7,159,217 B2
• Filed: 09/19/2002
• Issued: 01/02/2007
• Est. Priority Date: 12/20/2001
• Status: Expired due to Fees

• Current Assignee: Cadence Design Systems Incorporated (Cadence Group)
• Original Assignee: Cadence Design Systems Incorporated (Cadence Group)
• Inventors: Pulsipher, Darren, Hannaford, Nancy
• Primary Examiner(s): An, Meng-Al T.
• Assistant Examiner(s): Tang, Kenneth
• Application Number:
  US10/247,970
• Publication Number:
  US 20030120708A1
• Time in Patent Office: 1,566 Days
• Field of Search: 718/100, 718/102, 718/104, 718106-108, 709/226
• US Class Current: 718/100
• CPC Class Codes: G06F 2209/484 Precedence
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues

Abstract

According to one aspect, a work request that specifies first and second jobs is received. The first job comprises a first task and the second job comprises a second task. The work request is processed to automatically determine whether the jobs have any dependencies that have not been satisfied. In response to a determination that the jobs have no dependencies that have not been satisfied, the jobs are caused to be executed in parallel. As a default manner of operation, the tasks included in each respective job are collectively executed in parallel, whereas tasks within a given job are not executed in parallel. In an embodiment, the tasks are executed on one or more servers of a group of networked servers.

Claims

1. A computer implemented method for managing execution of work in a networked multi-processor computing environment in which work is distributed between multiple processors, the method comprising:

receiving a first file that represents a work request, said first file specifying a first job and a second job for execution in a networked multi-processor computing environment in which work is distributed among multiple processors, said first job comprising a first task and said second job comprising a second task;

wherein said first task has a parent-child relationship with said first job such that said first task is a child of said first job, and wherein said second task has a parent-child relationship with said second job such that said second task is a child of said second job;

wherein said first and second jobs comprise respective aggregations of executable commands and wherein said first and second tasks each comprise at least one of said executable commands;

wherein dependencies among jobs are specified in a corresponding work request by hierarchically relating, in a file that represents the corresponding work request, interdependent jobs, and wherein dependencies among jobs specified in the corresponding work request contribute to a determination of an order in which commands are executed in the networked multi-processor computing environment;

interpreting said first file to automatically determine, based at least in part on whether said first and second jobs are hierarchically related in the first file, whether said first and second jobs have any dependencies that have not been satisfied;

in response to a determination that said first and second jobs have no dependencies that have not been satisfied, causing said first task of said first job and said second task of said second job to be executed in parallel based on a default work execution order which is to execute jobs in parallel unless otherwise specified in said file that represents said work request; and

wherein said work request does not explicitly specify that said first and second tasks be executed in parallel.

2. The method of claim 1, wherein causing includes:

placing both said first and second jobs into an active state;

after said first job is placed into an active state, causing said first task to be executed; and

after said second job is placed into an active state, causing said second task to be executed.

3. The method of claim 1, wherein said first job further comprises a third task, and wherein said method further comprises:

after execution of said first task has finished, causing said third task to be executed.

4. The method of claim 3, wherein said first and third tasks have a parent-child relationship with said first job such that said first task is a child of said first job and said third task is also a child of said first job.

5. The method of claim 1, wherein said networked multi-processor computing environment comprises a server farm, and wherein causing includes:

providing said first and second tasks to a distributed resource management application to enable said resource management application to distribute said first and second tasks to one or more servers in said server farm to execute said first and second tasks in parallel.

6. The method of claim 1, wherein said networked multi-processor computing environment comprises a server farm, and wherein causing includes:

distributing said first and second tasks to one or more servers in said server farm to execute said first and second tasks in parallel.

7. The method of claim 1,

wherein said first job further comprises a first sub-job that includes a fourth task;

wherein said first file specifies, via a hierarchical relationship, that said first sub-job has a parent-child relationship with said first job such that said first sub-job is a child of said first job;

wherein said first file specifies, via a hierarchical relationship, that said fourth task has a parent-child relationship with said first sub-job such that said fourth task is a child of said first sub-job;

and said method further comprises;

based at least in part on how said first task and said fourth task are hierarchically related in the first file, causing said first task and said fourth task to be executed in parallel.

8. The method of claim 7, wherein said first job further comprises a second sub-job that includes a fifth task;

wherein said first file specifies, via a hierarchical relationship, that said second sub-job has a parent-child relationship with said first job such that said second sub-job is a child of said first job;

wherein said first file specifies, via a hierarchical relationship, that said fifth task has a parent-child relationship with said second sub-job such that said fifth task is a child of said second sub-job;

and said method further comprises;

based at least in part on how said first task, said fourth task, and said fifth task are hierarchically related in the first file, causing said first task, said fourth task, and said fifth task to be executed in parallel.

9. The method of claim 1, wherein processing includes processing a sequential set of two or more tasks, and said method further comprises:

causing said sequential set of tasks to be executed as a job, wherein said sequential set of tasks are provided to a resource manager to enable said resource manager to distribute for execution, with a single submission to a group of servers, said two or more tasks to one or more servers in said group of servers.

10. A computer-readable storage medium carrying one or more sequences of instructions for managing execution of work in a networked multi-processor computing environment in which work is distributed between multiple processors, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform:

receiving a first file that represents a work request, said first file specifying a first job and a second job for execution in a networked multi-processor computing environment in which work is distributed among multiple processors, said first job comprising a first task and said second job comprising a second task;

wherein said first task has a parent-child relationship with said first job such that said first task is a child of said first job, and wherein said second task has a parent-child relationship with said second job such that said second task is a child of said second job;

wherein said first and second jobs comprise respective aggregations of executable commands and wherein said first and second tasks each comprise at least one of said executable commands;

wherein dependencies among jobs are specified in a corresponding work request by hierarchically relating, in a file that represents the corresponding work request, interdependent jobs, and wherein dependencies among jobs specified in the corresponding work request contribute to a determination of an order in which commands are executed in the networked multi-processor computing environment;

interpreting said first file to automatically determine, based at least in part on whether said first and second jobs are hierarchically related in the first file, whether said first and second jobs have any dependencies that have not been satisfied;

in response to a determination that said first and second jobs have no dependencies that have not been satisfied, causing said first task of said first job and said second task of said second job to be executed in parallel based on a default work execution order which is to execute jobs in parallel unless otherwise specified in said file that represents said work request; and

wherein said work request does not explicitly specify that said first and second tasks be executed in parallel.

11. The computer-readable storage medium of claim 10, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the causing by performing:

placing both said first and second jobs into an active state;

after said first job is placed into an active state, causing said first task to be executed; and

after said second job is placed into an active state, causing said second task to be executed.

12. The computer-readable storage medium of claim 10, wherein said first job further comprises a third task, and wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform:

after execution of said first task has finished, causing said third task to be executed.

13. The computer-readable storage medium of claim 12, wherein said first and third tasks have a parent-child relationship with said first job such that said first task is a child of said first job and said third task is also a child of said first job.

14. The computer-readable storage medium of claim 10, wherein said networked multi-processor computing environment comprises a server farm, and wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the causing by performing:

providing said first and second tasks to a distributed resource management application to enable said resource management application to distribute said first and second tasks to one or more servers in said server farm to execute said first and second tasks in parallel.

15. The computer-readable storage medium of claim 10, wherein said networked multi-processor computing environment comprises a server farm, and wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the causing by performing:

distributing said first and second tasks to one or more servers in said server farm to execute said first and second tasks in parallel.

16. The computer-readable storage medium of claim 10,

wherein said first job further comprises a first sub-job that includes a fourth task;

wherein said first file specifies, via a hierarchical relationship, that said first sub-job has a parent-child relationship with said first job such that said first sub-job is a child of said first job;

wherein said first file specifies, via a hierarchical relationship, that said fourth task has a parent-child relationship with said first sub-job such that said fourth task is a child of said first sub-job; and

wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform;

based at least in part on how said first task and said fourth task are hierarchically related in the first file, causing said first task and said fourth task to be executed in parallel.

17. The computer-readable storage medium of claim 16, wherein said first job further comprises a second sub-job that includes a fifth task;

wherein said first file specifies, via a hierarchical relationship, that said second sub-job has a parent-child relationship with said first job such that said second sub-job is a child of said first job;

wherein said first file specifies, via a hierarchical relationship, that said fifth task has a parent-child relationship with said second sub-job such that said fifth task is a child of said second sub-job; and

wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform;

based at least in part on how said first task, said fourth task, and said fifth task are hierarchically related in the first file, causing said first task, said fourth task, and said fifth task to be executed in parallel.

18. The computer-readable storage medium of claim 10, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the processing by processing a sequential set of two or more tasks, and

wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform:

causing said sequential set of tasks to be executed as a job, wherein said sequential set of tasks are provided to a resource manager to enable said resource manager to distribute for execution, with a single submission to a group of servers, said two or more tasks to one or more servers in said group of servers.

19. A network device that can manage execution of work, the network device comprising:

a network interface;

a processor coupled to the network interface and receiving network messages from the network through the network interface;

a computer-readable storage medium comprising one or more stored sequences of instructions which, when executed by the processor, cause performance of;

receiving a first file that represents a work request, said first file specifying a first job and a second job for execution in a networked multi-processor computing environment in which work is distributed among multiple processors, said first job comprising a first task and said second job comprising a second task;

wherein said first task has a parent-child relationship with said first job such that said first task is a child of said first job, and wherein said second task has a parent-child relationship with said second job such that said second task is a child of said second job;

wherein said first and second jobs comprise respective aggregations of executable commands and wherein said first and second tasks each comprise at least one of said executable commands;

wherein dependencies among jobs are specified in a corresponding work request by hierarchically relating, in a file that represents the corresponding work request, interdependent jobs, and wherein dependencies among jobs specified in the corresponding work request contribute to a determination of an order in which commands are executed in the networked multi-processor computing environment;

interpreting said first file to automatically determine, based at least in part on whether said first and second jobs are hierarchically related in the first file, whether said first and second jobs have any dependencies that have not been satisfied;

in response to a determination that said first and second jobs have no dependencies that have not been satisfied, causing said first task of said first job and said second task of said second job to be executed in parallel based on a default work execution order which is to execute jobs in parallel unless otherwise specified in said file that represents said work request; and

wherein said work request does not explicitly specify that said first and second tasks be executed in parallel.