RUNTIME TASK WITH INHERITED DEPENDENCIES FOR BATCH PROCESSING

• US 20100162245A1
• Filed: 12/19/2008
• Published: 06/24/2010
• Est. Priority Date: 12/19/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Tao, Kurup, Vijay B., Plancarte, Gustavo A.
• Granted Patent: US 8,990,820 B2
• Time in Patent Office: Days
• US Class Current: 718/101
• CPC Class Codes: G06F 2209/5017 Task decomposition
• CPC Class Codes: G06F 9/4843 by program, e.g. task dispatcher, supervisor, operating system

Abstract

A batch job processing architecture that dynamically creates runtime tasks for batch job execution and to optimize parallelism. The task creation can be based on the amount of processing power available locally or across batch servers. The work can be allocated across multiple threads in multiple batch server instances as there are available. A master task splits the items to be processed into smaller parts and creates a runtime task for each. The batch server picks up and executes as many runtime tasks as the server is configured to handle. The runtime tasks can be run in parallel to maximize hardware utilization. Scalability is provided by splitting runtime task execution across available batch server instances, and also across machines. During runtime task creation, all dependency and batch group information is propagated from the master task to all runtime tasks. Dependencies and batch group configuration are honored by the batch engine.

Claims

1. A computer-implemented batch processing system, comprising:

a partitioning component for splitting a task of a batch job into runtime tasks based on associated items to be processed; and

an execution component for processing the runtime tasks concurrently using threads of available batch server instances.

2. The system of claim 1, further comprising a dependency component for propagating dependency information and batch group information of a master task to the runtime tasks.

3. The system of claim 2, wherein the partitioning component, execution component, and dependency component are utilized as part of an enterprise resource planning (ERP) framework.

4. The system of claim 1, wherein the execution component persists runtime task status information and log information in one or more history tables.

5. The system of claim 1, wherein the partitioning component recreates the runtime tasks in response to the task being processed again, the runtime tasks recreated based on an amount of work to be performed at a specific time.

6. The system of claim 1, wherein the threads are associated with different server machines and the runtime tasks are distributed for concurrent processing on the different server machines.

7. The system of claim 1, wherein the threads are applied on a per instance basis.

8. The system of claim 1, wherein the execution component deletes the runtime tasks and associated dependencies upon completion of the task.

9. A computer-implemented batch processing system, comprising:

a partitioning component for splitting a task of a batch job into one or more runtime tasks based on associated items to be processed at a specific time;

an execution component for processing the one or more runtime tasks concurrently using threads of available batch server instances; and

a dependency component for propagating dependency information and batch group information of the batch job to the one or more runtime tasks.

10. The system of claim 9, wherein the partitioning component, execution component, and dependency component are utilized as part of an enterprise resource planning (ERP) framework.

11. The system of claim 9, wherein the execution component persists runtime task status information and log information in one or more history tables, and deletes the runtime tasks and associated dependencies upon completion of the task.

12. The system of claim 9, wherein the threads are associated with different server machines, the runtime tasks are distributed for concurrent processing on the different server machines and the threads are applied on a per instance basis on the server machines.

13. A computer-implemented method of processing tasks, comprising:

receiving a batch job of tasks, each task associated with one or more items to be processed;

selecting a current task having multiple items;

creating a runtime task for each item of the current task;

executing the runtime tasks;

deleting the runtime tasks after completing the current task; and

completing the batch job when all tasks are finished.

14. The method of claim 13, further comprising defining new dependencies for the runtime tasks based on dependencies of the current task.

15. The method of claim 14, further comprising deleting the new dependencies after completing the current task.

16. The method of claim 13, further comprising persisting status information and log information associated with the runtime tasks in one or more history tables.

17. The method of claim 13, further comprising assigning the runtime tasks to different processor cores for concurrent task execution.

18. The method of claim 13, further comprising recreating the runtime tasks based on the batch job being run again.

19. The method of claim 13, further comprising creating a defined number of the runtime tasks based on an amount of work that is to be performed for the current task.

20. The method of claim 13, further comprising executing the runtime tasks based on server behavior defined by server configuration data.