PROCESSING PATTERN FRAMEWORK FOR DISPATCHING AND EXECUTING TASKS IN A DISTRIBUTED COMPUTING GRID

• US 20120197959A1
• Filed: 01/26/2012
• Published: 08/02/2012
• Est. Priority Date: 01/28/2011
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Oliver, Brian, Arliss, Noah, Fahlgren, Christer
• Granted Patent: US 9,164,806 B2
• Time in Patent Office: Days
• US Class Current: 709/201
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5038 considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration scheduling strategies G06F9/4881 and subgroups

Abstract

A processing pattern is described for dispatching and executing tasks in a distributed computing grid, such as a cluster network. The grid includes a plurality of computer nodes that store a set of data and perform operations on that data. The grid provides an interface that allows clients to submit tasks to the cluster for processing. The interface can be used to establish a session between the client and the cluster, which will be used to submit a task for processing by the plurality of computer nodes of the cluster. A dispatcher receives a submission of the task over the interface and routes the task to at least one node in the cluster that is designated to process the task. A task processor then processes the task on the designated node(s), generates a submission outcome and indicates to the client that the submission outcome is available.

Claims

1. A system for dispatching and executing tasks in a distributed computing environment, said system comprising:

a cluster including a plurality of computer nodes that store a set of data and perform operations on said data, said cluster providing an interface to submit tasks to the cluster;

a session established between a client and the cluster using said interface, said session used by said client to submit a task for processing by the plurality of computer nodes of the cluster;

a dispatcher that receives a submission of the task using said interface and routes said task to at least one node in the cluster that is designated to process said task; and

a task processor that processes the task in said node, generates a submission outcome and indicates to said client that the submission outcome is available.

2. The system of claim 1, wherein the session includes an identifier for reconnecting to said session, wherein the client submits the task to the session, disconnects from said session for at least a period of time and reconnects to the session to retrieve the submission outcome.

3. The system of claim 1, wherein the task stores intermediate state, wherein the intermediate state is used to suspend the task and to resume said task.

4. The system of claim 1, further comprising:

a chain of dispatchers for processing said task, wherein said dispatchers are registered in a registration order to process said task; and

a dispatch controller that dispatches the submission of said task to the chain of registered dispatchers in said registration order.

5. The system of claim 4, wherein the chain of dispatchers further includes:

a logging dispatcher that logs all submissions of the tasks in a log file;

a Java dispatcher that executes Java Runnables interface and Java Callables interface; and

a default task dispatcher that dispatches the tasks to task processor implementations.

6. The system of claim 1, further comprising:

a submissions cache distributed on said cluster that stores the submission before said submission is provided to the dispatcher.

7. The system of claim 1, further comprising:

a submissions result cache distributed on said cluster that stores the submission outcome before the submission outcome is provided to the client.

8. The system of claim 1, wherein the dispatcher dispatches said tasks to task processors using at least one of a set of dispatch policies that includes a round robin dispatch policy, a random dispatch policy and an attribute matching dispatch policy.

9. The system of claim 1, wherein the task is associated with a set of metadata that is used by the dispatcher to dispatch the task to the task processor.

10. The system of claim 1, further comprising:

a plurality of task processors that are configured in a cache configuration file.

11. A method for dispatching and executing tasks in a distributed computing environment, said method comprising:

establishing a session between a client and a cluster of computer nodes, said session used by said client to submit a task for processing by the computer nodes of the cluster;

receiving a submission of said task from the client;

dispatching said task to at least one node in the cluster that is designated to process said task;

processing the task on said node and generating a submission outcome for said task, wherein the client is capable of disconnecting from the session and reconnecting to the session during the processing of said task; and

indicating to the client that the submission outcome is available.

12. The method of claim 11, wherein the session includes an identifier used for reconnecting to said session after a connection between the client and the session has been terminated.

13. The method of claim 11, further comprising:

storing intermediate state for said task, wherein the intermediate state is used to suspend the task and to resume said task.

14. The method of claim 11, further comprising:

registering a chain of dispatchers in a particular registration order for processing said task; and

providing the submission of said task to a dispatch controller that dispatches the submission of said task to the chain of registered dispatchers in said registration order.

15. The method of claim 14, wherein the chain of dispatchers further includes:

a logging dispatcher that logs all submissions of the tasks in a log file;

a Java dispatcher that executes Java Runnables interface and Java Callables interface; and

a default task dispatcher that dispatches the tasks to task processor implementations.

16. The method of claim 11, further comprising:

storing the submission in a submissions cache distributed on said cluster before said submission is provided to the dispatcher.

17. The method of claim 11, further comprising:

storing the submission outcome in a submissions result cache distributed on said cluster before the submission outcome is provided to the client.

18. The method of claim 11, wherein said tasks are dispatched to a set of task processors using at least one of a set of dispatch policies that includes a round robin dispatch policy, a random dispatch policy and an attribute matching dispatch policy.

19. The method of claim 11, wherein the task is associated with a set of metadata that is used to dispatch the task to a task processor.

20. A non-transitory computer readable storage medium storing a set of instructions executed by one or more processors to perform a sequence of steps comprising:

establishing a session between a client and a cluster of computer nodes, said session used by said client to submit a task for processing by the computer nodes of the cluster;

receiving a submission of said task from the client;

dispatching said task to at least one node in the cluster that is designated to process said task; and

processing the task on said node and generating a submission outcome for said task, wherein the client is capable of disconnecting from the session and reconnecting to the session during the processing of said task; and

indicating to the client that the submission outcome is available.