Processing pattern framework for dispatching and executing tasks in a distributed computing grid

• US 9,164,806 B2
• Filed: 01/26/2012
• Issued: 10/20/2015
• Est. Priority Date: 01/28/2011
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Oliver, Brian, Fahlgren, Christer, Arliss, Noah
• Primary Examiner(s): Baturay, Alicia
• Application Number:
  US13/359,391
• Publication Number:
  US 20120197959A1
• Time in Patent Office: 1,363 Days
• Field of Search: 709/227
• US Class Current: 1/1
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5038 considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration scheduling strategies G06F9/4881 and subgroups

Abstract

A processing pattern is described for dispatching and executing tasks in a distributed computing grid, such as a cluster network. The grid includes a plurality of computer nodes that store a set of data and perform operations on that data. The grid provides an interface that allows clients to submit tasks to the cluster for processing. The interface can be used to establish a session between the client and the cluster, which will be used to submit a task for processing by the plurality of computer nodes of the cluster. A dispatcher receives a submission of the task over the interface and routes the task to at least one node in the cluster that is designated to process the task. A task processor then processes the task on the designated node(s), generates a submission outcome and indicates to the client that the submission outcome is available.

Claims

1. A system for dispatching and executing tasks in a distributed computing environment, said system comprising:

a computer including memory and one or more microprocessors;

a cluster including a plurality of computer nodes that store a set of data and perform operations on said data, said cluster providing an application programming interface (API) used to submit tasks to the cluster;

a session established between a client and the cluster using said API, said session adapted to be used by said client to submit a task for processing by the plurality of computer nodes of the cluster;

a dispatch controller configured to dispatch a submission of a task to a plurality of dispatchers registered in a particular order, wherein the dispatchers include a logging dispatcher for logging the task submission, an executor dispatcher for executing one or more interfaces to process the task, and a default task dispatcher for routing said task to a node designated to process said task in the cluster; and

a task processor, on the node, that is selected for processing the routed task by matching an attribute supplied with the task processor and an attribute supplied with the task;

wherein after the task is processed, an indication is made to said client that a submission outcome is available.

2. The system of claim 1, wherein the session includes an identifier used for reconnecting to the session after a connection between the client and the session has been terminated.

3. The system of claim 1, wherein the task stores an intermediate state used to suspend and resume the task.

4. The system of claim 1, wherein the logging dispatcher logs the task submission in a log file;

and wherein the executor dispatcher executes a Java Runnable interface and a Java Callable interface.

5. The system of claim 1, further comprising:

a submissions cache distributed on said cluster, wherein the submissions cache stores the task submission before said task submission is provided to the dispatch controller.

6. The system of claim 1, further comprising:

a submissions result cache distributed on said cluster, wherein the submissions result cache stores the submission outcome before the submission outcome is provided to the client.

7. The system of claim 1, wherein the default task dispatcher is configured to dispatch said task to the task processor using at least one of a set of dispatch policies that includes a round robin dispatch policy and a random dispatch policy.

8. The system of claim 1, wherein the task is associated with a set of metadata that is used by the default task dispatcher to dispatch the task to the task processor.

9. The system of claim 1, further comprising:

a plurality of task processors that are configured in a cache configuration file.

19. The system of claim 1, wherein the plurality of dispatchers are consulted in the particular order to determine whether the task submission should be processed or routed.

10. A method for dispatching and executing tasks in a distributed computing environment, said method comprising:

establishing a session between a client and a cluster of computer nodes using an application programming interface (API) provided by the cluster, wherein the API and said session are adapted to be used by said client to submit a task for processing by the computer nodes of the cluster;

receiving a submission of said task from the client;

dispatching, via a dispatcher controller, the submission of said task to a plurality of dispatchers registered in a particular order, wherein the dispatchers include a logging dispatcher for logging the task submission, an executor dispatcher for executing an interface to process the task, and a default task dispatcher for routing said task to a task processor on a node designated to process said task in the cluster, wherein the task processor is selected by matching an attribute supplied with said task against an attribute supplied with a definition of the task processor; and

indicating to the client that a submission outcome is available.

11. The method of claim 10, wherein the session includes an identifier used for reconnecting to said session after a connection between the client and the session has been terminated.

12. The method of claim 10, further comprising:

storing an intermediate state for said task, wherein the intermediate state is used to suspend and resume said task.

13. The method of claim 10, wherein the logging dispatcher logs the task submission in a log file;

and wherein the executor dispatcher executes a Java Runnable interface and a Java Callable interface.

14. The method of claim 10, further comprising:

storing the task submission in a submissions cache distributed on said cluster before said task submission is provided to the dispatcher.

15. The method of claim 10, further comprising:

storing the submission outcome in a submissions result cache distributed on said cluster before the submission outcome is provided to the client.

16. The method of claim 10, wherein the task dispatcher is configured to dispatch said task to the task processor using at least one of a set of dispatch policies that includes a round robin dispatch policy, and a random dispatch policy.

17. The method of claim 10, wherein the task is associated with a set of metadata that is used to dispatch the task to the task processor.

20. The method of claim 10, wherein the plurality of dispatchers are consulted in the particular order to determine whether the task submission should be processed or routed.

18. A non-transitory computer readable storage medium storing a set of instructions executed by one or more processors to perform a sequence of steps comprising:

establishing a session between a client and a cluster of computer nodes using an application programming interface (API) provided by the cluster, wherein the API and said session are used by said client to submit a task for processing by the computer nodes of the cluster;

receiving a submission of said task from the client;

dispatching, via a dispatcher controller, the submission of said task to a plurality of dispatchers registered in a particular order, wherein the dispatchers include a logging dispatcher for logging the task submission, an executor dispatcher for executing an interface to process the task, and a default task dispatcher for routing said task to a task processor on a node designated to process said task in the cluster, wherein the task processor is selected by matching an attribute supplied with said task against an attribute supplied with a definition of the task processor; and

indicating to the client that a submission outcome is available.