Method and system for using arbitrary computing devices for distributed data processing

• US 20150067019A1
• Filed: 08/16/2014
• Published: 03/05/2015
• Est. Priority Date: 08/28/2013
• Status: Abandoned Application

• Current Assignee: Clipchamp Ip Pty Ltd
• Original Assignee: Clipchamp Ip Pty Ltd
• Inventors: Balko, Soeren
• Application Number:
  US14/461,400
• Publication Number:
  US 20150067019A1
• Time in Patent Office: Days
• US Class Current: 709/202
• CPC Class Codes: G06F 9/5038 considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration scheduling strategies G06F9/4881 and subgroups
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

A worker client (101, 102) has a runtime environment (120) obtained previously from a broker system (201) having a broker reference (422). It further has an interface component (111) communicating with the broker system (201) using the broker reference (422) previously obtained from a further computing device. The interface component (111) is adapted to receive at least one computing task specification. The runtime environment (120) is configured to process task input data according to the at least one computing task specification with a task program (122) resulting in task output data. The interface component (111) is further adapted to send the task output data to a previously determined recipient device.

Claims

1. A worker client comprising:

a runtime environment obtained previously from a broker system having a broker reference;

an interface component communicating with the broker system using the broker reference previously obtained from a further computing device, the interface component adapted to receive at least one computing task specification;

wherein the runtime environment is configured to process task input data according to the at least one computing task specification with a task program resulting in task output data; and

the interface component further adapted to send the task output data to a previously determined recipient device.

2. The worker client of claim 1, wherein the computing task specification further includes a task program indicator, and wherein, in case the runtime environment is lacking the task program indicated in the computing task specification, the interface component is further adapted to receive corresponding task program code executable in the runtime environment.

3. The worker client of claims 1, wherein the computing task specification further includes a data chunk indicating task input data, and wherein, in case the runtime environment is lacking the task input data indicated in the task specification, the interface component is further adapted to receive the task input data.

4. The worker client of claim 1, wherein:

the interface component is further adapted to receive a client context request;

the runtime environment is further comprising a client context component configured to evaluate the worker client based on the client context request; and

the interface component is further adapted to send the evaluated client context data.

5. The worker client of claim 1, wherein:

the interface component is further adapted to receive benchmarking code;

the runtime environment further comprising a benchmark component configured to evaluate the worker client based on the benchmarking code; and

the interface component is further adapted to send evaluated benchmark data.

9. A system for data processing, comprising:

at least one worker client configured according to claim 1; and

a broker system configured according to claim 6, wherein the at least one worker client and the broker system is temporarily communicatively coupled by a data connection to interchange data.

6. A broker system, comprising:

a consumer interface component adapted to receive a job execution specification;

an evaluation component configured to evaluate worker clients based on the job execution specification;

a compute job component configured to create at least one computing task having a computing task specification according to the evaluation of the worker clients; and

a deployment component configured to deploy the at least one computing task to a respective evaluated worker client.

7. The broker system of claim 6, further comprising:

a worker client interface component adapted to receive at least one task output data resulting from the at least one computing task processed by the respective evaluated worker client to be stored on the broker system;

a composer component configured to compose the at least one task output data to job output data; and

in case the job output data corresponds to the job execution specification, the consumer interface component further configured to send the job output data to a previously determined recipient device.

8. The broker system of claim 7, further comprising:

a monitor component adapted to receive a computing task indicator associated with the at least one computing task;

in case the computing task indicator indicates a task interruption, the deployment component further configured to deploy the at least one computing task to the respective evaluated worker client or a further respective evaluated worker client; and

in case the computing task indicator indicates a task completion of all of the at least one computing task, triggering the composer component to compose the job output data.

10. A computer implemented method for data processing by a worker client having a runtime environment obtained previously from a broker system having a broker reference, wherein the broker reference has been obtained by the worker client from a further computing device, the method comprising:

receiving from the broker system at least one computing task specification;

processing in the runtime environment task input data according to the at least one computing task specification with a task program resulting in task output data; and

sending the task output data to a previously determined recipient device.

11. The method of claim 10, wherein the computing task specification further includes a task program indicator, and, in case the runtime environment is lacking the task program indicated in the computing task specification, the method further comprising:

requesting corresponding task program code executable in the runtime environment.

12. The method of claim 10, wherein the computing task specification further includes a data chunk indicating task input data, and, in case the runtime environment is lacking the task input data indicated in the task specification, the method further comprising:

requesting the task input data prior to the processing.

13. A computer implemented method for data brokering, comprising:

receiving a job execution specification through a consumer interface component;

evaluating worker clients based on the job execution specification;

creating at least one computing task having a computing task specification according to the evaluation of the worker clients; and

deploying the at least one computing task to a respective evaluated worker client.

14. A worker client computer program product comprising instructions that when loaded into a memory of the worker client cause at least one processor of the worker client to execute the following steps:

receiving from a broker system at least one computing task specification;

processing in a runtime environment task input data according to the at least one computing task specification with a task program resulting in task output data, wherein the runtime environment has been obtained previously from a broker system having a broker reference, wherein the broker reference has been obtained by the worker client from a further computing device; and

sending the task output data to a previously determined recipient device.

15. The worker client computer program product of claim 14, wherein the computing task specification further includes a task program indicator, and, in case the runtime environment is lacking the task program indicated in the computing task specification, the computer program product comprising further instructions configured to cause the at least one processor to further execute:

requesting corresponding task program code executable in the runtime environment.

16. The worker client computer program product of claim 14, wherein the computing task specification further includes a data chunk indicating task input data, and, in case the runtime environment is lacking the task input data indicated in the task specification, the computer program product comprising further instructions configured to cause the at least one processor to further execute:

requesting the task input data prior to the processing.

17. A computer program product for data brokering, instructions that when loaded into a memory of a broker computer cause at least one processor of the broker computer to execute the following steps:

receiving a job execution specification through a consumer interface component;

evaluating worker clients based on the job execution specification;

creating at least one computing task having a computing task specification according to the evaluation of the worker clients; and

deploying the at least one computing task to a respective evaluated worker client.