Distributing and executing tasks in peer-to-peer distributed computing

• US 20040019890A1
• Filed: 10/04/2002
• Published: 01/29/2004
• Est. Priority Date: 07/23/2002
• Status: Abandoned Application

• Current Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Sharapov, Ilya A., Ruetsch, Gregory R., Verbeke, Jerome M., Vernik, Michael J., Nadgir, Neelakanth M., Trang, Vu H.
• Application Number:
  US10/265,029
• Publication Number:
  US 20040019890A1
• Time in Patent Office: Days
• US Class Current: 718/100
• CPC Class Codes: G06F 2209/5017 Task decomposition
• CPC Class Codes: G06F 9/5055 considering software capabilities, i.e. software resources associated or available to the machine

Abstract

The present invention utilizes peer groups in a distributed architecture to decentralize its task dispatching and post-processing functions and to provide the ability to manage and run many different applications simultaneously, in an efficient and reliable manner. Jobs may be submitted to a task dispatcher or to a monitor which distributes the jobs to task dispatchers. Through a series of processes, the task dispatchers may then distribute the jobs to workers. This allows work to be distributed without utilizing a centralized server.

Claims

1. A method for coordinating a job submission in a distributed computing framework, comprising:

receiving an identification of a code to be executed from a job submitter;

uploading said code to be executed to said code repository;

creating a job repository corresponding to said job submission;

receiving one or more tasks from a job submitter; and

storing each of said one or more tasks in a task repository linked to said job repository.

2. The method of claim 1, further comprising:

accessing a repository manager to determine whether said identification of said code to be executed already exists in a code repository;

requesting said code to be executed from said job submitter if said identification of said code to be executed does not already exist in said code repository; and

receiving said code to be executed from said job submitter if said identification of said code to be executed does not already exist in said code repository.

3. The method of claim 2, wherein said uploading comprises uploading said code to be executed to said code repository if said identification of said code to be executed does not already exist in said code repository.

4. The method of claim 1, wherein said job repository is stored on multiple computers.

5. The method of claim 1, wherein said creating and storing are performed by a repository manager.

6. The method of claim 1, wherein said job repository is part of a repository peer group.

7. The method of claim 1, wherein said receiving, uploading, creating, receiving, and storing are performed using a peer-to-peer protocol.

8. The method of claim 7 wherein said peer-to-peer protocol is Juxtapose (JXTA).

9. A method for coordinating a job in a distributed computing framework, comprising:

receiving a poll from an idle worker, said poll including information regarding resources available from said idle worker;

polling a repository for tasks to be performed on available codes;

distributing one or more of said tasks to said worker, said one or more tasks chosen based on said information;

receiving a result of a task execution from said worker; and

updating said repository with information about task completion.

10. The method of claim 9, wherein said information regarding resources includes information regarding codes cached by said worker.

11. The method of claim 9, further comprising:

polling a repository for code to be downloaded to said worker; and

downloading said code to be downloaded to said worker.

12. The method of claim 9, wherein said polling a repository comprises contacting a repository manager.

13. The method of claim 12, wherein said repository manager controls one or more repositories in a repository peer group.

14. The method of claim 9, wherein said receiving, polling, distributing, receiving, and updating are performed using a peer-to-peer protocol.

15. The method of claim 14 wherein said peer-to-peer protocol is Juxtapose (JXTA).

19. The method of claim 14 wherein said peer-to-peer protocol is Juxtapose (JXTA).

16. A method for coordinating execution of a task by an idle worker in a distributed computing framework, comprising:

polling a task dispatcher to inform said task dispatcher that said worker is idle and provide information regarding resources available from said worker;

receiving one or more tasks from said task dispatcher;

executing said one or more tasks; and

returning results of said execution to said task dispatcher.

18. The method of claim 16, wherein said polling, receiving, and returning are performed using a peer-to-peer protocol.

20. The method of claim 16, wherein said task dispatcher is a task dispatcher manager.

21. The method of claim 20, wherein said task dispatcher manager controls one or more task dispatchers in a task dispatcher peer group.

17. The method of claim 17, wherein said information regarding resources includes information regarding codes cached by said worker.

22. An apparatus for coordinating a job submission in a distributed computing framework, comprising:

a code to be executed identification receiver;

a code to be executed code repository uploader coupled to said code to be executed identification receiver;

a job repository creator coupled to said code to be executed identification receiver;

a job submitter task receiver; and

a task repository storer coupled to said job submitter task receiver and to said job repository creator.

23. The apparatus of claim 22, further comprising:

a repository manager accessor;

a code to be executed requester coupled to said repository manager accessor; and

a code to be executed receiver coupled to said code to be executed code repository uploader.

24. An apparatus for coordinating a job in a distributed computing framework, comprising:

an idle worker poll receiver;

a repository poller coupled to said idle worker poll receiver;

a worker task distributor coupled to said repository poller;

a task execution result receiver; and

a repository information updater coupled to said task execution result receiver.

25. The apparatus of claim 24, further comprising:

a worker code repository poller; and

a worker code downloader coupled to said worker code repository poller.

26. An apparatus for coordinating execution of a task by an idle worker in a distributed computing framework, comprising:

a task dispatcher poller;

a task receiver;

a task executor coupled to said task receiver; and

an execution result returner coupled to said task executor.

27. An apparatus for coordinating a job submission in a distributed computing framework, comprising:

means for receiving an identification of a code to be executed from a job submitter;

means for uploading said code to be executed to said code repository;

means for creating a job repository corresponding to said job submission;

means for receiving one or more tasks from a job submitter; and

means for storing each of said one or more tasks in a task repository linked to said job repository.

28. The apparatus of claim 27, further comprising:

means for accessing a repository manager to determine whether said identification of said code to be executed already exists in a code repository;

means for requesting said code to be executed from said job submitter if said identification of said code to be executed does not already exist in said code repository; and

means for receiving said code to be executed from said job submitter if said identification of said code to be executed does not already exist in said code repository.

29. The apparatus of claim 28, wherein said means for uploading comprises means for uploading said code to be executed to said code repository if said identification of said code to be executed does not already exist in said code repository.

30. The apparatus of claim 27, wherein said job repository is stored on multiple computers.

31. The apparatus of claim 27, wherein said means for creating and means for storing are a repository manager.

32. The apparatus of claim 27, wherein said job repository is part of a repository peer group.

33. The apparatus of claim 27, wherein said means for receiving, means for uploading, means for creating, means for receiving, and means for storing use a peer-to-peer protocol.

34. The apparatus of claim 33 wherein said peer-to-peer protocol is Juxtapose (JXTA).

35. An apparatus for coordinating a job in a distributed computing framework, comprising:

means for receiving a poll from an idle worker, said poll including information regarding resources available from said idle worker;

means for polling a repository for tasks to be performed on available codes;

means for distributing one or more of said tasks to said worker, said one or more tasks chosen based on said information;

means for receiving a result of a task execution from said worker; and

means for updating said repository with information about task completion.

36. The apparatus of claim 35, wherein said information regarding resources includes information regarding codes cached by said worker.

37. The apparatus of claim 35, further comprising:

means for polling a repository for code to be downloaded to said worker; and

means for downloading said code to be downloaded to said worker.

38. The apparatus of claim 35, wherein said means for polling a repository comprises means for contacting a repository manager.

39. The apparatus of claim 38, wherein said repository manager controls one or more repositories in a repository peer group.

40. The apparatus of claim 36, wherein said means for receiving, means for polling, means for distributing, means for receiving, and means for updating are performed using a peer-to-peer protocol.

41. The apparatus of claim 40 wherein said peer-to-peer protocol is Juxtapose (JXTA).

42. An apparatus for coordinating execution of a task by an idle worker in a distributed computing framework, comprising:

means for polling a task dispatcher to inform said task dispatcher that said worker is idle and provide information regarding resources available from said worker;

means for receiving one or more tasks from said task dispatcher;

means for executing said one or more tasks; and

means for returning results of said execution to said task dispatcher.

43. The apparatus of claim 42, wherein said information regarding resources includes information regarding codes cached by said worker.

44. The apparatus of claim 42, wherein said means for polling, means for receiving, and means for returning are performed using a peer-to-peer protocol.

45. The apparatus of claim 44 wherein said peer-to-peer protocol is Juxtapose (JXTA).

46. The apparatus of claim 42, wherein said task dispatcher is a task dispatcher manager.

47. The apparatus of claim 46, wherein said task dispatcher manager controls one or more task dispatchers in a task dispatcher peer group.

48. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform a method for coordinating a job submission in a distributed computing framework, the method comprising:

receiving an identification of a code to be executed from a job submitter;

uploading said code to be executed to said code repository;

creating a job repository corresponding to said job submission;

receiving one or more tasks from a job submitter; and

storing each of said one or more tasks in a task repository linked to said job repository.

49. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform a method for coordinating a job in a distributed computing framework, the method comprising:

receiving a poll from an idle worker, said poll including information regarding resources available from said idle worker;

polling a repository for tasks to be performed on available codes;

distributing one or more of said tasks to said worker, said one or more tasks chosen based on said information;

receiving a result of a task execution from said worker; and

updating said repository with information about task completion.

50. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform a method for coordinating execution of a task by an idle worker in a distributed computing framework, the method comprising:

polling a task dispatcher to inform said task dispatcher that said worker is idle and provide information regarding resources available from said worker;

receiving one or more tasks from said task dispatcher;

executing said one or more tasks; and

returning results of said execution to said task dispatcher.