System and method for rapid completion of data processing tasks distributed on a network

• US 6,775,831 B1
• Filed: 02/11/2000
• Issued: 08/10/2004
• Est. Priority Date: 02/11/2000
• Status: Expired due to Term

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Overture Services Incorporated (Apollo Global Management, Inc.)
• Inventors: Doliov, Stephan, Carrasco, John Joseph, Ehrenfried, Frank B.
• Primary Examiner(s): An, Meng-Al T.
• Assistant Examiner(s): Vo, Lilian
• Application Number:
  US09/502,692
• Time in Patent Office: 1,642 Days
• Field of Search: 709/100, 709/102, 709/106, 709/107, 709/200, 707/3, 707/200, 707/10, 707/103, 718/100, 718/102, 718/106, 718/107
• US Class Current: 718/100
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

A computer program and task request interface which enables large volumes of data to be processed efficiently and rapidly by multiple computers on a network. The data that requires processing can be in any format. The task request interface allows any arbitrary task to be performed on the input data. The number of computers which may participate in any given data processing task is essentially unlimited. The number of programs that can participate in the system is also essentially unlimited if the program uses run time evaluable code. Any computer operating system that runs a specified computer language interpreter can participate in the distributed processing of the input data.

Claims

1. A computer implemented method for running tasks on a network, comprising:

creating at least one sub-group of data from a universe of data;

identifying the sub-group of data with a header, the header containing executable code;

formatting a control file for evaluation as a hash table at run time;

reading the control file to determine an availability of a processor;

sending the sub-group of data to the available processor; and

performing tasks with the available processor to obtain result data using the sub-group of data and instructions contained in the executable code in the header.

2. The method according to claim 1, further including returning the result data files for each data sub-group to a main processor for possible further processing.

3. The method according to claim 2, wherein returned result data is reconstructed to form a single result data file which comprises many individual result data files.

4. The method according to claim 1 further including updating the control file by re-writing a new control file which indicates the status of the processor.

5. The method according to claim 1 further including:

assembling at least one command to be executed by the processor; and

sending the at least one command to the processor for execution.

6. The method according to claim 5, wherein the at least one command includes creating a name to identify tasks to run.

7. The method according to claim 1 further including marking a state of the tasks.

8. The method according to claim 7, wherein the state of the task comprises either running or completed.

9. The method according to claim 1 further including retrying, after a time period has elapsed, to determine if the availability of the processor if all processors were busy.

10. The method according to claim 9, wherein the time period comprises about fifteen minutes.

11. A computer implemented method for running tasks on a network, comprising:

creating at least one sub-group of data from a universe of data;

identifying the sub-group of data with a header, the header containing executable code;

formatting a control file for evaluation as a hash table at run time;

reading the control file to determine an availability of a processor;

sending the sub-group of data to the available processor;

performing tasks with the available processor to obtain result data using the sub-group of data and instructions contained in the executable code in the header; and

returning the result data to a main processor, wherein returned result data is reconstructed to form a result.

12. The method according to claim 11 further including updating the control file by re-writing a new control file which indicates the status of the processor.

13. The method according to claim 11 further including:

assembling at least one command to be executed by the processor; and

sending the at least one command to the processor for execution.

14. The method according to claim 13, wherein the at least one command includes creating a name to identify tasks to run.

15. A system for running tasks on a network, comprising:

a data preprocessing routine to create at least one sub-group of data from a universe of data;

a data header containing executable code that identifies the sub-group of data;

a control file capable of being read to determine an availability of a processor, wherein the control file is formatted for evaluation as a hash table at run time; and

a dispatch routine to send the sub-group of data to the available processor, wherein the available processor performs to obtain result data using the sub-group of data and instructions contained in the executable code in the header.

16. The system according to claim 15, further including a main processor to collect the result data files for each data sub-group.

17. The system according to claim 16, further including a data reconstructing routine to reconstruct the result data to form a result.

18. The system according to claim 15, wherein the dispatch routine updates the control file by re-writing a new control file that indicates the status of the processor.

19. The system according to claim 15, wherein the dispatch routine assembles at least one command to be executed by the processor and sends the at least one command to the processor for execution.

20. The system according to claim 19, wherein the command includes creating a name to identify the tasks running on the network.

21. The system according to claim 15, wherein the dispatch routine marks a state of the task.

22. The system according to claim 21, wherein the state of the task is either running or completed.

23. The system according to claim 22, wherein the dispatch routine retries, after a time period has elapsed, to determine the availability of the processor if all the processors were busy.

24. A system for running tasks on a network, comprising:

a preprocessing routine to create at least one sub-group of data from a universe of data;

a data header containing executable code that identifies the sub-group of data;

a control file capable of being read to determine an availability of a processor, wherein the control file is formatted for evaluation as a hash table at run time; and

a dispatch routine to send the sub-group of data to the available processor, wherein the available processor performs to obtain result data using the sub-group of data and instructions contained in the executable code in the header; and

a main processor to collect returned result data, wherein returned result data is reconstructed to form a result.

25. The system according to claim 24, wherein the dispatch routine updates the control file by re-writing a new control file that indicates the status of the processor.

26. The system according to claim 24, wherein the dispatch routine assembles at least one command to be executed by the processor and sends the at least one command to the processor for execution.

27. The system according to claim 26, wherein the command includes creating a name to identify the tasks running on the network.

28. The system according to claim 24, wherein the dispatch routine marks a state of the task.

29. The system according to claim 28, wherein the state of the task is either running or completed.

30. The system according to claim 29, wherein the dispatch routine retries, after a time period has elapsed, to determine the availability of the processor if all the processors were busy.