Systems and methods for caching task execution
First Claim
1. A method for processing a job in a form of computer-executable code, comprising:
- receiving, at a client device over a network, information representing the job;
receiving the job at a job scheduler of a master device;
dividing, by the job scheduler, the job into at least two tasks comprising a first task and a second task;
for the first task;
generating, by the job scheduler, a signature corresponding to the first task representative of whether the first task has been processed;
searching, by a task scheduler of the master device, a data structure for the generated signature;
if the signature is found in the data structure retrieving a result associated with the first task by the task scheduler;
if the signature is not found in the data structure,sending the first task by the task scheduler over the network to a task executor device,processing the first task by the task executor device,receiving a result of the first task processing by the task scheduler, andstoring the task result and a signature corresponding to the processed first task in the data structure by the task scheduler;
aggregating, by the job scheduler, the task result into a job result;
sending, by the job scheduler and over the network, the job result to the client device; and
processing the job result by the client device.
1 Assignment
0 Petitions
Accused Products
Abstract
Certain disclosed embodiments provide improved systems and methods for processing jobs. The method comprises steps including receiving, from a client device over a network, information representing a job and generating at least two tasks representative of the job. The method further comprises, for each task, assigning, by a processor, a signature to the task representative of whether the task has been processed, determining at least one dataset related to the task, and assigning a signature to the determined at least one dataset. The method further comprises searching, by the processor, a data structure for the task signature, and based on the searching, sending the task over a network to a task executor for processing or locating results associated with the task. The method further comprises sending, over a network, a job result to the client device. Systems and computer-readable media are also provided.
-
Citations
17 Claims
-
1. A method for processing a job in a form of computer-executable code, comprising:
-
receiving, at a client device over a network, information representing the job; receiving the job at a job scheduler of a master device; dividing, by the job scheduler, the job into at least two tasks comprising a first task and a second task; for the first task; generating, by the job scheduler, a signature corresponding to the first task representative of whether the first task has been processed; searching, by a task scheduler of the master device, a data structure for the generated signature; if the signature is found in the data structure retrieving a result associated with the first task by the task scheduler; if the signature is not found in the data structure, sending the first task by the task scheduler over the network to a task executor device, processing the first task by the task executor device, receiving a result of the first task processing by the task scheduler, and storing the task result and a signature corresponding to the processed first task in the data structure by the task scheduler; aggregating, by the job scheduler, the task result into a job result; sending, by the job scheduler and over the network, the job result to the client device; and processing the job result by the client device. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17)
-
Specification