Distributed data set storage and analysis reproducibility
First Claim
1. An apparatus comprising a processor and a storage to store instructions that, when executed by the processor, cause the processor to perform operations comprising:
- receive, at a portal, and from a remote device via a the network, a first request to execute at least one task routine specified by the first request as stored in a federated area to perform at least one corresponding task of a job flow specified in a job flow definition stored in the federated area with at least one data set specified by the first request as stored in the federated area, wherein;
the portal is provided on the network to control access to the federated area by the remote device via the network; and
the federated area is maintained within one or more storage devices to store multiple data sets, multiple job flow definitions, multiple task routines, multiple result reports and multiple instance logs;
retrieve the job flow definition from among the multiple job flow definitions stored in the federated area;
retrieve the at least one data set from among the multiple data sets stored in the federated area;
determine whether there is at least one instance log among the multiple instance logs stored in the federated area that was generated by a previous performance of the at least one task of the job flow with the at least one data set;
in response to a determination that there is just a single instance log among the multiple instance logs stored in the federated area that was generated by a previous performance of the at least one task of the job flow with the at least one data set, the processor is caused to perform operations comprising;
retrieve a version specified by the single instance log of each task routine of the at least one task routine from among the multiple task routines stored in the federated area;
execute the retrieved version of each task routine of the at least one task routine to perform the at least one corresponding task of the job flow with the at least one data set to generate a new result report and a new instance log;
store the new result report among the multiple result reports in the federated area;
store the new instance log among the multiple instance logs in the federated area; and
provide access to the new result report to the remote device via the portal; and
in response to a determination that there is more than one instance log among the multiple instance logs that were each generated by a previous performance of the at least one task of the job flow with the at least one data set, the processor is caused to perform operations comprising;
select the most recently generated one of the more than one instance log to be the single instance log;
retrieve the version specified by the single instance log of each task routine of the at least one task routine from among the multiple task routines stored in the federated area;
execute the retrieved version of each task routine of the at least one task routine to perform the at least one corresponding task of the job flow with the at least one data set to generate a new result report and a new instance log;
store the new result report among the multiple result reports in the federated area;
store the new instance log among the multiple instance logs in the federated area; and
provide access to the new result report to the remote device via the portal.
0 Assignments
0 Petitions
Accused Products
Abstract
An apparatus includes a processor and a storage storing instructions causing the processor to: maintain a federated area; receive a request to perform a job flow with a data set from a remote device; retrieve a job flow definition specifying the tasks of the job flow from the federated area; determine whether there is an instance log in the federated area generated by a previous performance of the job flow with the data set; in response to there being such an instance log, retrieve the version specified in the instance log of each task routine for each task from the federated area; in response to there being no such instance log, retrieve the most recent version of each task routine; perform the job flow with the retrieved versions of the task routines and the data set to generate a result report; and provide the result report to the remote device.
-
Citations
30 Claims
-
1. An apparatus comprising a processor and a storage to store instructions that, when executed by the processor, cause the processor to perform operations comprising:
-
receive, at a portal, and from a remote device via a the network, a first request to execute at least one task routine specified by the first request as stored in a federated area to perform at least one corresponding task of a job flow specified in a job flow definition stored in the federated area with at least one data set specified by the first request as stored in the federated area, wherein; the portal is provided on the network to control access to the federated area by the remote device via the network; and the federated area is maintained within one or more storage devices to store multiple data sets, multiple job flow definitions, multiple task routines, multiple result reports and multiple instance logs; retrieve the job flow definition from among the multiple job flow definitions stored in the federated area; retrieve the at least one data set from among the multiple data sets stored in the federated area; determine whether there is at least one instance log among the multiple instance logs stored in the federated area that was generated by a previous performance of the at least one task of the job flow with the at least one data set; in response to a determination that there is just a single instance log among the multiple instance logs stored in the federated area that was generated by a previous performance of the at least one task of the job flow with the at least one data set, the processor is caused to perform operations comprising; retrieve a version specified by the single instance log of each task routine of the at least one task routine from among the multiple task routines stored in the federated area; execute the retrieved version of each task routine of the at least one task routine to perform the at least one corresponding task of the job flow with the at least one data set to generate a new result report and a new instance log; store the new result report among the multiple result reports in the federated area; store the new instance log among the multiple instance logs in the federated area; and provide access to the new result report to the remote device via the portal; and in response to a determination that there is more than one instance log among the multiple instance logs that were each generated by a previous performance of the at least one task of the job flow with the at least one data set, the processor is caused to perform operations comprising; select the most recently generated one of the more than one instance log to be the single instance log; retrieve the version specified by the single instance log of each task routine of the at least one task routine from among the multiple task routines stored in the federated area; execute the retrieved version of each task routine of the at least one task routine to perform the at least one corresponding task of the job flow with the at least one data set to generate a new result report and a new instance log; store the new result report among the multiple result reports in the federated area; store the new instance log among the multiple instance logs in the federated area; and provide access to the new result report to the remote device via the portal. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, the computer-program product including instructions operable to cause a processor to perform operations comprising:
-
receive, at a portal, and from a remote device via a the network, a first request to execute at least one task routine specified by the first request as stored in a federated area to perform at least one corresponding task of a job flow specified in a job flow definition stored in the federated area with at least one data set specified by the first request as stored in the federated area, wherein; the portal is provided on the network to control access to the federated area by the remote device via the network; and the federated area is maintained within one or more storage devices to store multiple data sets, multiple job flow definitions, multiple task routines, multiple result reports and multiple instance logs; retrieve the job flow definition from among the multiple job flow definitions stored in the federated area; retrieve the at least one data set from among the multiple data sets stored in the federated area; determine whether there is at least one instance log among the multiple instance logs stored in the federated area that was generated by a previous performance of the at least one task of the job flow with the at least one data set; in response to a determination that there is just a single instance log among the multiple instance logs stored in the federated area that was generated by a previous performance of the at least one task of the job flow with the at least one data set, the processor is caused to perform operations comprising; retrieve a version specified by the single instance log of each task routine of the at least one task routine from among the multiple task routines stored in the federated area; execute the retrieved version of each task routine of the at least one task routine to perform the at least one corresponding task of the job flow with the at least one data set to generate a new result report and a new instance log; store the new result report among the multiple result reports in the federated area; store the new instance log among the multiple instance logs in the federated area; and provide access to the new result report to the remote device via the portal; and in response to a determination that there is more than one instance log among the multiple instance logs that were each generated by a previous performance of the at least one task of the job flow with the at least one data set, the processor is caused to perform operations comprising; select the most recently generated one of the more than one instance log to be the single instance log; retrieve the version specified by the single instance log of each task routine of the at least one task routine from among the multiple task routines stored in the federated area; execute the retrieved version of each task routine of the at least one task routine to perform the at least one corresponding task of the job flow with the at least one data set to generate a new result report and a new instance log; store the new result report among the multiple result reports in the federated area; store the new instance log among the multiple instance logs in the federated area; and provide access to the new result report to the remote device via the portal. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20)
-
-
21. A computer-implemented method comprising:
-
receiving, at a server by a processor, and via a portal on a network from a remote device, a first request to execute at least one task routine specified by the first request as stored in a federated area to perform at least one corresponding task of a job flow specified in a job flow definition stored in the federated area with at least one data set specified by the first request as stored in the federated area, wherein; the portal is provided on the network to control access to the federated area by the remote device via the network; and the federated area is maintained within one or more storage devices to store multiple data sets, multiple job flow definitions, multiple task routines, multiple result reports and multiple instance logs; retrieving, by the processor, the job flow definition from among the multiple job flow definitions stored in the federated area; retrieving, by the processor, the at least one data set from among the multiple data sets stored in the federated area; determining, by the processor, whether there is at least one instance log among the multiple instance logs stored in the federated area that was generated by a previous performance of the at least one task of the job flow with the at least one data set; in response to determining that there is just a single instance log among the multiple instance logs stored in the federated area that was generated by a previous performance of the at least one task of the job flow with the at least one data set, performing operations comprising; retrieving, by the processor, a version specified by the single instance log of each task routine of the at least one task routine from among the multiple task routines stored in the federated area; executing the retrieved version of each task routine of the at least one task routine to perform the at least one corresponding task of the job flow with the at least one data set to generate a new result report and a new instance log; storing, by the processor, the new result report among the multiple result reports in the federated area; storing, by the processor, the new instance log among the multiple instance logs in the federated area; and providing access to the new result report to the remote device via the portal; and in response to determining that there is more than one instance log among the multiple instance logs that were each generated by a previous performance of the at least one task of the job flow with the at least one data set, performing operations comprising; selecting, by the processor, the most recently generated one of the more than one instance log to be the single instance log; retrieving, by the processor, the version specified by the single instance log of each task routine of the at least one task routine from among the multiple task routines stored in the federated area; executing the retrieved version of each task routine of the at least one task routine to perform the at least one corresponding task of the job flow with the at least one data set to generate a new result report and a new instance log; storing, by the processor, the new result report among the multiple result reports in the federated area; storing, by the processor, the new instance log among the multiple instance logs in the federated area; and providing access to the new result report to the remote device via the portal. - View Dependent Claims (22, 23, 24, 25, 26, 27, 28, 29, 30)
-
Specification