System and method for providing intelligent pre-staging of data in a compute environment
First Claim
Patent Images
1. A method comprising:
- determining availability of compute resources in a compute environment including availability timeframes to process a workload submitted to the compute environment to yield a determined availability;
determining data requirements for processing the workload, the data requirements being associated with data to be retrieved from a storage medium for use by the workload, to yield determined data requirements;
based on the determined availability and the determined data requirements, establishing a data stage-in reservation associated with the data requirements for the workload to process that starts at a first start time, the data stage-in reservation only reserving resources associated with retrieving the data from the storage medium for use by the workload, wherein establishing the data stage-in reservation is based at least in part on (1) creating an availability range list from a calculation of existing resource guarantees and reservations and (2) converting the availability range list into a start range list;
performing data pre-staging according to the data stage-in reservation at the first start time; and
after performing the data pre-staging, processing the workload using the data and according to a compute reservation that starts at a second start time, which is later than the first start time.
8 Assignments
0 Petitions
Accused Products
Abstract
A system, computer-readable medium and method for performing intelligent data pre-staging for a job submitted to a cluster environment. The method aspect comprises determining availability of compute resources including availability timeframes to process the submitted job, determining data requirements for processing the job and determining a co-allocation in time reservation.
-
Citations
16 Claims
-
1. A method comprising:
-
determining availability of compute resources in a compute environment including availability timeframes to process a workload submitted to the compute environment to yield a determined availability; determining data requirements for processing the workload, the data requirements being associated with data to be retrieved from a storage medium for use by the workload, to yield determined data requirements; based on the determined availability and the determined data requirements, establishing a data stage-in reservation associated with the data requirements for the workload to process that starts at a first start time, the data stage-in reservation only reserving resources associated with retrieving the data from the storage medium for use by the workload, wherein establishing the data stage-in reservation is based at least in part on (1) creating an availability range list from a calculation of existing resource guarantees and reservations and (2) converting the availability range list into a start range list; performing data pre-staging according to the data stage-in reservation at the first start time; and after performing the data pre-staging, processing the workload using the data and according to a compute reservation that starts at a second start time, which is later than the first start time. - View Dependent Claims (2, 3, 4, 5, 16)
-
-
6. A system comprising:
-
a processor; and a computer-readable medium storing instructions, which when executed by the processor, cause the processor to perform operations comprising; determining data requirements for processing a workload, the data requirements being associated with data to be retrieved from a storage medium for use by the workload, to yield determined data requirements; based on the determined availability and the determined data requirements, establishing a data stage-in reservation associated with the data requirements for the workload to process that starts at a first start time, the data stage-in reservation only reserving resources associated with retrieving the data from the storage medium for use by the workload, establishing the data stage-in reservation based at least in part on (1) creating an availability range list from a calculation of existing resource guarantees and reservations and (2) converting the availability range list into a start range list; performing data pre-staging according to the data stage-in reservation at the first start time; and after performing the data pre-staging, processing the workload using the data and according to a compute reservation that starts at a second start time, which is later than the first start time. - View Dependent Claims (7, 8, 9, 10)
-
-
11. A non-transitory computer-readable medium storing instructions for controlling a computing device, which, when executed on the computing device, perform operations comprising:
-
determining data requirements for processing a workload, the data requirements being associated with data to be retrieved from a storage medium for use by the workload, to yield determined data requirements; based on the determined availability and the determined data requirements, establishing a data stage-in reservation associated with the data requirements for the workload to process that starts at a first start time, the data stage-in reservation only reserving resources associated with retrieving the data from the storage medium for use by the workload, wherein establishing the data stage-in reservation is based at least in part on (1) creating an availability range list from a calculation of existing resource guarantees and reservations and (2) converting the availability range list into a start range list; performing data pre-staging according to the data stage-in reservation at the first start time; and after performing the data pre-staging, processing the workload using the data and according to a compute reservation that starts at a second start time, which is later than the first start time. - View Dependent Claims (12, 13, 14, 15)
-
Specification