Consolidating tasks into a composite request

• US 9,811,390 B1
• Filed: 03/30/2015
• Issued: 11/07/2017
• Est. Priority Date: 03/30/2015
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Ma, Vincent, Pendergraft, James
• Primary Examiner(s): Patel, Hiren
• Application Number:
  US14/673,677
• Time in Patent Office: 953 Days
• Field of Search: None
• CPC Class Codes: G06F 9/46 Multiprogramming arrangements
• CPC Class Codes: G06F 9/5038 considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration scheduling strategies G06F9/4881 and subgroups
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Described are techniques for consolidating tasks into a composite job. A server processes a composite request to create a composite job on the server. The composite request comprises a plurality of nested job create requests. The composite job comprises a plurality of jobs. Each job in the plurality of jobs corresponds to a job create request in the plurality of nested job create requests. The server generates a dependency tree indicating a dependency of each job in the plurality of jobs. The server executes the composite job in accordance with the dependency tree. On determining that a first job is dependent on a second job, the server executes the second job prior to executing the first job. The server passes the output of the second job as input to the first job.

Claims

1. A server comprising:

at least one processor;

a memory storage device associated with the at least one processor, the memory storage device comprising a memory area storing a system management component, wherein the at least one processor executes the system management component to;

process a composite hypertext transfer protocol (HTTP) POST job request, the composite HTTP POST job request comprising a plurality of nested job create requests, the plurality of nested job create requests defining a plurality of jobs, respectively, to be performed on one or more management resources of a data storage system;

create a composite job in accordance with the composite HTTP POST job request on at least one data storage system server, the composite job comprising the plurality of jobs to be performed on the one or more management resources, each job in the plurality of jobs corresponding to a respective nested job create request in the plurality of nested job create requests, the composite job having a single composite job identifier for tracking execution progress of the composite job and each job in the plurality of jobs corresponding to the respective nested job create request;

execute the composite job, by the at least one processor, in accordance with a dependency for each job in the plurality of jobs associated with the composite job; and

upon determining that the dependency indicates a first job in the plurality of jobs is dependent on a second job in the plurality of jobs, execute the second job prior to executing the first job and pass an output of the second job as an input to the first job,

wherein the composite HTTP POST job request further comprises;

a composite request HTTP header; and

a composite request uniform resource locator (URL),

wherein the at least one processor executes computer executable program code to apply the composite request HTTP header and the composite request URL to each nested job create request in the plurality of nested job create requests, and

wherein each nested job create request in the plurality of nested job create requests share a same HTTP header and URL provided for the composite HTTP POST job request.

2. The server of claim 1, wherein a nested job create request in the plurality of nested job create requests are encapsulated within a body of the composite HTTP POST job request, and wherein each nested job create request comprises a set of parameters, the set of parameters further comprising:

a name parameter, the name parameter specifying a name of a nested job corresponding to the nested job create request that is unique within the composite HTTP POST job request;

an object parameter, the object parameter specifying a target object name of the nested job create request;

an action parameter, the action parameter specifying a target action name of the nested job create request;

a parameters-in parameter, the parameters-in parameter specifying an output from at least one job that is required as input to the nested job corresponding to the nested job create request; and

a parameters-out parameter, the parameters-out parameter specifying an output to be generated by the nested job corresponding to the nested job create request.

3. The server of claim 1, further comprising:

a representational state transfer (REST) application programming interface (API), wherein the composite job identifier is returned to a client via the REST API.

4. The server of claim 1, wherein the at least one processor further executes the system management component to:

track a current status of the composite HTTP POST job request at the at least one data storage system server, wherein the current status of the composite HTTP POST job request comprises a status of each job in the plurality of jobs and an execution progress of each job in the plurality of jobs;

receive a composite job status update request from a client, the composite job status update request comprising the composite job identifier; and

return the current status of each job in the composite job to the client in response to the composite job status update request.

5. The server of claim 4, wherein the data storage system is a cloud storage system, wherein the composite HTTP POST job request comprises two or more tasks to be performed on cloud storage, and wherein a status of a job in the plurality of jobs comprises at least one of a job queued state, a job running state, a job suspended state, a job completed state, or a job failed state.

20. The server of claim 1, wherein the composite HTTP POST job request has a body, and wherein the at least one processor further executes the system management component to:

encapsulate the plurality of nested job create requests within the body of the composite HTTP POST job request; and

submit the composite HTTP POST job request to the one or more management resources for processing of the encapsulated nested job create requests.

6. A computer-implemented method for consolidating tasks into a composite job, the method comprising:

receiving a composite hypertext transfer protocol (HTTP) POST job request via an application programming interface of a computing device associated with a data storage system, the composite HTTP POST job request comprising a plurality of nested job requests encapsulated within a body of the composite HTTP POST job request, the plurality of nested job requests comprising a set of parameters defining a plurality of individual jobs, respectively, to be performed on one or more management resources of the data storage system;

creating a composite job corresponding to the composite HTTP POST job request, the composite job comprising the plurality of individual jobs to be performed on the one or more management resources, each individual job in the plurality of individual jobs corresponding to a respective nested job request in the plurality of nested job requests, and the composite job having a single composite job identifier for tracking execution progress of the composite job and each individual job in the plurality of individual jobs corresponding to the respective nested job request;

generating a dependency tree;

executing each individual job in the plurality of individual jobs associated with the composite job, by a processing device, in accordance with the dependency tree, the composite HTTP POST job request further comprising a composite request HTTP header, and a composite request uniform resource locator (URL); and

applying the composite request HTTP header and the composite request URL to each nested job request in the plurality of nested job requests, each nested job create request in the plurality of nested job create requests sharing a same HTTP header and URL provided for the composite HTTP POST job request.

7. The computer-implemented method of claim 6, wherein the composite job is created and executed on a data storage system server, and further comprising:

tracking an execution progress of the composite job, wherein tracking the execution progress of the composite job further comprises tracking an execution progress of each individual job in the plurality of individual jobs; and

on determining a request for a status update for the composite HTTP POST job request has been received from a client, returning a current status of the composite job to the client, wherein the request for the status update comprises the composite job identifier, and wherein the composite job status comprises a status and a progress of execution of each individual job in the plurality of individual jobs executing on the server.

8. The computer-implemented method of claim 6, further comprising:

generating the dependency tree during runtime, wherein the dependency tree comprises a first set of jobs in the plurality of individual jobs to be executed serially, and wherein the dependency tree specifies a second set of jobs in the plurality of individual jobs to be executed concurrently.

9. The computer-implemented method of claim 6, further comprising:

receiving the composite HTTP POST job request via an application programming interface of a cloud server, wherein the data storage system is a cloud storage system, wherein the composite HTTP POST job request comprises two or more tasks to be performed on cloud storage;

processing the composite HTTP POST job request by the processing device of the cloud server; and

returning a composite job update to a client, via the application programming interface, the composite job update comprising a status of each individual job in the plurality of individual jobs.

10. The computer-implemented method of claim 6, wherein the application programming interface is a representation state transfer (REST) application programming interface.

11. The computer-implemented method of claim 6, wherein executing each individual job in the plurality of individual jobs further comprises:

on determining a first job in the plurality of individual jobs is dependent upon a second job in the plurality of individual jobs, executing the second job prior to execution of the first job;

obtaining an output from the second job at completion of execution of the second job to form a second job output;

sending the second job output to the first job, wherein the second job output is an input to the first job; and

executing the first job serially after execution of the second job.

12. The computer-implemented method of claim 11, further comprising:

on determining a third job in the plurality of individual jobs is dependent upon the first job, executing the first job prior to beginning execution of the third job;

obtaining an output from the first job at completion of execution of the first job to form a first job output;

sending the first job output to the third job, wherein the first job output is an input to the third job; and

executing the third job after completion of execution of the first job.

13. The computer-implemented method of claim 6, further comprising:

receiving an explicit dependency for a selected job, wherein the explicit dependency is a user selected dependency indicating that the selected job is dependent on at least one individual job in the plurality of individual jobs; and

executing the selected job after execution of the at least one individual job.

14. One or more computer storage media embodying computer executable components, said components comprising a systems management component that, when executed, cause at least one processor to:

parse a composite hypertext transfer protocol (HTTP) POST job request received from a client, the composite HTTP POST job request comprising a plurality of nested job requests, the plurality of nested job requests defining a plurality of nested jobs, respectively, to be performed on one or more data storage system resources;

generate a composite job corresponding to the composite HTTP POST job request, the composite job comprising the plurality of nested jobs to be performed on the one or more data storage system resources, each nested job in the plurality of nested jobs corresponding to a respective nested job request in the plurality of nested job requests;

return a composite job identifier to the client, the composite job identifier for tracking execution progress of the composite job and each nested job in the plurality of nested jobs corresponding to the respective nested job request, the composite job identifier identifying the composite job;

track execution of the composite job to form a composite job status, the composite job status including an execution status of each nested job in the plurality of nested jobs;

return the composite job status to the client in response to receiving a composite job status update request, the composite job status comprising a current execution status of each nested job in the plurality of nested jobs, the composite job update request comprising the composite job identifier;

return a composite job complete status to the client on determining execution of the plurality of nested jobs is complete, the composite HTTP POST job request further comprising a composite request HTTP header, and a composite request uniform resource locator (URL); and

apply the composite request HTTP header and the composite request URL to each nested job request in the plurality of nested job requests, each nested job request in the plurality of nested job requests sharing a same HTTP header and URL provided for the composite HTTP POST job request.

15. The one or more computer storage media of claim 14, wherein the composite HTTP POST job request comprises two or more tasks to be performed on cloud storage, and wherein the system management component is further executed to cause the at least one processor to:

re-start execution of one or more jobs on a cloud server associated with the cloud storage on determining that the execution of the one or more jobs in the plurality of nested jobs is interrupted.

16. The one or more computer storage media of claim 14, wherein the composite HTTP POST job request is received from the client via a representational state transfer (REST) application programming interface.

17. The one or more computer storage media of claim 14, wherein a first job is dependent on a second job, and wherein the wherein the system management component is further executed to cause the at least one processor to:

execute the second job prior to execution of the first job; and

pass an output of the second job to the first job, wherein the second job output is an input to the first job.

18. The one or more computer storage media of claim 17, wherein a third job is dependent on the first job and the second job, wherein the first job, second job, and third job are executed serially, and wherein the system management component is further executed to cause the at least one processor to:

execute the first job prior to execution of the third job;

pass an output of the second job to the third job, wherein the second job output is a first input to the third job; and

pass an output of the first job to the third job, wherein the first job output is a second input to the third job.

19. The one or more computer storage media of claim 14, wherein a first job in the plurality of nested jobs is free of dependencies on any other job in the plurality of nested jobs, wherein a second job in the plurality of nested jobs is also free of dependencies on any other jobs in the plurality of nested jobs, and wherein the system management component is further executed to cause the at least one processor to:

execute the first job and the second job concurrently.