Federated device support for generation of directed acyclic graphs

• US 10,157,086 B2
• Filed: 02/16/2018
• Issued: 12/18/2018
• Est. Priority Date: 02/05/2016
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Bequet, Henry Gabriel Victor, Zhang, Chaowang "Ricky"
• Primary Examiner(s): Patel, Hiren P
• Application Number:
  US15/898,248
• Publication Number:
  US 20180189113A1
• Time in Patent Office: 305 Days
• CPC Class Codes: G06F 16/9014 hash tables
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 16/90344 by using string matching techniques
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system

Abstract

An apparatus including a processor to: parse comments of multiple task routines to identify I/O parameters; for each task routine, generate a macro including its I/O parameters; transmit the macros to a requesting device to enable it to generate a visualization of a DAG to include visual representations of the task routines; wherein each representation includes a task graph object, an input data graph object representing an input and indicating a characteristic of the input, and an output data graph object representing an output and indicating a characteristic of the output; and wherein the requesting device is to: identify, in the I/O parameters, each dependency between an output and an input of a pair of task routines; augment the visualization, for each identified dependency, with a dependency marker that visually links the visual representations of the pair of task routines; and visually output the visualization.

Claims

1. An apparatus comprising a processor and a storage to store instructions that, when executed by the processor, cause the processor to perform operations comprising:

receive, at a portal, and from a requesting device via a network, a first request to provide objects to the requesting device to enable the requesting device to generate a visualization of a first directed acyclic graph (DAG) of multiple task routines of a first job flow of a first analysis routine, wherein;

the portal is provided on the network to control access to at least one federated area by the requesting device via the network;

the at least one federated area is maintained within one or more storage devices to store a plurality of data objects, a plurality of task routines and a plurality of job flow definitions;

each task routine of the plurality of task routines comprises comments that specify input and output (I/O) parameters that comprise at least one characteristic of an input to the task routine and at least one characteristic of an output generated during execution of the task routine;

the I/O parameters of the comments of a subset of task routines of the plurality of task routines comprises an identifier of at least one data object of the plurality of data objects;

each task routine of the multiple task routines comprises executable instructions to perform a task when executed; and

each job flow definition specifies multiple tasks of a job flow of an analysis routine to be performed by task routines of the plurality of task routines;

retrieve each task routine of the multiple task routines from the at least one federated area;

parse the comments of each task routine of the multiple task routines to identify and retrieve the specification of the I/O parameters of the task routine from the comments;

for each task routine of the multiple task routines, generate a corresponding macro of multiple macros that comprises an indication of the I/O parameters of the task routine;

transmit the multiple macros to the requesting device via the network to enable the requesting device to generate, from the multiple macros, the visualization of the first DAG to include a visual representation of each task routine of the multiple task routines, and to enable the requesting device to visually output the visualization for display, wherein;

each representation of a task routine comprises;

a task graph object comprising an identifier of the task routine;

at least one input data graph object that represents an input to the task routine, that is visually connected to the task graph object in the visualization, and that comprises a visual indication of the at least one characteristic of the input; and

at least one output data graph object that represents an output of the task routine, that is visually connected to the task graph object in the visualization, and that comprises an indication of the at least one characteristic of the output; and

to generate the visualization, the requesting device is to;

compare identifiers of data objects in the I/O parameters specified in the comments among the multiple task routines to identify each dependency between an output of one task routine and an input of another task routine; and

for each dependency identified between an output and an input of a pair of task routines of the multiple task routines, visually present a dependency marker that visually links the visual representations of the pair of task routines in the visualization;

receive a second request from the requesting device to store a second DAG based on the visualization as edited in response to at least one command, wherein;

the second DAG depicts a reuse of a portion of the first job flow depicted in the first DAG to define at least a portion of a second job flow of a second analysis routine derived from the first analysis routine; and

at least a subset of task routines to be executed in a performance of the first job flow, as indicated in the first DAG, are to be executed in a performance of the second job flow, as indicated in the second DAG;

determine whether each task routine identified in the second DAG is stored within the at least one federated area or is included in the second request;

determine whether each task data object identified in the second DAG is stored within the at least one federated area or is included in the second request; and

in response to a determination that each task routine identified in the second DAG and each data object identified in the second DAG is stored within the at least one federated area or is included in the second request, store the second DAG as a second job flow definition among the plurality of job flow definitions.

2. The apparatus of claim 1, wherein:

to identify the multiple task routines, the first request comprises a job flow identifier of a first job flow definition of the first job flow among the plurality of job flow definitions;

the first job flow definition comprises a flow task identifier of multiple flow task identifiers for each task of the multiple tasks; and

the processor is caused to perform operations comprising;

use the job flow identifier of the first request to retrieve the first job flow definition from among the plurality of job flow definitions; and

use the multiple flow task identifiers of the first job flow definition to retrieve the multiple task routines from among the plurality of task routines.

3. The apparatus of claim 2, wherein:

multiple versions of at least one task routine of the multiple task routines are stored within the at least one federated area;

use of the multiple flow task identifiers of the first job flow definition to retrieve the multiple task routines comprises retrieving the multiple versions of each task routine of the at least one task routine; and

the processor is caused to perform operations comprising, for each task routine of the at least one task routine;

parse the comments of each version of the multiple versions to identify and retrieve the specification of the I/O parameters from the comments of each version of the multiple versions;

compare the specification of I/O parameters retrieved from the comments of each version of the multiple versions; and

in response to a determination that a difference exists among the multiple versions, generate the corresponding macro to indicate the difference to enable the requesting device to visually present the difference in the visualization.

4. The apparatus of claim 1, wherein:

to identify the multiple task routines, the first request comprises an instance log identifier of an instance log that documents a performance of the multiple tasks of the first job flow;

the instance log comprises a task routine identifier for each task routine of the multiple task routines; and

the processor is caused to perform operations comprising;

store a plurality of instance logs within the at least one federated area;

use the instance log identifier of the first request to retrieve the instance log from among the plurality of instance logs; and

use the task routine identifiers of the instance log to retrieve the multiple task routines from among the plurality of task routines.

5. The apparatus of claim 4, wherein:

the instance log comprises a data object identifier for each data object used by a task routine of the multiple task routines as an input and for each data object generated by a task routine of the multiple task routines during the document performance of the multiple tasks of the first job flow; and

the processor is caused to perform operations comprising;

for each task routine of the multiple task routines for which the indications of the I/O parameters in the corresponding macro do not identify a data object as an input to the task routine, but for which the instance log provides a data object identifier of a data object used as an input to the task routine, augment the I/O parameters in the corresponding macro with the data object identifier provided by the instance log; and

for each task routine of the multiple task routines for which the indications of the I/O parameters in the corresponding macro do not identify a data object as an output of the task routine, but for which the instance log provides a data object identifier of a data object output by the task routine, augment the I/O parameters in the corresponding macro with the data object identifier provided by the instance log.

6. The apparatus of claim 4, wherein:

the instance log comprises a job flow identifier of a first job flow definition of the first job flow that comprises a flow task identifier of multiple flow task identifiers for each task of the multiple tasks; and

the processor is caused to;

use the job flow identifier to retrieve the first job flow definition from among the plurality of job flow definitions;

use the multiple flow task identifiers of the first job flow definition to retrieve at least a most recent version of each task routines of the multiple task routines from among the plurality of task routines; and

for each task routine of the multiple task routines in which the task routine is not the most recent version, augment the corresponding macro with an indication of the most recent version to enable the requesting device to indicate that the task routine visually presented in the visualization is not the most recent version.

7. The apparatus of claim 1, wherein:

the at least one federated area comprises at least one linear hierarchy of multiple federated areas;

the multiple task routines are distributed among the multiple federated areas; and

the processor is caused to augment each macro with an identifier of the federated area of the multiple federated areas in which the corresponding task routine is stored to enable the requesting device to visually present, at each visual representation of one of the multiple task routines, the identifier of the federated area of the multiple federated areas in which the corresponding task routine is stored, wherein the identifier of the federated area comprises a universal resource locator (URL).

8. The apparatus of claim 1, wherein the processor is caused to perform operations comprising:

parse the executable instructions of each task routine of the multiple task routines to identify and retrieve indications of implementation of the I/O parameters of the task routine;

compare the specification of I/O parameters retrieved from the comments to the indications of implementation of the I/O parameters of the task routine;

in response to a determination that a discrepancy exists between the specification of the I/O parameters of an input retrieved from the comments and the implementation of the I/O parameters of the input in executable instructions of the task routine, augment the corresponding macro to indicate the discrepancy to enable the requesting device to visually present the discrepancy in the visualization; and

in response to a determination that a discrepancy exists between the specification of the I/O parameters of an output retrieved from the comments and the implementation of the I/O parameters of the output in executable instructions of the task routine, augment the corresponding macro to indicate the discrepancy to enable the requesting device to visually present the discrepancy in the visualization.

9. The apparatus of claim 1, wherein the requesting device is to:

operate an input device and a display device to provide a user interface (UI) to enable receipt of commands to edit the visualization;

receive the command from the input device to change a visual indication of a dependency in the visualization between two visual representations of task routines in the visualization; and

in response to receipt of the command, perform the change specified in the command in the visual indication of a dependency in the visualization as part of the generation of the second DAG from the first DAG to generate the second job flow of the analysis routine from the first job flow of the first analysis routine.

10. The apparatus of claim 9, wherein the processor is caused to perform operations comprising:

in response to receipt of the second request, retrieve task routine identifiers for all task routines identified in the second DAG and data object identifiers for all data objects identified in the second DAG;

use the task routine identifiers retrieved from the second DAG to to make the determination of whether each task routine identified in the second DAG is stored within the at least one federated area or is included in the second request; and

use the data object identifiers retrieved from the second DAG to to make the determination of whether each task data object identified in the second DAG is stored within the at least one federated area or is included in the second request.

11. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, the computer-program product including instructions operable to cause a processor to perform operations comprising:

receive, at a portal, and from a requesting device via a network, a first request to provide objects to the requesting device to enable the requesting device to generate a visualization of a first directed acyclic graph (DAG) of multiple task routines of a first job flow of a first analysis routine, wherein;

the portal is provided on the network to control access to at least one federated area by the requesting device via the network;

the at least one federated area is maintained within one or more storage devices to store a plurality of data objects, a plurality of task routines and a plurality of job flow definitions;

each task routine of the plurality of task routines comprises comments that specify input and output (I/O) parameters that comprise at least one characteristic of an input to the task routine and at least one characteristic of an output generated during execution of the task routine;

the I/O parameters of the comments of a subset of task routines of the plurality of task routines comprises an identifier of at least one data object of the plurality of data objects;

each task routine of the multiple task routines comprises executable instructions to perform a task when executed; and

each job flow definition specifies multiple tasks of a job flow of an analysis routine to be performed by task routines of the plurality of task routines;

retrieve each task routine of the multiple task routines from the at least one federated area;

parse the comments of each task routine of the multiple task routines to identify and retrieve the specification of the I/O parameters of the task routine from the comments;

for each task routine of the multiple task routines, generate a corresponding macro of multiple macros that comprises an indication of the I/O parameters of the task routine;

transmit the multiple macros to the requesting device via the network to enable the requesting device to generate, from the multiple macros, the visualization of the first DAG to include a visual representation of each task routine of the multiple task routines, and to enable the requesting device to visually output the visualization for display, wherein;

each representation of a task routine comprises;

a task graph object comprising an identifier of the task routine;

at least one input data graph object that represents an input to the task routine, that is visually connected to the task graph object in the visualization, and that comprises a visual indication of the at least one characteristic of the input; and

at least one output data graph object that represents an output of the task routine, that is visually connected to the task graph object in the visualization, and that comprises an indication of the at least one characteristic of the output; and

to generate the visualization, the requesting device is to;

compare identifiers of data objects in the I/O parameters specified in the comments among the multiple task routines to identify each dependency between an output of one task routine and an input of another task routine; and

for each dependency identified between an output and an input of a pair of task routines of the multiple task routines, visually present a dependency marker that visually links the visual representations of the pair of task routines in the visualization;

receive a second request from the requesting device to store a second DAG based on the visualization as edited in response to at least one command, wherein;

the second DAG depicts a reuse of a portion of the first job flow depicted in the first DAG to define at least a portion of a second job flow of a second analysis routine derived from the first analysis routine; and

at least a subset of task routines to be executed in a performance of the first job flow, as indicated in the first DAG, are to be executed in a performance of the second job flow, as indicated in the second DAG;

determine whether each task routine identified in the second DAG is stored within the at least one federated area or is included in the second request;

determine whether each task data object identified in the second DAG is stored within the at least one federated area or is included in the second request; and

in response to a determination that each task routine identified in the second DAG and each data object identified in the second DAG is stored within the at least one federated area or is included in the second request, store the second DAG as a second job flow definition among the plurality of job flow definitions.

12. The computer-program product of claim 11, wherein:

to identify the multiple task routines, the first request comprises a job flow identifier of a first job flow definition of the first job flow among the plurality of job flow definitions;

the first job flow definition comprises a flow task identifier of multiple flow task identifiers for each task of the multiple tasks; and

the processor is caused to perform operations comprising;

use the job flow identifier of the first request to retrieve the first job flow definition from among the plurality of job flow definitions; and

use the multiple flow task identifiers of the first job flow definition to retrieve the multiple task routines from among the plurality of task routines.

13. The computer-program product of claim 12, wherein:

multiple versions of at least one task routine of the multiple task routines are stored within the at least one federated area;

use of the multiple flow task identifiers of the first job flow definition to retrieve the multiple task routines comprises retrieving the multiple versions of each task routine of the at least one task routine; and

the processor is caused to perform operations comprising, for each task routine of the at least one task routine;

parse the comments of each version of the multiple versions to identify and retrieve the specification of the I/O parameters from the comments of each version of the multiple versions;

compare the specification of I/O parameters retrieved from the comments of each version of the multiple versions; and

in response to a determination that a difference exists among the multiple versions, generate the corresponding macro to indicate the difference to enable the requesting device to visually present the difference in the visualization.

14. The computer-program product of claim 11, wherein:

to identify the multiple task routines, the first request comprises an instance log identifier of an instance log that documents a performance of the multiple tasks of the first job flow;

the instance log comprises a task routine identifier for each task routine of the multiple task routines; and

the processor is caused to perform operations comprising;

store a plurality of instance logs within the at least one federated area;

use the instance log identifier of the first request to retrieve the instance log from among the plurality of instance logs; and

use the task routine identifiers of the instance log to retrieve the multiple task routines from among the plurality of task routines.

15. The computer-program product of claim 14, wherein:

the instance log comprises a data object identifier for each data object used by a task routine of the multiple task routines as an input and for each data object generated by a task routine of the multiple task routines during the document performance of the multiple tasks of the first job flow; and

the processor is caused to perform operations comprising;

for each task routine of the multiple task routines for which the indications of the I/O parameters in the corresponding macro do not identify a data object as an input to the task routine, but for which the instance log provides a data object identifier of a data object used as an input to the task routine, augment the I/O parameters in the corresponding macro with the data object identifier provided by the instance log; and

for each task routine of the multiple task routines for which the indications of the I/O parameters in the corresponding macro do not identify a data object as an output of the task routine, but for which the instance log provides a data object identifier of a data object output by the task routine, augment the I/O parameters in the corresponding macro with the data object identifier provided by the instance log.

16. The computer-program product of claim 14, wherein:

the instance log comprises a job flow identifier of a first job flow definition of the first job flow that comprises a flow task identifier of multiple flow task identifiers for each task of the multiple tasks; and

the processor is caused to;

use the job flow identifier to retrieve the first job flow definition from among the plurality of job flow definitions;

use the multiple flow task identifiers of the first job flow definition to retrieve at least a most recent version of each task routines of the multiple task routines from among the plurality of task routines; and

for each task routine of the multiple task routines in which the task routine is not the most recent version, augment the corresponding macro with an indication of the most recent version to enable the requesting device to indicate that the task routine visually presented in the visualization is not the most recent version.

17. The computer-program product of claim 11, wherein:

the at least one federated area comprises at least one linear hierarchy of multiple federated areas;

the multiple task routines are distributed among the multiple federated areas; and

the processor is caused to augment each macro with an identifier of the federated area of the multiple federated areas in which the corresponding task routine is stored to enable the requesting device to visually present, at each visual representation of one of the multiple task routines, the identifier of the federated area of the multiple federated areas in which the corresponding task routine is stored, wherein the identifier of the federated area comprises a universal resource locator (URL).

18. The computer-program product of claim 11, wherein the processor is caused to perform operations comprising:

parse the executable instructions of each task routine of the multiple task routines to identify and retrieve indications of implementation of the I/O parameters of the task routine;

compare the specification of I/O parameters retrieved from the comments to the indications of implementation of the I/O parameters of the task routine;

in response to a determination that a discrepancy exists between the specification of the I/O parameters of an input retrieved from the comments and the implementation of the I/O parameters of the input in executable instructions of the task routine, augment the corresponding macro to indicate the discrepancy to enable the requesting device to visually present the discrepancy in the visualization; and

in response to a determination that a discrepancy exists between the specification of the I/O parameters of an output retrieved from the comments and the implementation of the I/O parameters of the output in executable instructions of the task routine, augment the corresponding macro to indicate the discrepancy to enable the requesting device to visually present the discrepancy in the visualization.

19. The computer-program product of claim 11, wherein the requesting device is to:

operate an input device and a display device to provide a user interface (UI) to enable receipt of commands to edit the visualization;

receive the command from the input device to change a visual indication of a dependency in the visualization between two visual representations of task routines in the visualization; and

in response to receipt of the command, perform the change specified in the command in the visual indication of a dependency in the visualization as part of the generation of the second DAG from the first DAG to generate the second job flow of the analysis routine from the first job flow of the first analysis routine.

20. The computer-program product of claim 19, wherein the processor is caused to perform operations comprising:

in response to receipt of the second request, retrieve task routine identifiers for all task routines identified in the second DAG and data object identifiers for all data objects identified in the second DAG;

use the task routine identifiers retrieved from the second DAG to to make the determination of whether each task routine identified in the second DAG is stored within the at least one federated area or is included in the second request; and

use the data object identifiers retrieved from the second DAG to to make the determination of whether each task data object identified in the second DAG is stored within the at least one federated area or is included in the second request.

21. A computer-implemented method comprising:

receiving, by a processor at a federated device, and via a portal on a network from a requesting device, a first request to provide objects to the requesting device to enable the requesting device to generate a visualization of a first directed acyclic graph (DAG) of multiple task routines of a first job flow of a first analysis routine, wherein;

the portal is provided on the network to control access to at least one federated area by the requesting device via the network;

the at least one federated area is maintained within one or more storage devices to store a plurality of data objects, a plurality of task routines and a plurality of job flow definitions;

each task routine of the plurality of task routines comprises comments that specify input and output (I/O) parameters that comprise at least one characteristic of an input to the task routine and at least one characteristic of an output generated during execution of the task routine;

the I/O parameters of the comments of a subset of task routines of the plurality of task routines comprises an identifier of at least one data object of the plurality of data objects;

each task routine of the multiple task routines comprises executable instructions to perform a task when executed; and

each job flow definition specifies multiple tasks of a job flow of an analysis routine to be performed by task routines of the plurality of task routines;

retrieving each task routine of the multiple task routines from the at least one federated area;

parsing, by the processor, the comments of each task routine of the multiple task routines to identify and retrieve the specification of the I/O parameters of the task routine from the comments;

for each task routine of the multiple task routines, generating a corresponding macro of multiple macros that comprises an indication of the I/O parameters of the task routine;

transmitting, from the federated device, the multiple macros to the requesting device via the network to enable the requesting device to generate, from the multiple macros, the visualization of the first DAG to include a visual representation of each task routine of the multiple task routines, and to enable the requesting device to visually output the visualization for display, wherein;

each representation of a task routine comprises;

a task graph object comprising an identifier of the task routine;

at least one input data graph object that represents an input to the task routine, that is visually connected to the task graph object in the visualization, and that comprises a visual indication of the at least one characteristic of the input; and

at least one output data graph object that represents an output of the task routine that is visually connected to the task graph object in the visualization, and that comprises an indication of the at least one characteristic of the output; and

to generate the visualization, the requesting device is to;

compare identifiers of data objects in the I/O parameters specified in the comments among the multiple task routines to identify each dependency between an output of one task routine and an input of another task routine; and

for each dependency identified between an output and an input of a pair of task routines of the multiple task routines, visually present a dependency marker that visually links the visual representations of the pair of task routines in the visualization;

receiving, by the processor at the federated device, a second request from the requesting device to store a second DAG based on the visualization as edited in response to at least one command, wherein;

the second DAG depicts a reuse of a portion of the first job flow depicted in the first DAG to define at least a portion of a second job flow of a second analysis routine derived from the first analysis routine; and

at least a subset of task routines to be executed in a performance of the first job flow, as indicated in the first DAG, are to be executed in a performance of the second job flow, as indicated in the second DAG;

determining, by the processor, whether each task routine identified in the second DAG is stored within the at least one federated area or is included in the second request;

determining, by the processor, whether each task data object identified in the second DAG is stored within the at least one federated area or is included in the second request; and

in response to a determination that each task routine identified in the second DAG and each data object identified in the second DAG is stored within the at least one federated area or is included in the second request, storing the second DAG as a second job flow definition among the plurality of job flow definitions.

22. The method of claim 21, wherein:

to identify the multiple task routines, the first request comprises a job flow identifier of a first job flow definition of the first job flow among the plurality of job flow definitions;

the first job flow definition comprises a flow task identifier of multiple flow task identifiers for each task of the multiple tasks; and

the method comprises;

using the job flow identifier of the first request to retrieve the first job flow definition from among the plurality of job flow definitions; and

using the multiple flow task identifiers of the first job flow definition to retrieve the multiple task routines from among the plurality of task routines.

23. The method of claim 22, wherein:

multiple versions of at least one task routine of the multiple task routines are stored within the at least one federated area;

use of the multiple flow task identifiers of the first job flow definition to retrieve the multiple task routines comprises retrieving the multiple versions of each task routine of the at least one task routine; and

the method comprises, for each task routine of the at least one task routine;

parsing, by the processor, the comments of each version of the multiple versions to identify and retrieve the specification of the I/O parameters from the comments of each version of the multiple versions;

comparing the specification of I/O parameters retrieved from the comments of each version of the multiple versions; and

in response to a determination, by the processor, that a difference exists among the multiple versions, generating the corresponding macro to indicate the difference to enable the requesting device to visually present the difference in the visualization.

24. The method of claim 21, wherein:

to identify the multiple task routines, the first request comprises an instance log identifier of an instance log that documents a performance of the multiple tasks of the first job flow;

the instance log comprises a task routine identifier for each task routine of the multiple task routines; and

the method comprises;

storing a plurality of instance logs within the at least one federated area;

using the instance log identifier of the first request to retrieve the instance log from among the plurality of instance logs; and

using the task routine identifiers of the instance log to retrieve the multiple task routines from among the plurality of task routines.

25. The method of claim 24, wherein:

the instance log comprises a data object identifier for each data object used by a task routine of the multiple task routines as an input and for each data object generated by a task routine of the multiple task routines during the document performance of the multiple tasks of the first job flow; and

the method comprises;

for each task routine of the multiple task routines for which the indications of the I/O parameters in the corresponding macro do not identify a data object as an input to the task routine, but for which the instance log provides a data object identifier of a data object used as an input to the task routine, augmenting the I/O parameters in the corresponding macro with the data object identifier provided by the instance log; and

for each task routine of the multiple task routines for which the indications of the I/O parameters in the corresponding macro do not identify a data object as an output of the task routine, but for which the instance log provides a data object identifier of a data object output by the task routine, augmenting the I/O parameters in the corresponding macro with the data object identifier provided by the instance log.

26. The method of claim 24, wherein:

the instance log comprises a job flow identifier of a first job flow definition of the first job flow that comprises a flow task identifier of multiple flow task identifiers for each task of the multiple tasks; and

the method comprises;

using the job flow identifier to retrieve the first job flow definition from among the plurality of job flow definitions;

using the multiple flow task identifiers of the first job flow definition to retrieve at least a most recent version of each task routines of the multiple task routines from among the plurality of task routines; and

for each task routine of the multiple task routines in which the task routine is not the most recent version, augmenting the corresponding macro with an indication of the most recent version to enable the requesting device to indicate that the task routine visually presented in the visualization is not the most recent version.

27. The method of claim 21, wherein:

the at least one federated area comprises at least one linear hierarchy of multiple federated areas;

the multiple task routines are distributed among the multiple federated areas; and

the method comprises augmenting each macro with an identifier of the federated area of the multiple federated areas in which the corresponding task routine is stored to enable the requesting device to visually present, at each visual representation of one of the multiple task routines, the identifier of the federated area of the multiple federated areas in which the corresponding task routine is stored, wherein the identifier of the federated area comprises a universal resource locator (URL).

28. The method of claim 21, comprising:

parsing, by the processor, the executable instructions of each task routine of the multiple task routines to identify and retrieve indications of implementation of the I/O parameters of the task routine;

comparing the specification of I/O parameters retrieved from the comments to the indications of implementation of the I/O parameters of the task routine;

in response to a determination by the processor that a discrepancy exists between the specification of the I/O parameters of an input retrieved from the comments and the implementation of the I/O parameters of the input in executable instructions of the task routine, augmenting the corresponding macro to indicate the discrepancy to enable the requesting device to visually present the discrepancy in the visualization; and

in response to a determination by the processor that a discrepancy exists between the specification of the I/O parameters of an output retrieved from the comments and the implementation of the I/O parameters of the output in executable instructions of the task routine, augmenting the corresponding macro to indicate the discrepancy to enable the requesting device to visually present the discrepancy in the visualization.

29. The method of claim 21, wherein the requesting device is to:

operate an input device and a display device to provide a user interface (UI) to enable receipt of commands to edit the visualization;

receive the command from the input device to change a visual indication of a dependency in the visualization between two visual representations of task routines in the visualization; and

in response to receipt of the command, perform the change specified in the command in the visual indication of a dependency in the visualization as part of the generation of the second DAG from the first DAG to generate the second job flow of the analysis routine from the first job flow of the first analysis routine.

30. The method of claim 29, comprising:

in response to receipt of the second request, retrieving task routine identifiers for all task routines identified in the second DAG and data object identifiers for all data objects identified in the second DAG;

using the task routine identifiers retrieved from the second DAG to to make the determination of whether each task routine identified in the second DAG is stored within the at least one federated area or is included in the second request; and

using the data object identifiers retrieved from the second DAG to to make the determination of whether each task data object identified in the second DAG is stored within the at least one federated area or is included in the second request.