Automated transfer of objects among federated areas

• US 10,095,552 B2
• Filed: 02/14/2018
• Issued: 10/09/2018
• Est. Priority Date: 02/05/2016
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Bequet, Henry Gabriel Victor, Chen, Huina
• Primary Examiner(s): PATEL, HIREN P
• Application Number:
  US15/896,727
• Publication Number:
  US 20180181445A1
• Time in Patent Office: 237 Days
• Field of Search: None
• CPC Class Codes: G06F 16/9014 hash tables
• CPC Class Codes: G06F 16/90344 by using string matching techniques
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound

Abstract

An apparatus includes a processor to: receive, from a first remote device, a request to perform at least one iteration of a first job flow at least partly within a first federated area, wherein access to the first federated area is granted to the first remote device and not a second remote device, access to a second federated area is granted to the second remote device and not the first remote device, and a transfer area is maintained to transfer an object between the first and second federated areas; perform the at least one iteration of the first job flow; and analyze an output object generated in each iteration to determine whether a condition has been met to transfer an object from the first federated area to the transfer area to enable its transfer to the second federated area to enable its use in a second job flow.

Claims

1. An apparatus comprising a first processor and a first storage to store first instructions that, when executed by the first processor, cause the first processor to perform operations comprising:

receive, at a portal and from a first remote device via a network, a first request to perform at least one iteration of a first job flow at least partly within a first federated area, wherein;

the portal is provided on the network to control access to federated areas by remote devices via the network;

the first federated area of a set of related federated areas is maintained within one or more storage devices to store multiple objects comprising at least one of multiple data sets, multiple job flow definitions, multiple task routines, multiple result reports and multiple instance logs;

access via the network to the first federated area is granted to the first remote device and is denied to a second remote device;

a second federated area of the set of related federated areas is maintained within the one or more storage devices to store multiple objects comprising at least one of multiple data sets, multiple job flow definitions, multiple task routines, multiple result reports and multiple instance logs;

access via the network to the second federated area is granted to the second remote device and is denied to the first remote device;

a transfer area is maintained within the one or more storage devices to support a transfer of at least a first transfer object between the first federated area and the second federated area; and

the transfer area is instantiated within a storage area within the one or more storage devices at which the first federated area and the second federated area overlap, or the transfer area is instantiated within a base federated area of a hierarchical tree of federated area in which each of the first federated area and the second federated area branch from the base federated area;

perform the at least one iteration of the first job flow at least partly within the first federated area, wherein;

each iteration of performance of the first job flow comprises an iteration of execution, by the first processor, of instructions of a first task routine to perform a first task of the first job flow; and

each iteration of execution of the instructions of the first task routine generates an iteration of a first result report;

in response to each iteration of performance of the first job flow, analyze at least one indication of the first result report of the performance of the first task routine to determine, by the first processor, whether a first predetermined condition has been met through the at least one iteration of the first job flow;

in response to the first predetermined condition having been met, transfer the first task routine as the first transfer object from the first federated area to the transfer area to enable the first task routine to be transferred to the second federated area to enable a second job flow to be performed with the first task routine at least partly within the second federated area to determine whether second predetermined condition associated with an extent to which the first task routine performs the first task has been met, wherein, in response to each iteration of performance of the second job flow, an iteration of a second result report that is output during the iteration of performance of the second job flow is analyzed to determine whether the second predetermined condition has been met; and

in response to at least a determination that the second predetermined condition has been met, provide access, to other devices via the network, to the first task routine to enable use of the first task routine by the other devices to perform the first task.

2. The apparatus of claim 1, comprising a second processor and a second storage to store second instructions that, when executed by the second processor, cause the second processor to perform operations comprising:

monitor the transfer area for the transfer of the first task routine into the transfer area;

in response to the transfer of the first task routine into the transfer area, transfer the first task routine into the second federated area; and

in response to the transfer of the first task routine into the second federated area, and in response to a receipt of a second request at the portal, and from the second remote device via the network, to perform at least one iteration of the second job flow at least partly within the second federated area, perform at least one iteration of the second job flow at least partly within the second federated area, wherein;

each iteration of performance of the second job flow comprises an iteration of execution, by the second processor, of instructions of a second task routine to perform a second task of the second job flow.

3. The apparatus of claim 2, wherein:

each iteration of execution of the instructions of the second task routine generates an iteration of the second result report; and

the second processor is caused to perform operations comprising;

in response to each iteration of performance of the second job flow, analyze the second result report to determine, by the second processor, whether the second predetermined condition has been met through the at least one iteration of the second job flow; and

in response to the second predetermined condition having not been met, transfer a second transfer object from the second federated area to the transfer area to enable the second transfer object to be transferred to the first federated area.

4. The apparatus of claim 1, wherein:

the set of federated areas comprises the hierarchical tree of federated areas;

the tree comprises the base federated area at a root position of the tree;

the first federated area is positioned within a first branch of the tree;

the second federated area is positioned within a second branch of the tree separate from the first branch; and

the first processor is caused to perform operations comprising;

maintain a first inheritance relationship between the first federated area and the base federated area in which the first processor is caused to make a base object stored within the base federated area as accessible to the first remote device as a first object stored within the first federated area; and

maintain a first priority relationship between the first federated area and the base federated area as an exception to the first inheritance relationship in which the first processor is caused to;

search within the first federated area for a first required object that is required to enable the performance of the first job flow;

in response to locating the first required object within the first federated area, refrain from searching the base federated area for the first required object; and

in response to failing to locate the first required object within the first federated area, search within the base federated area for the first required object.

5. The apparatus of claim 4, wherein:

the transfer area is instantiated in the base federated area; and

aspects of at least the transfer of the first transfer object from the first federated area to the second federated area through the transfer area are defined by a transfer flow definition stored in the base federated area.

6. The apparatus of claim 4, comprising a second processor and a second storage to store second instructions that, when executed by the second processor, cause the second processor to perform operations comprising:

maintain a second inheritance relationship between the second federated area and the base federated area in which the second processor is caused to make a base object stored within the base federated area as accessible to the second remote device as a second object stored within the second federated area; and

maintain a second priority relationship between the second federated area and the base federated area as an exception to the second inheritance relationship in which the second processor is caused to;

search within the second federated area for a second required object that is required to enable the performance of the second job flow;

in response to locating the second required object within the second federated area, refrain from searching the base federated area for the second required object; and

in response to failing to locate the second required object within the second federated area, search within the base federated area for the second required object.

7. The apparatus of claim 6, wherein:

the first task routine is stored in one of the first federated area and the base federated area; and

a second task routine comprising instructions that are executed in each iteration of performance of the second job flow is stored in one of the second federated area and the base federated area.

8. The apparatus of claim 1, wherein the first processor is caused to perform operations comprising:

receive, at the portal and from one of the first remote device and the second remote device, a third request to instantiate the transfer area;

examine a first security credential associated with the first remote device to determine whether the third request is authorized for the first federated area;

examine a second security credential associated with the second remote device to determine whether the third request is authorized for the second federated area;

in response to a determination that the third request is authorized by the first remote device, and in response to a determination that the third request is authorized by the second remote device, the first processor is caused to perform operations comprising;

define the storage area within the one or more storage devices at which the first federated area and the second federated area overlap;

instantiate the transfer area within the storage area;

store, within the first federated area, a first transfer routine comprising instructions to cause the analysis of the first result report and the transfer of the first task routine from the first federated area to the transfer area; and

store, within the second federated area, a second transfer routine comprising instructions to cause monitoring of the transfer area and the transfer of the first task routine from the transfer area into the second federated area.

9. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, the computer-program product including first instructions operable to cause a first processor to perform operations comprising:

receive, at a portal and from a first remote device via a network, a first request to perform at least one iteration of a first job flow at least partly within a first federated area, wherein;

the portal is provided on the network to control access to federated areas by remote devices via the network;

the first federated area of a set of related federated areas is maintained within one or more storage devices to store multiple objects comprising at least one of multiple data sets, multiple job flow definitions, multiple task routines, multiple result reports and multiple instance logs;

access via the network to the first federated area is granted to the first remote device and is denied to a second remote device;

a second federated area of the set of related federated areas is maintained within the one or more storage devices to store multiple objects comprising at least one of multiple data sets, multiple job flow definitions, multiple task routines, multiple result reports and multiple instance logs;

access via the network to the second federated area is granted to the second remote device and is denied to the first remote device;

a transfer area is maintained within the one or more storage devices to support a transfer of at least a first transfer object between the first federated area and the second federated area; and

the transfer area is instantiated within a storage area within the one or more storage devices at which the first federated area and the second federated area overlap, or the transfer area is instantiated within a base federated area of a hierarchical tree of federated area in which each of the first federated area and the second federated area branch from the base federated area;

perform the at least one iteration of the first job flow at least partly within the first federated area, wherein;

each iteration of performance of the first job flow comprises an iteration of execution, by the first processor, of instructions of a first task routine to perform a first task of the first job flow; and

each iteration of execution of the instructions of the first task routine generates an iteration of a first result report;

in response to each iteration of performance of the first job flow, analyze at least one indication of the first result report of the performance of the first task routine to determine, by the first processor, whether a first predetermined condition has been met through the at least one iteration of the first job flow;

in response to the first predetermined condition having been met, transfer the first task routine as the first transfer object from the first federated area to the transfer area to enable the first task routine to be transferred to the second federated area to enable a second job flow to be performed with the first task routine at least partly within the second federated area to determine whether second predetermined condition associated with an extent to which the first task routine performs the first task has been met, wherein, in response to each iteration of performance of the second job flow, an iteration of a second result report that is output during the iteration of performance of the second job flow is analyzed to determine whether the second predetermined condition has been met; and

in response to at least a determination that the second predetermined condition has been met, provide access, to other devices via the network, to the first task routine to enable use of the first task routine by the other devices to perform the first task.

10. The computer-program product of claim 9, the computer-program product including second instructions operable to cause a second processor to perform operations comprising:

monitor the transfer area for the transfer of the first task routine into the transfer area;

in response to the transfer of the first task routine into the transfer area, transfer the first transfer object task routine into the second federated area; and

in response to the transfer of the first task routine into the second federated area, and in response to a receipt of a second request at the portal, and from the second remote device via the network, to perform at least one iteration of the second job flow at least partly within the second federated area, perform at least one iteration of the second job flow at least partly within the second federated area, wherein;

each iteration of performance of the second job flow comprises an iteration of execution, by the second processor, of instructions of a second task routine to perform a second task of the second job flow.

11. The computer-program product of claim 10, wherein:

each iteration of execution of the instructions of the second task routine generates an iteration of the second result report; and

the second processor is caused to perform operations comprising;

in response to each iteration of performance of the second job flow, analyze the second result report to determine, by the second processor, whether the second predetermined condition has been met through the at least one iteration of the second job flow; and

in response to the second predetermined condition having not been met, transfer a second transfer object from the second federated area to the transfer area to enable the second transfer object to be transferred to the first federated area.

12. The computer-program product of claim 9, wherein:

the set of federated areas comprises the hierarchical tree of federated areas;

the tree comprises the base federated area at a root position of the tree;

the first federated area is positioned within a first branch of the tree;

the second federated area is positioned within a second branch of the tree separate from the first branch; and

the first processor is caused to perform operations comprising;

maintain a first inheritance relationship between the first federated area and the base federated area in which the first processor is caused to make a base object stored within the base federated area as accessible to the first remote device as a first object stored within the first federated area; and

maintain a first priority relationship between the first federated area and the base federated area as an exception to the first inheritance relationship in which the first processor is caused to;

search within the first federated area for a first required object that is required to enable the performance of the first job flow;

in response to locating the first required object within the first federated area, refrain from searching the base federated area for the first required object; and

in response to failing to locate the first required object within the first federated area, search within the base federated area for the first required object.

13. The computer-program product of claim 12, wherein:

the transfer area is instantiated in the base federated area; and

aspects of at least the transfer of the first transfer object from the first federated area to the second federated area through the transfer area are defined by a transfer flow definition stored in the base federated area.

14. The computer-program product of claim 12, the computer-program product including second instructions operable to cause a second processor to perform operations comprising:

maintain a second inheritance relationship between the second federated area and the base federated area in which the second processor is caused to make a base object stored within the base federated area as accessible to the second remote device as a second object stored within the second federated area; and

maintain a second priority relationship between the second federated area and the base federated area as an exception to the second inheritance relationship in which the second processor is caused to;

search within the second federated area for a second required object that is required to enable the performance of the second job flow;

in response to locating the second required object within the second federated area, refrain from searching the base federated area for the second required object; and

in response to failing to locate the second required object within the second federated area, search within the base federated area for the second required object.

15. The computer-program product of claim 14, wherein:

the first task routine is stored in one of the first federated area and the base federated area; and

a second task routine comprising instructions that are executed in each iteration of performance of the second job flow is stored in one of the second federated area and the base federated area.

16. The computer-program product of claim 9, wherein the first processor is caused to perform operations comprising:

receive, at the portal and from one of the first remote device and the second remote device, a third request to instantiate the transfer area;

examine a first security credential associated with the first remote device to determine whether the third request is authorized for the first federated area;

examine a second security credential associated with the second remote device to determine whether the third request is authorized for the second federated area;

in response to a determination that the third request is authorized by the first remote device, and in response to a determination that the third request is authorized by the second remote device, the first processor is caused to perform operations comprising;

define the storage area within the one or more storage devices at which the first federated area and the second federated area overlap;

instantiate the transfer area within the storage area;

store, within the first federated area, a first transfer routine comprising instructions to cause the analysis of the first result report and the transfer of the first task routine from the first federated area to the transfer area; and

store, within the second federated area, a second transfer routine comprising instructions to cause monitoring of the transfer area and the transfer of the first task routine from the transfer area into the second federated area.

17. A computer-implemented method comprising:

receiving, at a server by a first processor, and via a portal on a network from a first remote device, a first request to perform at least one iteration of a first job flow at least partly within a first federated area, wherein;

the portal is provided on the network to control access to federated areas by remote devices via the network;

the first federated area of a set of related federated areas is maintained within one or more storage devices to store multiple objects comprising at least one of multiple data sets, multiple job flow definitions, multiple task routines, multiple result reports and multiple instance logs;

access via the network to the first federated area is granted to the first remote device and is denied to a second remote device;

a second federated area of the set of related federated areas is maintained within the one or more storage devices to store multiple objects comprising at least one of multiple data sets, multiple job flow definitions, multiple task routines, multiple result reports and multiple instance logs;

access via the network to the second federated area is granted to the second remote device and is denied to the first remote device;

a transfer area is maintained within the one or more storage devices to support a transfer of at least a first transfer object between the first federated area and the second federated area; and

the transfer area is instantiated within a storage area within the one or more storage devices at which the first federated area and the second federated area overlap, or the transfer area is instantiated within a base federated area of a hierarchical tree of federated area in which each of the first federated area and the second federated area branch from the base federated area;

performing the at least one iteration of the first job flow at least partly within the first federated area, wherein;

each iteration of performance of the first job flow comprises an iteration of executing, by the first processor, instructions of a first task routine to perform a first task of the first job flow; and

each iteration of execution of the instructions of the first task routine generates an iteration of a first result report;

in response to each iteration of performance of the first job flow, analyzing at least one indication of the first result report of the performance of the first task routine to determine, by the first processor, whether a first predetermined condition has been met through the at least one iteration of the first job flow;

in response to the first predetermined condition having been met, transferring the first task routine as the first transfer object from the first federated area to the transfer area to enable the first task routine to be transferred to the second federated area to enable a second job flow to be performed with the first task routine at least partly within the second federated area to determine whether second predetermined condition associated with an extent to which the first task routine performs the first task has been met, wherein, in response to each iteration of performance of the second job flow, an iteration of a second result report that is output during the iteration of performance of the second job flow is analyzed to determine whether the second predetermined condition has been met; and

in response to at least a determination that the second predetermined condition has been met, providing access, to other devices via the network, to the first task routine to enable use of the first task routine by the other devices to perform the first task.

18. The computer-implemented method of claim 17, comprising:

monitoring the transfer area for the transfer of the first task routine into the transfer area;

in response to the transfer of the first task routine into the transfer area, transferring the first task routine into the second federated area; and

in response to the transfer of the first task routine into the second federated area, and in response to a receipt of a second request at the portal, and from the second remote device via the network, to perform at least one iteration of the second job flow at least partly within the second federated area, performing at least one iteration of the second job flow at least partly within the second federated area, wherein;

each iteration of performance of the second job flow comprises an iteration of executing, by a second processor, instructions of a second task routine to perform a second task of the second job flow.

19. The computer-implemented method of claim 18, wherein:

each iteration of execution of the instructions of the second task routine comprises generating an iteration of the second result report; and

the method comprises;

in response to each iteration of performance of the second job flow, analyzing the second result report to determine, by the second processor, whether the second predetermined condition has been met through the at least one iteration of the second job flow; and

in response to the second predetermined condition having not been met, transferring a second transfer object from the second federated area to the transfer area to enable the second transfer object to be transferred to the first federated area.

20. The computer-implemented method of claim 17, wherein:

the set of federated areas comprises the hierarchical tree of federated areas;

the tree comprises the base federated area at a root position of the tree;

the first federated area is positioned within a first branch of the tree;

the second federated area is positioned within a second branch of the tree separate from the first branch; and

the method comprises;

maintaining a first inheritance relationship between the first federated area and the base federated area by making a base object stored within the base federated area as accessible to the first remote device as a first object stored within the first federated area; and

maintaining a first priority relationship between the first federated area and the base federated area as an exception to the first inheritance relationship by performing operations comprising;

searching within the first federated area for a first required object that is required to enable the performance of the first job flow;

in response to locating the first required object within the first federated area, refraining from searching the base federated area for the first required object; and

in response to failing to locate the first required object within the first federated area, searching within the base federated area for the first required object.

21. The computer-implemented method of claim 20, wherein:

the transfer area is instantiated in the base federated area; and

aspects of at least the transfer of the first transfer object from the first federated area to the second federated area through the transfer area are defined by a transfer flow definition stored in the base federated area.

22. The computer-implemented method of claim 20, comprising:

maintaining a second inheritance relationship between the second federated area and the base federated area by making a base object stored within the base federated area as accessible to the second remote device as a second object stored within the second federated area; and

maintaining a second priority relationship between the second federated area and the base federated area as an exception to the second inheritance relationship by performing operations comprising;

searching within the second federated area for a second required object that is required to enable the performance of the second job flow;

in response to locating the second required object within the second federated area, refraining from searching the base federated area for the second required object; and

in response to failing to locate the second required object within the second federated area, searching within the base federated area for the second required object.

23. The computer-implemented method of claim 22, wherein:

the first task routine is stored in one of the first federated area and the base federated area; and

a second task routine comprising instructions that are executed in each iteration of performance of the second job flow is stored in one of the second federated area and the base federated area.

24. The computer-implemented method of claim 17, comprising:

receiving, at the server by the first processor, and via the portal on the network from the second remote device, a third request to instantiate the transfer area;

examining a first security credential associated with the first remote device to determine whether the third request is authorized for the first federated area;

examining a second security credential associated with the second remote device to determine whether the third request is authorized for the second federated area;

in response to a determination that the third request is authorized by the first remote device, and in response to a determination that the third request is authorized by the second remote device, performing operations comprising;

defining the storage area within the one or more storage devices at which the first federated area and the second federated area overlap;

instantiating the transfer area within the storage area;

storing, within the first federated area, a first transfer routine comprising instructions to cause the analysis of the first result report and the transfer of the first task routine from the first federated area to the transfer area; and

storing, within the second federated area, a second transfer routine comprising instructions to cause monitoring of the transfer area and the transfer of the first task routine from the transfer area into the second federated area.