CROSS-CLOUD ORCHESTRATION OF DATA ANALYTICS FOR A PLURALITY OF RESEARCH DOMAINS

• US 20170090995A1
• Filed: 09/30/2015
• Published: 03/30/2017
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: Mitre Corporation
• Original Assignee: Mitre Corporation
• Inventors: JUBINSKI, Joseph Peter, WINDER, Ransom Kershaw, O'HANLON, Angela McIntee, GILES, Nathan Louis
• Granted Patent: US 10,360,072 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2457 with adaptation to user needs
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5038 considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration scheduling strategies G06F9/4881 and subgroups
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications

Abstract

Disclosed herein are system, apparatus, article of manufacture, method, and/or computer program product embodiments for a cross-cloud orchestration of data analytics for a plurality of research domains. An embodiment operates by receiving one or more command and control (C&C) requests to execute one or more analytic applications of a workflow. The workflow may include the one or more analytic applications for execution. The embodiment may further operate by generating one or more native access requests to execute the analytic applications at one or more analytics computing environments, and transmitting one or more native access requests to the analytics computing environments, wherein at least two native access requests are configured for different access protocol.

Claims

1. A system comprising:

at least one processor; and

a memory operatively coupled to the at least one processor, the processor configured to;

receive a command and control (C&

C) request to execute a first analytic application of a workflow, the workflow including one or more analytic applications for execution;

generate a first native access request to execute the first analytic application at a first analytics computing environment; and

transmit the first native access request to the first analytics computing environment.

2. The system of claim 1, wherein the processor is further configured to:

transmit an adapter request to an adapter component to transfer execution result data generated by the execution of the first analytic application to a knowledge datastore.

3. The system of claim 1, where in the processor is further configured to:

receive a C&

C request to execute a second analytic application of the workflow;

generate a second native access request to execute a second analytic application at a second analytics computing environment; and

transmit the second native access request to the second analytics computing environment.

4. The system of claim 3, wherein the one or more analytic applications include the first and the second analytic applications, and the first and the second analytic applications are executed by the first and the second analytic computing environment in a sequence.

5. The system of claim 3, wherein the first native process request is a different access protocol than the second access protocol request, and the first analytics computing environment is a different analytics computing environment than the second analytics computing environment.

6. The system of claim 1, wherein the C&

C request includes at least one of the execution application information identifying an analytic application, execution host information identifying an analytics computing environment, execution session information identifying the session associated with the execution of the analytic application, execution parameters information identifying one or more parameters for the analytic application to be executed, execution input information identifying input data, and/or execution output information identifying output data.

7. The system of claim 2, wherein the knowledge datastore includes an ontology datastore, the ontology datastore comprises one or more ontological elements in one or more knowledge domains, wherein at least one ontological element is representative of a definition for a term or an assertion in the one or more knowledge domains.

21. The system of claim 1, wherein the execution of the first analytic application generates execution result data for storage in a knowledge data store, the knowledge datastore including an ontology datastore, wherein the ontology datastore comprises one or more ontology data models represented as at least one of:

upper ontology data model, lower ontology data model, and cross research domain ontology data model.

22. The system of claim 21, wherein the one or more ontology data models include a first lower ontology data model, a second lower ontology data model, and an upper ontology data model, and

wherein the first lower ontology data model and the second lower ontology data model inherit from the upper ontology data model to form a cross domain ontology data model.

8. A computer implemented method, comprising:

receiving one or more command and control (C&

C) requests to execute one or more analytic applications of a workflow, the workflow including the one or more analytic applications for execution;

generating one or more native access requests to execute the one or more analytic applications at one or more analytics computing environments; and

transmitting one or more native access requests to the one or more analytics computing environments, wherein at least two native access requests are configured for different access protocols.

9. The computer implemented method of claim 8, further comprising:

transmitting one or more adapter requests to an adapter component to transfer execution result data generated by execution of the one or more analytic applications to a knowledge datastore, the knowledge datastore comprising a provenance datastore.

10. The computer implemented method of claim 8, further comprising:

transmitting one or more storage requests to an storage system to transfer immutable source data to the one or more analytics computing environments.

11. The computer implemented method of claim 9, wherein each analytic computing environment is configured to generate provenance class information for each execution of the one or more analytic application based at least partially on one or more of execution application information, execution host information, execution parameters information, execution input information, execution output information associated with each analytic application, and transmit the provenance class information to the provenance datastore.

12. The computer implemented method of claim 11, wherein

the knowledge datastore includes a provenance datastore for storing provenance class information for the one or more analytic applications, and

the provenance class information includes an aggregate of provenance instance information.

13. The computer implemented method of claim 10, wherein each analytics computing environment is configured to transmit the generated provenance class information to the provenance datastore after completing execution of analytic applications in the workflow.

14. The computer implemented method of claim 11, wherein the provenance instance information includes one or more of provenance application information identifying an analytic application, provenance application version information identifying a version of the analytic application, provenance timestamp information identifying time and date of the analytic application execution, provenance parameters information identifying execution parameters of the analytic application, provenance execution inputs information identifying the execution inputs of the analytic application, and provenance execution outputs information identifying the execution outputs of the analytic application.

15. A non-transitory tangible computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:

receiving a command and control (C&

C) request to execute a first analytic application of a workflow, the workflow including one or more analytic applications for execution;

generating a first native access request to execute the first analytic application at a first analytics computing environment; and

transmitting the first native access request to the first analytics computing environment.

16. The computer-readable device of claim 15, the operations further comprising:

transmitting a storage request to a storage system to transfer immutable source data to the first analytic computing environment; and

transmitting an adapter request to an adapter component to transfer execution result data generated by the execution of the first analytic application to a knowledge datastore.

17. The computer-readable device of claim 16, the operations further comprising:

receiving a C&

C request to execute a second analytic application of the workflow;

generating a second native access request to execute a second analytic application at a second analytics computing environment; and

transmitting the second native access request to the second analytics computing environment.

18. The computer-readable device of claim 17, the operations further comprising:

transmitting an adapter request to the adapter component to transfer execution result data generated by the execution of the second analytic application to the knowledge datastore.

19. The computer-readable device of claim 18, wherein the knowledge datastore includes an ontology datastore, the ontology datastore includes one or more ontological elements associated with one or more knowledge domains, and at least one ontological element is representative of a definition for a term or an assertion in the one or more knowledge domains.

20. The computer-readable device of claim 19, wherein at least one ontological element added to the ontology datastore for a knowledge domain includes an ontological element determined based on the execution result data generated by the execution of the second analytic application.