Task scheduling on hybrid clouds using anonymization

• US 10,242,209 B2
• Filed: 08/27/2015
• Issued: 03/26/2019
• Est. Priority Date: 08/27/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Rooney, John G., Stuedi, Patrick M.
• Primary Examiner(s): Doan, Trang T
• Application Number:
  US14/837,122
• Publication Number:
  US 20170061143A1
• Time in Patent Office: 1,307 Days
• Field of Search: None
• CPC Class Codes: G06F 21/6263 during internet communication, e.g. revealing personal data from cookies
• CPC Class Codes: G06F 2209/5017 Task decomposition
• CPC Class Codes: G06F 9/5072 Grid computing

Abstract

Task scheduling in a hybrid cloud that includes a private cloud and an external cloud is performed. First a job to be performed is identified. Then, the job identified is decomposed into a set of tasks, whereby a task schedule is obtained for tasks of the set of tasks to be executed across the hybrid cloud. Next, a task to be executed in the external cloud that requires private data from the private cloud for its execution is detected from the task schedule. Finally, one or more non-anonymized portions of the private data are anonymized before execution of the detected task in the external cloud, by executing an anonymizing function from the private cloud. De-anonymization functionality may similarly be involved.

Claims

1. A computer-implemented method of task scheduling in a hybrid cloud that comprises a private cloud and an external cloud, the method comprising:

identifying, by one or more processing elements, a job to be performed;

decomposing, by the one or more processing elements, the job identified into a set of tasks, the set of tasks to be executed on the hybrid cloud;

obtaining, by the one or more processing elements, a task schedule of the execution of the set of tasks, the obtaining based on one or more constraints that certain tasks must be performed before others;

detecting, by the one or more processing elements, from the task schedule, a task to be executed in the external cloud that requires private data from the private cloud for its execution, wherein at least a portion of the private data is non-anonymized; and

anonymizing, by the one or more processing elements, the non-anonymized portion of the private data before execution of the detected task in the external cloud, by executing an anonymizing function from the private cloud.

2. The method of claim 1, wherein detecting comprises

detecting, from the set of tasks, a task to be executed in the external cloud that requires, as input, private data to be produced as an output of a task to be executed in the private cloud.

3. The method of claim 1, wherein

the anonymizing function used at anonymizing is initially stored on and executed, at least partly, in the private cloud.

4. The method of claim 1, wherein

the anonymizing function used at anonymizing is a reversible function;

and

the private cloud further comprises a de-anonymizing function designed to de-anonymize data that have been anonymized by the anonymizing function.

5. The method of claim 1, further comprising:

de-anonymizing a portion of an output of a task executed in the external cloud, by executing a de-anonymizing function from the private cloud.

6. The method of claim 5, further comprising:

detecting, from the set of tasks, a task to be executed in the private cloud that requires, as input, an output of a given task to be executed in the external cloud,

and wherein de-anonymizing comprises de-anonymizing a portion of an output of said given task upon completion of the latter.

7. The method of claim 6, wherein

the de-anonymizing function used at de-anonymizing is initially stored on and executed, at least partly, in the private cloud.

8. The method of claim 1, wherein

the set of tasks is representable as a directed acyclic graph, or DAG, of execution, whereby tasks of the set of tasks are ordered for execution across the hybrid cloud, subject to constraints that certain tasks of the set of tasks must be performed before other tasks of the set of tasks, said tasks and constraints respectively represented as vertices and edges of the DAG.

9. The method of claim 8, wherein the method further comprises:

detecting, from the DAG, a first set of edges, each edge of the first set of edges linking a task to be executed in the private cloud to a task to be subsequently executed in the external cloud;

detecting, from the DAG, a second set of edges, each edge of the second set of edges linking a task to be executed in the external cloud to a task to be subsequently executed in the private cloud;

invoking an anonymizing function from the private cloud, for at least some of the detected edges of the first set of edges; and

invoking a de-anonymizing function from the private cloud, for at least some of the detected edges of the second set of edges, to anonymize and de-anonymize data, respectively.

10. The method of claim 9, wherein

said anonymizing function and said de-anonymizing function are, each, implemented as standard features of a file system used in one or each of the private cloud and the external cloud.

11. The method of claim 1, wherein

a file system used in one or each of the private cloud and the external cloud is a Hadoop distributed file system, or HDFS.

12. The method of claim 1, wherein

decomposing the job and detecting a task to be executed in the external cloud is performed by a task scheduler in a parallel execution framework.

13. The method of claim 12, further comprising:

instructing, by the task scheduler and upon completion of the execution of the anonymizing function, to execute the detected task at a computer resource of the external cloud, the computer resource allocated by the task scheduler in accordance with the parallel execution framework.

14. The method of claim 13, wherein

the computer resource is allocated according to input data required as input by said task to be executed in the external cloud.

15. The method of claim 13, wherein

the computer resource is allocated according to a policy, the method further comprising:

accessing the policy prior to allocating the computer resource.

16. The method of claim 15, wherein

the policy is at least partly user-selectable.

17. The method of claim 1, further comprising:

selecting the anonymizing function based on a policy, prior to anonymizing.

18. The method of claim 1, further comprising:

selecting the anonymizing function based on a user input, prior to anonymizing.

19. A system of task scheduling in a hybrid cloud that comprises a private cloud and an external cloud, the system comprising:

a memory;

a processor communicatively coupled to the memory, the processor configured to perform a method comprising;

identifying, by one or more processing elements, a job to be performed;

decomposing, by the one or more processing elements, the job identified into a set of tasks, the set of tasks to be executed on the hybrid cloud;

obtaining, by the one or more processing elements, a task schedule of the execution of the set of tasks, the obtaining based on one or more constraints that certain tasks must be performed before others;

detecting, by the one or more processing elements, from the task schedule, a task to be executed in the external cloud that requires private data from the private cloud for its execution, wherein at least a portion of the private data is non-anonymized; and

anonymizing, by the one or more processing elements, the non-anonymized portion of the private data before execution of the detected task in the external cloud, by executing an anonymizing function from the private cloud.

20. A computer program product of task scheduling in a hybrid cloud that comprises a private cloud and an external cloud, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable via one or more processing elements to:

identify, by one or more processing elements, a job to be performed;

decompose, by the one or more processing elements, the job identified into a set of tasks, the set of tasks to be executed on the hybrid cloud;

obtain, by the one or more processing elements, a task schedule of the execution of the set of tasks, the obtaining based on one or more constraints that certain tasks must be performed before others;

detect, by the one or more processing elements, from the task schedule, a task to be executed in the external cloud that requires private data from the private cloud for its execution, wherein at least a portion of the private data is non-anonymized; and

anonymize, by the one or more processing elements, the non-anonymized portion of the private data before execution of the detected task in the external cloud, by executing an anonymizing function from the private cloud.