ADAPTIVE DATACENTER TOPOLOGY FOR DISTRIBUTED FRAMEWORKS JOB CONTROL THROUGH NETWORK AWARENESS

• US 20160140001A1
• Filed: 11/14/2014
• Published: 05/19/2016
• Est. Priority Date: 11/14/2014
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: KULKARNI, Karthik Krishna, NAMBIAR, Raghunath
• Granted Patent: US 9,785,522 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/2033 switching over of hardware resources
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3024 where the computing system component is a central processing unit [CPU]
• CPC Class Codes: G06F 11/3404 for parallel or distributed programming
• CPC Class Codes: G06F 11/3428 Benchmarking
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2209/5011 Pool
• CPC Class Codes: G06F 2209/5021 Priority
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451
• CPC Class Codes: H04L 41/147 for predicting network behaviour
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Systems, methods, and computer program products to perform an operation comprising receiving a priority of a distributed computing job, an intermediate traffic type of the distributed computing job, and a set of candidate compute nodes available to process the distributed computing job, the candidate compute nodes each available to process at least one input split of the distributed computing job, and selecting a mapper node from the candidate compute nodes, for one of the input splits, wherein the mapper node is selected based on the priority and the intermediate traffic type of the distributed computing job, wherein the mapper compute node is further selected upon determining that the mapper node is not affected by an error, and a resource utilization score for the mapper node does not exceed a utilization threshold.

Claims

1. A method, comprising:

receiving a priority of a distributed computing job, an intermediate traffic type of the distributed computing job, and a set of candidate compute nodes available to process the distributed computing job, the candidate compute nodes each available to process at least one input split of the distributed computing job; and

selecting a mapper node from the candidate compute nodes, for one of the input splits, wherein the mapper node is selected based on the priority and the intermediate traffic type of the distributed computing job, wherein the mapper compute node is further selected upon determining that;

the mapper node is not experiencing an error; and

a resource utilization score for the mapper node does not exceed a utilization threshold.

2. The method of claim 1, wherein the priority of the distributed computing job comprises a high priority job relative to other distributed computing jobs, wherein the mapper node is selected upon determining that an overall score for mapper node exceeds an overall score of each other candidate compute node.

3. The method of claim 1, wherein the error comprises at least one of a network error and an error of the mapper node, the method further comprising:

upon determining that the mapper node is experiencing by the error;

removing the mapper node from the set of candidate compute nodes; and

selecting a second mapper node, of the candidate compute nodes, as the mapper node for the first input split, wherein the second mapper node is selected based on the priority and the intermediate traffic type of the distributed computing job, wherein the second compute node is further selected upon determining that;

the second compute node is not experiencing an error; and

a resource utilization score for the second compute node does not exceed the utilization threshold.

4. The method of claim 1, wherein the resource utilization score of the mapper node is less than the utilization threshold and greater than a minimum utilization threshold, wherein the mapper node is selected as the mapper node for the first input split upon further determining at least one of:

(i) that each other candidate compute node is experiencing an error and (ii) that the resource utilization score of each of the other candidate compute nodes exceeds the utilization threshold.

5. The method of claim 1, further comprising:

determining that a second node, of the plurality of compute nodes, is not experiencing an error;

determining that a resource utilization score of the second compute node does not exceed the utilization threshold; and

selecting the second node as a reduce node for the distributed computing job.

6. The method of claim 1, wherein the resource utilization score is based on:

(i) a current level of utilization of one or more resources by the mapper node, and (ii) a forecasted level of utilization of the one or more resources by the mapper node, wherein the one or more resources comprise;

(i) CPU utilization, (ii) network utilization, (iii) memory utilization, and (iv) I/O utilization.

7. The method of claim 1, further comprising:

upon determining that a second input split for a different distributed computing job is being processed by the mapper node;

preempting the processing of the second input split by the mapper node; and

processing the first input split by the mapper node, wherein the distributed computing job has a greater priority than the different distributed computing job.

8. A system, comprising:

a computer processor; and

a memory containing a program, which when executed by the processor, performs an operation comprising;

receiving a priority of a distributed computing job, an intermediate traffic type of the distributed computing job, and a set of candidate compute nodes available to process the distributed computing job, the candidate compute nodes each available to process at least one input split of the distributed computing job; and

selecting a mapper node from the candidate compute nodes, for one of the input splits, wherein the mapper node is selected based on the priority and the intermediate traffic type of the distributed computing job, wherein the mapper compute node is further selected upon determining that;

the mapper node is not experiencing an error; and

a resource utilization score for the mapper node does not exceed a utilization threshold.

9. The system of claim 8, wherein the priority of the distributed computing job comprises a high priority job relative to other distributed computing jobs, wherein the mapper node is selected upon determining that an overall score for mapper node exceeds an overall score of each other candidate compute node.

10. The system of claim 8, wherein the error comprises at least one of a network error and an error of the mapper node, the operation further comprising:

upon determining that the mapper node is experiencing by the error;

removing the mapper node from the set of candidate compute nodes; and

selecting a second mapper node, of the candidate compute nodes, as the mapper node for the first input split, wherein the second mapper node is selected based on the priority and the intermediate traffic type of the distributed computing job, wherein the second compute node is further selected upon determining that;

the second compute node is not experiencing an error; and

a resource utilization score for the second compute node does not exceed the utilization threshold.

11. The system of claim 8, wherein the resource utilization score of the mapper node is less than the utilization threshold and greater than a minimum utilization threshold, wherein the mapper node is selected as the mapper node for the first input split upon further determining at least one of:

(i) that each other candidate compute node is experiencing an error and (ii) that the resource utilization score of each of the other candidate compute nodes exceeds the utilization threshold.

12. The system of claim 8, the operation further comprising:

determining that a second node, of the plurality of compute nodes, is not experiencing an error;

determining that a resource utilization score of the second compute node does not exceed the utilization threshold; and

selecting the second node as a reduce node for the distributed computing job.

13. The system of claim 8, wherein the resource utilization score is based on:

(i) a current level of utilization of one or more resources by the mapper node, and (ii) a forecasted level of utilization of the one or more resources by the mapper node, wherein the one or more resources comprise;

(i) CPU utilization, (ii) network utilization, (iii) memory utilization, and (iv) I/O utilization.

14. The system of claim 8, the operation further comprising:

upon determining that a second input split for a different distributed computing job is being processed by the mapper node;

preempting the processing of the second input split by the mapper node; and

processing the first input split by the mapper node, wherein the distributed computing job has a greater priority than the different distributed computing job.

15. A computer program product, comprising:

computer-readable code that when executed by a processor, performs an operation to optimize placement of a distributed computing job on a plurality of compute nodes, the operation comprising;

receiving a priority of a distributed computing job, an intermediate traffic type of the distributed computing job, and a set of candidate compute nodes available to process the distributed computing job, the candidate compute nodes each available to process at least one input split of the distributed computing job; and

selecting a mapper node from the candidate compute nodes, for one of the input splits, wherein the mapper node is selected based on the priority and the intermediate traffic type of the distributed computing job, wherein the mapper compute node is further selected upon determining that;

the mapper node is not experiencing an error; and

a resource utilization score for the mapper node does not exceed a utilization threshold.

16. The computer program product of claim 15, wherein the priority of the distributed computing job comprises a high priority job relative to other distributed computing jobs, wherein the mapper node is selected upon determining that an overall score for mapper node exceeds an overall score of each other candidate compute node.

17. The computer program product of claim 15, wherein the error comprises at least one of a network error and an error of the mapper node, the operation further comprising:

upon determining that the mapper node is experiencing by the error;

removing the mapper node from the set of candidate compute nodes; and

selecting a second mapper node, of the candidate compute nodes, as the mapper node for the first input split, wherein the second mapper node is selected based on the priority and the intermediate traffic type of the distributed computing job, wherein the second compute node is further selected upon determining that;

the second compute node is not experiencing an error; and

a resource utilization score for the second compute node does not exceed the utilization threshold.

18. The computer program product of claim 15, wherein the resource utilization score of the mapper node is less than the utilization threshold and greater than a minimum utilization threshold, wherein the mapper node is selected as the mapper node for the first input split upon further determining at least one of:

(i) that each other candidate compute node is experiencing an error and (ii) that the resource utilization score of each of the other candidate compute nodes exceeds the utilization threshold.

19. The computer program product of claim 15, the operation further comprising:

determining that a second node, of the plurality of compute nodes, is not experiencing an error;

determining that a resource utilization score of the second compute node does not exceed the utilization threshold; and

selecting the second node as a reduce node for the distributed computing job.

20. The computer program product of claim 15, wherein the resource utilization score is based on:

(i) a current level of utilization of one or more resources by the mapper node, and (ii) a forecasted level of utilization of the one or more resources by the mapper node, wherein the one or more resources comprise;

(i) CPU utilization, (ii) network utilization, (iii) memory utilization, and (iv) I/O utilization.