CENTRALIZED CONFIGURATION AND MONITORING OF A DISTRIBUTED COMPUTING CLUSTER

• US 20130204948A1
• Filed: 08/03/2012
• Published: 08/08/2013
• Est. Priority Date: 02/07/2012
• Status: Active Grant

• Current Assignee: Cloudera Incorporated
• Original Assignee: Cloudera Incorporated
• Inventors: Zeyliger, Philip, Langdale, Philip L., Hunt, Patrick D.
• Granted Patent: US 9,172,608 B2
• Time in Patent Office: Days
• US Class Current: 709/206
• CPC Class Codes: G06F 11/3051 Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs verification or detection of system hardware configuration G06F11/2247
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 11/328 Computer systems status display G06F11/327 takes precedence
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 9/44505 Configuring for program initiating, e.g. using registry, configuration files
• CPC Class Codes: H04L 41/0816 the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/5041 characterised by the time relationship between creation and deployment of a service
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 67/51 Discovery or management thereof, e.g. service location protocol [SLP] or web services

Abstract

Systems and methods for centralized configuration and monitoring of a distributed computing cluster are disclosed. One embodiment of the disclose technology enables deployment and central operation a complete Hadoop stack. The application automates the installation process and reduces deployment time from weeks to minutes. One embodiment further provides a cluster-wide, real time view of the services running and the status of the host machines in a cluster via a single, central place to enact configuration changes across the computing cluster which further incorporates reporting and diagnostic tools to optimize cluster performance and utilization.

Claims

1. A method of centralized configuration of a distributed computing cluster, the method, comprising:

providing a user environment enabling a selection of a service to be run on hosts in the distributed computing cluster;

wherein, the user environment further enables configuration of the service or hosts in the distributed computer cluster;

configuring each of the hosts in the distributed computing cluster to run the service based on a set of configuration settings.

2. The method of claim 1, further comprising, deploying agents to the hosts in the distributed computing cluster to configure each of the hosts.

3. The method of claim 1, further comprising, generating recommended configuration settings of the service or the hosts in the distributed computing cluster to run the service;

wherein, the recommended configuration settings include suggested ranges for parameters and invalid values for the parameters.

4. The method of claim 1, wherein, the user environment further enables, actions to be performed on the set of configuration settings, the actions including, one or more of, reading, validating, updating, and deleting.

5. The method of claim 1, wherein, the user environment further enables addition of services and reconfiguration of the service.

6. The method of claim 1, wherein, the user environment further enables assignment and re-assignment of roles to each of the hosts to indicate a function under the service that each of the hosts is to perform.

7. The method of claim 1, wherein, the user environment further enables user configuration of the hosts with functions to perform under the service.

8. The method of claim 1, wherein, the user environment further enables the selection of the service during an installation phase.

9. The method of claim 1, wherein, the user environment further displays current or historical health status of the hosts in the distributed computing cluster.

10. The method of claim 1, wherein, the user environment further displays current or historical disk usage, CPU, virtual memory consumption, or RAM usage of the hosts in the distributed computing cluster.

11. The method of claim 1, wherein, the user environment further indicates current or historical performance metrics or operational status of the hosts in the distributed computing cluster.

12. The method of claim 1, wherein, the user environment further indicates, one or more of, current or historical performance metrics of the service, a history of actions performed on the service, or a log of configuration changes of the service.

13. The method of claim 1, wherein, the user environment further displays performance metrics of a job or comparison or performance of similar jobs.

14. The method of claim 1, wherein, the user environment further indicates current user activities or historical user activities on the distributed computing cluster.

15. The method of claim 1, wherein, the user environment further displays history of activity monitoring data and configuration changes of the hosts in the distributed computing cluster.

16. The method of claim 1, wherein, the user environment further provides access to log entries associated with the service or events in the distributed computing cluster.

17. The method of claim 16, wherein, the events include, one or more of, a record of change of state of health of the server, a message has been logged, a service has been added or reconfigured, a new job has been setup, an error, a change in operational or on/off state of a given host.

18. The method of claim 16, wherein, the events include, one or more of, a health check event, a log message event, an audit event, and an activity event.

19. The method of claim 16, wherein, the events are searchable via the user environment.

20. The method of claim 17, wherein, the user environment further enables search or filtering of the log entries by one or more of, time range, service, host, keyword, and user.

21. The method of claim 17, wherein, the user environment further depicts alerts triggered by certain events or actions in the distributed computing cluster.

22. The method of claim 1, wherein, the user environment further displays operational reports.

23. The method of claim 20, wherein, the operational reports include, one or more of, disk use by user, user group, or directory, cluster job activity by user, group or job ID.

24. The method of claim 1, wherein, the user environment further enables configuration of delivery of alerts.

25. The method of claim 1, wherein, the service includes one or more Hadoop services.

26. The method of claim 3, wherein, the Hadoop services include one or more of, MapReduce, HDFS, Hue, ZooKeeper and Oozie.

27. The method of claim 1, wherein, the user environment is accessed via a web browser.

28. A system to manage and configure a computing cluster, the system, comprising:

a server, which when in operation, tracks or updates a data model with a catalog of hosts in the computing cluster;

provides a console for management and configuration of services to be deployed in the computing cluster;

configures agents to be deployed to the hosts in the computing cluster based on configuration settings;

monitors health and performance metrics of the hosts and the services;

depicts in the console, the health and the performance metrics of the hosts and the services.

29. The system of claim 28, wherein, the server further maintains a history of the health and the performance metrics.

30. The system of claim 28, wherein, the server deploys the agents to each of the hosts and communicates with the agents to send the configuration settings to configure each of the hosts in the computing cluster.

31. The system of claim 30, wherein, the agents start and stop daemons on each of the hosts to run the services.

32. The system of claim 30, wherein, the agents create directories, processes, and files on hosts in a user-specific manner.

33. The system of claim 30, wherein, the agents aggregate statistics regarding each of the hosts.

34. The system of claim 31, wherein, the server performs health calculations of the hosts based on the statistics collected by the agents.

35. The system of claim 28, wherein, the services include Hadoop services.

36. The system of claim 28, wherein, the data model is stored in a repository coupled to the server;

wherein, the data model further specifies, one or more of, services, roles, and configurations assigned to each of the hosts.

37. The system of claim 30, wherein, the data model further stores configuration or monitoring information regarding the daemons on each of the hosts.

38. A method to manage and configure a computing cluster with a Hadoop stack, the method, comprising:

configuring hosts in the computing cluster based on configuration settings and Hadoop services to be run in the computing cluster;

monitors health and performance metrics of the hosts and the Hadoop services;

computing, the health and the performance metrics of the hosts and the Hadoop services.

39. The method of claim 38, wherein, the configuration settings are specified via a console accessible via a web interface.

40. The method of claim 38, further comprising, depicting in the console, the health and the performance metrics of the hosts and the services.

41. The method of claim 38, wherein, the console includes a time selector feature allowing selection of a time range within which to view the services, activities, logs, or events.

42. The method of claim 38, further comprising, providing, via the console, enablement of a selection of a service during installation to be run on hosts in the distributed computing cluster.

43. The method of claim 38, further comprising, providing recommended configuration settings of the Hadoop service or the hosts in the computing cluster to run the service;

wherein, the recommended configuration settings include suggested ranges for parameters and invalid values for the parameters.

44. The method of claim 38, wherein, the health and the performance metrics includes current information regarding the computing cluster in real time or near real time;

wherein, the current information is accessible via a console accessible through a web interface which is also used to configure the hosts.

45. The method of claim 38, wherein, the health and the performance metrics includes historical information regarding the computing cluster;

wherein, the current information is accessible via a console accessible through a web interface which is also used to configure the hosts.

46. The method of claim 38, further comprising:

generating an alert responsive to detecting an event in the computing cluster meeting a criteria;

wherein, the alert is deliverable via email.

47. A computing cluster configured for distributed computing, the computing cluster, comprising:

multiple machines coupled in a network;

wherein, each machine is able to run one or more services for distributed computing in the computing cluster;

wherein, the configuration of the one or more services on each machine in the computing cluster is configurable via a centralized management console;

wherein, health or performance information regarding each machine is also monitored and accessed via the centralized management console.

48. The computing cluster of claim 47, wherein, the centralized management console is accessible via a web browser.

49. The computing cluster of claim 47, wherein, each of the multiple machines are configured by a server remotely coupled to the computing cluster which hosts the centralized management console.

50. The computing cluster of claim 49, wherein, logs of the services and events are accessible and searchable via the centralized management console.

51. The computing cluster of claim 49, wherein, the server computes statistics from the health and performance information and stores a history of the health and performance information for access and search via the centralized management console.

52. The computing cluster of claim 47, wherein:

roles are assigned to each of the machines to indicate a function under the service that each of the hosts is to perform.

53. The computing cluster of claim 48, wherein, the computing cluster is configured to run a distributed file system service;

wherein, via the centralized management console, one of the machines is configured to run as NameNode, one of the machines is selected to run as a Secondary NameNode, another one of the machines is selected to run as a Balance, and one or more of the remaining machines as Datanodes.

54. The computing cluster of claim 49, wherein, the distributed file system service is a Hadoop distributed file system service (HDFS).

55. The computing cluster of claim 48, wherein, the computing cluster is configured to run a parallel processing service;

wherein, via the centralized management console, one of the machines is configured to delegate tasks to select machines in the computing cluster, and configure the select machines to accept the tasks.

56. The computing cluster of claim 51, wherein, the parallel processing service includes a MapReduce service, and the one of the machines is configured to run as a JobTracker role instance and the select machines are configured to run as TaskTracker role instances.

57. The computing cluster of claim 47, wherein, the service includes one or more Hadoop services.

58. The computing cluster of claim 47, wherein, the Hadoop services include one or more of, Hbase, Hue, ZooKeeper and Oozie.

59. The computing cluster of claim 47, wherein, the Hadoop services include one or more of, Hadoop Common, Avro, Cassandra, Chukwa, Hive, Mahout, and Pig.