Some days back I installed and configured Cloudera Manager and related services on 2 nodes with CentOS. I faced some issues while installation, so I thought to share those.
My Environment Setup:
CentOS release 6.2 (Final)
Cloudera Manager 3.7
Cloudera 4.1.3 (#456)
Prerequisites for Cloudera Manager:
- Cloudera Manager is supported only on 64-bit Unix OS. Check Cloudera site for more details on supported OS.
- For Hadoop to work properly, the same version of the same operating system must be installed on all cluster hosts.
- Cluster hosts must have DNS and reverse DNS properly configured i.e. your /etc/hosts file on all nodes must be able to resolve IP and hostname. You should add IP addresses, FQDN of all nodes in hosts file. I am pasting my hosts file content as a reference:
127.0.0.1 localhost.localdomain localhostYou should copy the same hosts file to all nodes.
::1 localhost6.localdomain6 localhost6
10.16.1.125 bigdata1.localdomain bigdata1
10.16.1.126 bigdata2.localdomain bigdata2
10.16.1.127 bigdata3.localdomain bigdata3
10.16.1.128 bigdata4.localdomain bigdata4
- The Cloudera Manager Server must have SSH access to the cluster hosts when you run the installation or upgrade wizard.
- Disable SELinux (set SELinux=disabled in /etc/selinux/config).
- Make sure port 7180 is open. I generally disable iptables during installation and and configure the ports after installation is complete. You can check Configuring Ports for Cloudera Manager Free Edition for more information on required ports.
- It is advisable to set a local Cloudera repository to save network bandwidth and your time, follow this link Creating a Local Yum Repository.
Install Cloudera Manager:To install Cloudera Manager Free Edition, you will:
- Install the Cloudera Manager Server on one cluster host machine.
- Install CDH and the Cloudera Manager Agents on the other cluster host machines.
- Download cloudera-manager-installer.bin from the Cloudera Downloads page on a single node.
- Give executable permission to cloudera-manager-installer.bin:
chmod u+x cloudera-manager-installer.bin
- Run cloudera-manager-installer.bin.
- Note: To use local repositoriesrun the cloudera-manager-installer.bin with the --skip_repo_package=1 option.
sudo ./cloudera-manager-installer.bin --skip_repo_package=1
- Read and accept the licenses.
- Note the complete URL provided for the Cloudera Manager Admin Console, including the port
number, which is 7180 by default.
Start the Cloudera Manager Admin Console
- In a web browser, enter the URL, including the port, for the Cloudera Server.
- Log into Cloudera Manager. The default credentials are:
Use CM for Automated CDH Installation and Configuration:
Step1: Enter the IP address of hosts and click on Search.
Step2: Next screen will show the matched hosts and display if CDH is already running on them. Click on Install CDH on Selected Hosts.
Step3: In next screen, you can choose versions of CDH, Impala and CM which you want to install on your hosts. Please remember to use local repository which we created earlier.
Step4: In next screen, enter ssh login credentials (I used root user for installation) and click on Start Installation.
Step5: It will take some time for installation on all the nodes, you can see below screen on successful installation:
Step6: In next screen, choose the services which you want to start on your cluster:
Step7: In screens to follow you will get options to change default configuration for the services. Leave it unchanged unless you are sure of your actions. Suggest to take a note of all these values.
Step8: After all configuration CM will start the services on selected hosts:
Step9: Click on continue:
And everything is done :)
Hope this visual guide helps some readers to setup Cloudera cluster easily and more efficiently.