How do I automate installation of Cloudera CDH on 100 Node cluster without using Cloudera manager?

[#1]

How do I install Cloudera CDH on 100 Node cluster without using Cloudera manager? Installing and configuring CDH manually on each node in a cluster is a difficult task. What tools and technologies are used to automate the task in production?

[#2]

If you are going to use CDH 5.4 and CentOS 6, you can try https://github.com/sergevs/ansible-cloudera-hadoop

Let me to know if you have questions.

Regards,

Serge.

[#3]

Thanks Sergevs for your response. Your answer is helpful if am using CDH 5.4 version. However am using CDH 4 on one cluster and CDH 5 on another cluster. Appreciate anyone's help in letting me know the overview of the tools and technologies used to automate CDH installation without Cloudera Manager.

[#4 · Accepted solution]

We're using puppet in the production:

• https://forge.puppetlabs.com/cesnet/hadoop
• https://forge.puppetlabs.com/cesnet/hive
• https://forge.puppetlabs.com/cesnet/hbase
• https://forge.puppetlabs.com/cesnet/spark
• ...

It is not "one-click" setup. But it is well described with examples.

We have cluster with 24 (+3) nodes (1 PB) and quite difficult environment (Kerberos enabled, with cross-realms, multihome network, HA enabled, ...).

[#5]

Thanks Valtri. I will have a look into it and let me know for any queries or help.