Support Questions
Find answers, ask questions, and share your expertise

Configuring Hadoop

Contributor

Can anyone provide me a link/video on how to tune and configure the hadoop like memory settings for MR, Yarn, hive, TEZ and advanced configurations (configuring high availability, zookeepers quorums, spark etc) understanding the properties based on the cluster size (computing resources, memory), understanding the architecture and other applications running along side hadoop. How to configure properties for different hadoop components. I would really appreciate if someone can guide me on understanding the advanced properties and adjusting its values according to the cluster size.

4 REPLIES 4

Re: Configuring Hadoop

@Praneender Vuppala

The short answer is that tuning a hadoop cluster is not as simple as turning a couple of knobs. The reality is there are many components within a cluster, often each of them need to be tuned independently. Additionally, tuning each component will vary widely on the usage pattern of the cluster and the respective component.

Here are a couple of articles that you may find helpful:

https://community.hortonworks.com/articles/14309/demystify-tez-tuning-step-by-step.html

https://community.hortonworks.com/articles/22419/hive-on-tez-performance-tuning-determining-reducer....

https://community.hortonworks.com/articles/46220/phoenix-hbase-tuning-quick-hits.html

https://community.hortonworks.com/articles/16763/cheat-sheet-and-tips-for-a-custom-install-of-horto....

Re: Configuring Hadoop

Contributor

@Michael Young

Thanks for the help.

Re: Configuring Hadoop

Super Guru

Re: Configuring Hadoop

Contributor

@Kuldeep Kulkarni

Thanks for your blogs