question Re: i have 6 nodes with hdp 2.1 cluster my jobs are running slow than previous . how can i improve my cluster performs in Archives of Support Questions (Read Only)

i have 6 nodes with hdp 2.1 cluster my jobs are running slow than previous . how can i improve my cluster performs

gsr_5001 — Thu, 19 May 2016 01:40:14 GMT

Re: i have 6 nodes with hdp 2.1 cluster my jobs are running slow than previous . how can i improve my cluster performs

cstanca — Thu, 19 May 2016 02:08:58 GMT

This is a very loaded question. Many things can change and impact performance. You can have suddenly more data that meets the criteria for that job processing, some other process is using more resources, your network band can be impacted by some other process or have some dropped packets. You can also have a setting that due to change in conditions could be insufficient, e.g. JVM heap, file descriptors, etc. You must analyze data provided by your monitoring tools and identify what seems to be the bottleneck: CPU, I/O, network, RAM etc. Check your Ganglia and Nagios monitoring and see what they report. In most recent versions of HDP, you could benefit of SmartSense which you may have based on your subscription level. This is an awesome tool designed by engineers in pain of administrating clusters and dealing with mis-configuration and performance issues. As a short-term approach, you should troubleshoot the current issues, address as much as you can, and then plan for an HDP upgrade to use SmartSense. Read more about SmartSense here: http://hortonworks.com/services/smartsense/

Re: i have 6 nodes with hdp 2.1 cluster my jobs are running slow than previous . how can i improve my cluster performs

cstanca — Thu, 19 May 2016 02:11:31 GMT

@sankar rao

In most recent

In most recent version of Ambari, SmartSense can be installed as any other service. If you don't have an Ambari capable to do so, you could upgrade Ambari and then install SmartSense, or install SmartSense manually. Instructions here: https://docs.hortonworks.com/HDPDocuments/SS1/SmartSense-1.1.0/bk_smartsense_admin/content/ch01s02s01.html

Re: i have 6 nodes with hdp 2.1 cluster my jobs are running slow than previous . how can i improve my cluster performs

gsr_5001 — Tue, 31 May 2016 22:13:55 GMT

Thanks for valuable information...

Re: i have 6 nodes with hdp 2.1 cluster my jobs are running slow than previous . how can i improve my cluster performs

cstanca — Tue, 31 May 2016 22:27:34 GMT

Sure. Post any interesting findings that may require feed-back. Please vote the response above, if useful.

Re: i have 6 nodes with hdp 2.1 cluster my jobs are running slow than previous . how can i improve my cluster performs

cstanca — Tue, 31 May 2016 22:30:13 GMT

@sankar rao

In case that you did not do it already, enable Ambari metrics for services of interest and see what they show. Also, check the logs for services running slow. Quite often they may show errors related to JVM settings etc.