Hi. I have a qestion regarding hdp-conf-utils script and Ambari recommendations. I installed 8 node managers on my cluster. Node hardware spec: cores - 4, ram - 15, disk - 4. I execute hdp-conf-utils.py script and I got something like that: Using cores=4 memory=15GB disks=4 hbase=False Profile: cores=4 memory=14336MB reserved=1GB usableMem=14GB disks=4 Num Container=8 Container Ram=1792MB Used Ram=14GB Unused Ram=1GB ***** mapred-site.xml ***** mapreduce.map.memory.mb=1792 mapreduce.map.java.opts=-Xmx1280m mapreduce.reduce.memory.mb=3584 mapreduce.reduce.java.opts=-Xmx2560m mapreduce.task.io.sort.mb=640 ***** yarn-site.xml ***** yarn.scheduler.minimum-allocation-mb=1792 yarn.scheduler.maximum-allocation-mb=14336 yarn.nodemanager.resource.memory-mb=14336 yarn.app.mapreduce.am.resource.mb=1792 yarn.app.mapreduce.am.command-opts=-Xmx1280m ***** tez-site.xml ***** tez.am.resource.memory.mb=3584 tez.am.java.opts=-Xmx2560m ***** hive-site.xml ***** hive.tez.container.size=1792 hive.tez.java.opts=-Xmx1280m hive.auto.convert.join.noconditionaltask.size=402653000 I wanted to set this recommendations to YARN, but Ambari recommends me something else: yarn.nodemanager.resource.memory-mb=5120 yarn.scheduler.minimum-allocation-mb=512 yarn.scheduler.maximum-allocation-mb=5120 Can anyone explain me why Ambari and hdp-conf-utils recommend something else? I would be really grateful. ... View more

Hi. I have a question regarding Stack Advisor script. I want to tune my cluster - change YARN, Tez and Hive configuration files. I can use hdp-configuration-utils.py script, but it is too simple. I read that there is stack and service advisors which can give me recommendations. My question is - can I use stack advisor script after HDP installation or I should not use it, cause Ambari uses it during HDP installation only? And if I can use it - how can I do this? ... View more

I have setup https for Ambari. It works good, but I have a problem with redirecting. When I try http url, I see an error that page is not available. The page works fine when I try https url. Is there any way to redirect Ambari http url to https? ... View more

Hi. I am looking for some script which can help me configure my cluster based on cluster hardware. I know that there is a script hdp-configuration-utils (hdp-configuration-utils on GitHub) but it is old and I don't know if it does make any sense to use it. My question is - is there any tool or script which can help configure HDP cluster? If hdp-configuration-utils is still good option for that kind of task? ... View more