now we install again ( re-install ) , from ambari GUI ( upgrade ) but we get now the following down its very strange because we set in the ambari GUI the repo for HDP , SDP-GPL and HDP-UTILS and also the 3 repo also exists on each machine what is unclear is that when we try to install the pkg on the machine /usr/bin/yum -d 0 -e 0 -y install '--disablerepo=*' --enablerepo=HDP-2.6-GPL-repo-53,HDP-2.6-repo-53,HDP-UTILS-1.1.0.22-repo-53 hdp-select Package hdp-select-2.6.4.0-91.noarch already installed and latest version then its tell us that pkg already exists but from the log seems that he not found the pkg - hdp-select how it can be? <br>2018-02-26 13:59:25,315 - Will install packages for repository version 2.6.4.0 2018-02-26 13:59:25,316 - Repository['HDP-2.6-repo-53'] {'append_to_file': False, 'base_url': 'http://98.45.232.167/HDP/centos7/2.6.4.0-91', 'action': ['create'], 'components': [u'HDP', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-53', 'mirror_list': None} 2018-02-26 13:59:25,333 - File['/etc/yum.repos.d/ambari-hdp-53.repo'] {'content': '[HDP-2.6-repo-53]\nname=HDP-2.6-repo-53\nbaseurl=http://98.45.232.167/HDP/centos7/2.6.4.0-91\n\npath=/\nenabled=1\ngpgcheck=0'} 2018-02-26 13:59:25,334 - Writing File['/etc/yum.repos.d/ambari-hdp-53.repo'] because contents don't match 2018-02-26 13:59:25,335 - Repository['HDP-2.6-GPL-repo-53'] {'append_to_file': True, 'base_url': 'http://98.45.232.167/HDP-GPL/centos7/2.6.4.0-91', 'action': ['create'], 'components': [u'HDP-GPL', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-53', 'mirror_list': None} 2018-02-26 13:59:25,343 - File['/etc/yum.repos.d/ambari-hdp-53.repo'] {'content': '[HDP-2.6-repo-53]\nname=HDP-2.6-repo-53\nbaseurl=http://98.45.232.167/HDP/centos7/2.6.4.0-91\n\npath=/\nenabled=1\ngpgcheck=0\n[HDP-2.6-GPL-repo-53]\nname=HDP-2.6-GPL-repo-53\nbaseurl=http://98.45.232.167/HDP-GPL/centos7/2.6.4.0-91\n\npath=/\nenabled=1\ngpgcheck=0'} 2018-02-26 13:59:25,343 - Writing File['/etc/yum.repos.d/ambari-hdp-53.repo'] because contents don't match 2018-02-26 13:59:25,344 - Repository['HDP-UTILS-1.1.0.22-repo-53'] {'append_to_file': True, 'base_url': 'http://98.45.232.167/', 'action': ['create'], 'components': [u'HDP-UTILS', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-53', 'mirror_list': None} 2018-02-26 13:59:25,352 - File['/etc/yum.repos.d/ambari-hdp-53.repo'] {'content': '[HDP-2.6-repo-53]\nname=HDP-2.6-repo-53\nbaseurl=http://98.45.232.167/HDP/centos7/2.6.4.0-91\n\npath=/\nenabled=1\ngpgcheck=0\n[HDP-2.6-GPL-repo-53]\nname=HDP-2.6-GPL-repo-53\nbaseurl=http://98.45.232.167/HDP-GPL/centos7/2.6.4.0-91\n\npath=/\nenabled=1\ngpgcheck=0\n[HDP-UTILS-1.1.0.22-repo-53]\nname=HDP-UTILS-1.1.0.22-repo-53\nbaseurl=http://98.45.232.167/\n\npath=/\nenabled=1\ngpgcheck=0'} 2018-02-26 13:59:25,353 - Writing File['/etc/yum.repos.d/ambari-hdp-53.repo'] because contents don't match 2018-02-26 13:59:25,354 - Yum non-completed transactions check passed 2018-02-26 13:59:25,354 - call[('ambari-python-wrap', u'/usr/bin/hdp-select', 'versions')] {} 2018-02-26 13:59:25,426 - call returned (0, '2.6.0.3-8\n2.6.4.0-91') 2018-02-26 13:59:25,427 - Package['hdp-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5, 'use_repos': {u'HDP-2.6-repo-53': u'ambari-hdp-53', u'HDP-2.6-GPL-repo-53': u'ambari-hdp-53', u'HDP-UTILS-1.1.0.22-repo-53': u'ambari-hdp-53'}, 'action': ['upgrade']} 2018-02-26 13:59:25,428 - Installing package hdp-select ('/usr/bin/yum -d 0 -e 0 -y install '--disablerepo=*' --enablerepo=HDP-2.6-GPL-repo-53,HDP-2.6-repo-53,HDP-UTILS-1.1.0.22-repo-53 hdp-select') 2018-02-26 13:59:27,565 - Looking for matching packages in the following repositories: HDP-2.6-repo-53, HDP-2.6-GPL-repo-53, HDP-UTILS-1.1.0.22-repo-53 2018-02-26 13:59:35,337 - No package found for hadoop_${stack_version}(hadoop_(\d|_)+$)<br> <br> ... View more

<br> @Jay thank you so much for the solution just one question - can we be sure that set the folders to be under /var and not under /var/run , not Damage to ambari cluster functionality ? first I will change the code in the ambari Kit in order to update ambari.properties file per your great recommendation so bootstrap.dir , pid.dir , recommendations.dir will be under /var and not under /var/run (this is for scratch installation) # grep run /etc/ambari-server/conf/ambari.properties bootstrap.dir=/var/ambari-server/bootstrap pid.dir=/var/ambari-server recommendations.dir=/var/ambari-server/stack-recommendations<br> in order to fix the current ambari cluster we will change ambari.properties the same as below ( scratch installation ) , and we will run ambari setup to recreate the missing folders please just approve my steps ... View more

yes , but only change the owner back to ambari is a partial solution , because we have the missing folder - stack-recommendations and bootstrap , so need to find the complete solution how to recreate this folders ... View more

just to be clear , according to your answer we understand that MTU=9000 recomended to be set on all machines in the cluster include master machines . workers machines and kafka machines , am I corecyt ? ... View more