@Michael Bronson Thank you for appreciation and i apologies for making this thread a bit confusing in few of my previous update 😉 ... View more

@Michael Bronson Yes, if ambari does not create them and we want to run the backup command then we can create the directories manually. It should be Ok. ... View more

@Michael Bronson Yes, that is correct that the "ambar-server backup" utility will check if the "stack-recommendation" directory exists or not? If this directory does not exist then we will need to manually create that directory so that it passes the check. That is a workaround. https://github.com/apache/ambari/blob/release-2.6.1/ambari-server/src/main/python/ambari_server/BackupRestore.py#L131-L139 the following 4 directories it checks: # The list of files where the ambari server state is kept on the filesystem AMBARI_FILESYSTEM_STATE = [AmbariPath.get("/etc/ambari-server/conf"), AmbariPath.get("/var/lib/ambari-server/resources"), AmbariPath.get("/var/run/ambari-server/bootstrap/"), AmbariPath.get("/var/run/ambari-server/stack-recommendations")] . In normal cases when the cluster is running for quite some time and some configuration changes have been made to the cluster then definitely we will see some contents inside this directory. even when the ambari is running as non root user. ... View more

@Michael Bronson What i wanted to say is that if we have setup the "pid.dir" , "bootstrap.dir" , "recommendations.dir" outside the "tmpfs" mount and the non root ambari user has proper read/write permission to these directories then we do not need to worry about these directoies anymore. Whenever we make any changes to these directories only once we will need to run the "ambari-server setup" command just to ensure that ambari creates some of these directories (not necesarily all immediately) like "recommendations.dir" ambari might create later as well. Ambari will create those directories on it's own. We do not need to run "ambari-serevr setup" command many times. . ... View more

@Anurag Mishra If this also does not work then please try this: On the ambari server host regenerate the ssh keys exactly as following: 1. Move the old SSH keys directory (take backup as following) # mv ~/.ssh ~/.ssh_OLD 2. Regenerate the SSH keys on ambari server host without entering any passphrase. NOTE : Just press enter when it asks for passphrase (please do not enter any value there) # ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Created directory '/root/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: . 3. Now from ambari server host check if you are able to do SSH with password or not? If ssh works then try doing the passwordless ssh using: # ssh-copy-id -i ~/.ssh/id_rsa.pub root@newhost.example.com . ... View more

@Anurag Mishra Please do the following check? Then we will check about Password less ssh. 1. Please check if SSH itself is possible from AmbariServer host to NewHost or not? # ssh root@<host_fqdn> 2. If above is not working then you will need to see if the new host has the proper SSH settings and it allows the password based authentication? So on the new node please check if the "PasswordAuthentication" is set as "yes" and the "PermitRootLogin" is set to "yes" # grep 'PasswordAuthentication' /etc/ssh/sshd_config PasswordAuthentication yes # grep 'Root' /etc/ssh/sshd_config PermitRootLogin yes . If these values are not set to "yes" then please set them and then restart the SSHD service on the new host. # systemctl restart sshd (OR) # service sshd restart<br> . 3. Now from ambari server host check if you are able to do SSH with password or not? If ssh works then try doing the passwordless ssh using: # ssh-copy-id -i ~/.ssh/id_rsa.pub root@newhost.example.com . ... View more

@Michael Bronson No if the AMS is running fine for now then we do not need to restart it. We can keep monitoring it for some time to see if everything is going fine. ... View more