Support Questions
Find answers, ask questions, and share your expertise

Sequence of the non-root Ambari and agent installations

Expert Contributor

Machines : (4 datanodes + 2 master(HA) + 1 management = 7 machines) . NO Internet, local repository only.

Target : Install Ambari 2.2 and using it, install HDP 2.4(the auto. install way)

As per this community thread, I have proceeded with a non-root but sudo-enabled user to install Ambari(server and agents) and HDP. In short, the Ambari agents need to be installed manually.

The steps that I have executed so far :

  1. On a 'management node' machine, logged in as a non-root but sudo-enabled user viz. ojoqcu
  2. Copied the repos. files and tarballs and put them under /etc/yum.repos and /var/www/html respectively
  3. Installed the ambari server but did NOT set it up, YET
  4. Using rysnc, copied the ambari.repo from the 'management node' to one of the 'worker node' where an ambari agent will be installed manually
  5. Attempted an ssh to another 'head node' but it asks for password
-bash-4.2$ whoami
ojoqcu
-bash-4.2$
-bash-4.2$
-bash-4.2$ ssh l4327pp.sss.com
ojoqcu@l4327pp.sss.com's password:
Last login: Wed Jun  8 11:30:06 2016 from rd0100170.global.scd.com
###############################################################
Welcome!

I have the following questions regarding the next steps :

  • During the ambari server set-up, I will provide the username 'ambari' during the 'Customize user account for ambari-server' step. I assume after the set-up completes, a new Linux user 'ambari' (which group?) will be created on the 'management node'. Now which user(ojoqcu or ambari or both) needs the password-less SSH ? Is the password-less SSH required even if the agents will be installed manually on all the nodes ?
  • As per the sudoer configuration documentation, I think the following entries need to be put in the /etc/sudoers file BUT will these entries be the same across all the nodes i.e same text in all the /etc/sudoers file ? When to put these entries ?
# Ambari Customizable Users
ambari ALL=(ALL) NOPASSWD:SETENV: /bin/su hdfs *,/bin/su ambari-qa *,/bin/su ranger *,/bin/su zookeeper *,/bin/su knox *,/bin/su falcon *,/bin/su ams *, /bin/su flume *,/bin/su hbase *,/bin/su spark *,/bin/su accumulo *,/bin/su hive *,/bin/su hcat *,/bin/su kafka *,/bin/su mapred *,/bin/su oozie *,/bin/su sqoop *,/bin/su storm *,/bin/su tez *,/bin/su atlas *,/bin/su yarn *,/bin/su kms *


# Ambari Non-Customizable Users
ambari ALL=(ALL) NOPASSWD:SETENV: /bin/su mysql *


# Ambari Commands
ambari ALL=(ALL) NOPASSWD:SETENV: /usr/bin/yum,/usr/bin/zypper,/usr/bin/apt-get, /bin/mkdir, /usr/bin/test, /bin/ln, /bin/chown, /bin/chmod, /bin/chgrp, /usr/sbin/groupadd, /usr/sbin/groupmod, /usr/sbin/useradd, /usr/sbin/usermod, /bin/cp, /usr/sbin/setenforce, /usr/bin/test, /usr/bin/stat, /bin/mv, /bin/sed, /bin/rm, /bin/kill, /bin/readlink, /usr/bin/pgrep, /bin/cat, /usr/bin/unzip, /bin/tar, /usr/bin/tee, /bin/touch, /usr/bin/hdp-select, /usr/bin/conf-select, /usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh, /usr/lib/hadoop/bin/hadoop-daemon.sh, /usr/lib/hadoop/sbin/hadoop-daemon.sh, /sbin/chkconfig gmond off, /sbin/chkconfig gmetad off, /etc/init.d/httpd *, /sbin/service hdp-gmetad start, /sbin/service hdp-gmond start, /usr/sbin/gmond, /usr/sbin/update-rc.d ganglia-monitor *, /usr/sbin/update-rc.d gmetad *, /etc/init.d/apache2 *, /usr/sbin/service hdp-gmond *, /usr/sbin/service hdp-gmetad *, /sbin/service mysqld *, /usr/bin/python2.6 /var/lib/ambari-agent/data/tmp/validateKnoxStatus.py *, /usr/hdp/current/knox-server/bin/knoxcli.sh *, /usr/bin/dpkg *, /bin/rpm *, /usr/sbin/hst *
# Ambari Ranger Commands
ambari ALL=(ALL) NOPASSWD:SETENV: /usr/hdp/*/ranger-usersync/setup.sh, /usr/bin/ranger-usersync-stop, /usr/bin/ranger-usersync-start, /usr/hdp/*/ranger-admin/setup.sh *, /usr/hdp/*/ranger-knox-plugin/disable-knox-plugin.sh *, /usr/hdp/*/ranger-storm-plugin/disable-storm-plugin.sh *, /usr/hdp/*/ranger-hbase-plugin/disable-hbase-plugin.sh *, /usr/hdp/*/ranger-hdfs-plugin/disable-hdfs-plugin.sh *,  /usr/hdp/current/ranger-admin/ranger_credential_helper.py, /usr/hdp/current/ranger-kms/ranger_credential_helper.py, /usr/hdp/*/ranger-*/ranger_credential_helper.py
  • Even on the 'worker nodes', I will be logging in as ojoqcu but as per the ambari-agent as non-root doc., I will set the 'run_as_user' as 'ambari' but this LINUX user will not be pre-existing on the 'worker node's - am I misunderstanding some thing here ?
  • Since I don't have the root access, what exactly I need to ask the Linux admin to do ?

How shall I proceed, step-wise ?

3 REPLIES 3

Hi @Kaliyug Antagonist.

1. Your customized ambari account will be part of the hadoop user group. All servers in the cluster should be configured for password-less SSH regardless of how the accounts are created.

2. You will need to update sudoers for each node running the ambari-agent. These tends to be all your data (worker) nodes.

3. If you have customized the agent account to run as ojoqcu then you will want to set the 'run-as-user' to the ojoqcu account.

Some admins want to pre-create all the accounts and groups. I've attached the /etc/passwd and /etc/group files from our sandbox. Also, here is a list of all the users and their corresponding groups. These accounts are customizable. If you customize a service account, you will need to configure the service to use the custom account during the initial install. passwd.txt

group.txt

USER GROUP
ambari hadoop
hdfs hadoop
ambari-qa hadoop
ranger hadoop, ranger
zookeeper hadoop
knox hadoop, knox
falcon hadoop
ams hadoop
flume hadoop
hbase hadoop
spark hadoop, spark
accumulo hadoop
hive hadoop
hcat hadoop
kafka hadoop
mapred hadoop
oozie hadoop
sqoop hadoop
storm hadoop
tez hadoop
atlas hadoop, atlas
yarn hadoop
kms hadoop, kms

@Scott Shaw - Does cluster creation take more time if ambari is configured for non root user ?

Cluster creation with ambari configured for root took 21 minutes to install and 15 minutes to start all services.
Cluster creation with ambari configured for non root took 29 minutes to install and 25 minutes to start all services.

Post cluster creation i see 'stop all' and 'start all' services is also taking a lil more time non root user clusters.

From the logs i see that

/var/lib/ambari-agent/tmp/changeUid.sh and ambari-python-wrap executions are taking a few more milliseconds when compared with the cluster with root user.

@Kaliyug Antagonist

pls find answer inline -

  • During the ambari server set-up, I will provide the username 'ambari' during the 'Customize user account for ambari-server' step. I assume after the set-up completes, a new Linux user 'ambari' (which group?) will be created on the 'management node'. Now which user(ojoqcu or ambari or both) needs the password-less SSH ? Is the password-less SSH required even the agents will be installed manually on all the nodes ?

-->

If you provide custom name [say - test-ambari ] while executing "ambari-server setup" then it will create linux user with name "test-ambari" and group with name "test-ambari"

If you have already install agent manually on nodes then from ambari UI you need to check "Performmanual registrationon hosts and do not use SSH", here it will not ask you for user/password.

  • As per the sudoer configuration documentation, I think the following entries need to be put in the /etc/sudoers file BUT will these entries be the same across all the nodes i.e same text in all the /etc/sudoers file

--> If you are using non root user for ambari agent also then you need to do the sudoers on agent nodes also.

You need to reproduce this scenario on your local cluster and gather the steps to ask to your linux admin.