Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Install HDP and HDF ambari clusters with non-root user

Install HDP and HDF ambari clusters with non-root user

New Contributor

I'm trying to install HDP and HDF ambaris clusters with non-root user. Please let me know the steps where i need to replace root user with non-root user in ambari scripts.

Thanks in advance.

8 REPLIES 8

Re: Install HDP and HDF ambari clusters with non-root user

Super Mentor

@Raj K

The following docs explains how to run the

Ambari Server as Non Root User: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.0/bk_ambari-security/content/how_to_configure...

Similarly Running Ambari Agent as Non Root User: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.0/bk_ambari-security/content/how_to_configure...

.

After following the mentioned docs your ambari cluster will run with non root privileges.

Re: Install HDP and HDF ambari clusters with non-root user

New Contributor

Thanks Jay.

I'm able to test HDF cluster with ambari user. But i only installed ZK service not able to installl Ambari-infra,Ambari-metrics and other services.

Below are the error from Ambari-infra log:

Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDF/2.0/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-31.json', '/var/lib/ambari-agent/cache/stacks/HDF/2.0/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-31.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1', '']
2018-06-01 16:18:47,210 - The repository with version 3.1.1.0-35 for this command has been marked as resolved. It will be used to report the version of the component which was installed
2018-06-01 16:18:47,211 - Skipping stack-select on AMBARI_INFRA because it does not exist in the stack-select package structure.

Re: Install HDP and HDF ambari clusters with non-root user

Super Mentor
@Raj K

can you please share the complete output from ambari UI ... including errors / output txt.

Re: Install HDP and HDF ambari clusters with non-root user

New Contributor
ERROR logs:
stderr:   /var/lib/ambari-agent/data/errors-57.txt
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/HDF/2.0/hooks/before-ANY/scripts/hook.py", line 35, in <module>
    BeforeAnyHook().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 375, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDF/2.0/hooks/before-ANY/scripts/hook.py", line 29, in hook
    setup_users()
  File "/var/lib/ambari-agent/cache/stacks/HDF/2.0/hooks/before-ANY/scripts/shared_initialization.py", line 51, in setup_users
    fetch_nonlocal_groups = params.fetch_nonlocal_groups,
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/accounts.py", line 84, in action_create
    shell.checked_call(command, sudo=True)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'useradd -m -u 1006 -G hadoop -g hadoop ams' returned 4. useradd: UID 1006 is not unique
Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDF/2.0/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-57.json', '/var/lib/ambari-agent/cache/stacks/HDF/2.0/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-57.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1', '']stdout:   /var/lib/ambari-agent/data/output-57.txt
2018-06-01 16:58:48,391 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=None -> 3.1
User Group mapping (user_group) is missing in the hostLevelParams
2018-06-01 16:58:48,395 - Group['hadoop'] {}
2018-06-01 16:58:48,397 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2018-06-01 16:58:48,398 - call['/var/lib/ambari-agent/tmp/changeUid.sh zookeeper'] {}
2018-06-01 16:58:48,407 - call returned (0, '1004')
2018-06-01 16:58:48,408 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': 1004}
2018-06-01 16:59:00,098 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2018-06-01 16:59:00,100 - call['/var/lib/ambari-agent/tmp/changeUid.sh ams'] {}
2018-06-01 16:59:00,122 - call returned (0, '1006')
2018-06-01 16:59:00,123 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': 1006}
2018-06-01 16:59:00,123 - Adding user User['ams']
Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDF/2.0/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-57.json', '/var/lib/ambari-agent/cache/stacks/HDF/2.0/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-57.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1', '']
2018-06-01 16:59:00,156 - The repository with version 3.1.1.0-35 for this command has been marked as resolved. It will be used to report the version of the component which was installed
2018-06-01 16:59:00,158 - Skipping stack-select on AMBARI_METRICS because it does not exist in the stack-select package structure.

Command failed after 1 tries<br>

Re: Install HDP and HDF ambari clusters with non-root user

Super Mentor

@Raj K

The failure is because Ambari is not able to add user due to duplicate UID 1006 which is reported as an issue

resource_management.core.exceptions.ExecutionFailed: Execution of 'useradd -m -u 1006 -G hadoop -g hadoop ams' returned 4. useradd: UID 1006 is not unique

.

Can you please run the following command on the failed machine? to see if UID 1006 already exist in your /etc/passwd

# awk -F: '($3 >= 1000) {printf "%s:%s\n",$1,$3}' /etc/passwd

.

Which ambari version are you using? I am suspecting that time issue might be related to :

https://issues.apache.org/jira/browse/AMBARI-21483

Re: Install HDP and HDF ambari clusters with non-root user

Super Mentor

@Raj K

there is a function "find_available_uid" in "changeToSecureUid.sh" script which looks for UIDs in the range of 1001 to 2000.

    function find_available_uid() {
     for ((i=1001; i<=2000; i++))
     do
       grep -q $i /etc/passwd
       if [ "$?" -ne 0 ]
       then
        newUid=$i
        break
       fi
     done
    }


For workaround to resolve this issue: (Better to use latest Ambari) or at leaset Ambari 2.6
You can change the UID range in above for loop(after making sure that range is clean and no UID is taken by LDAP user) and distribute this script on all the ambari agents at "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/files/changeToSecureUid.sh"

Highlighted

Re: Install HDP and HDF ambari clusters with non-root user

New Contributor

@Jay Kumar SenSharma

Thanks Jay,

It looks like the userid 1006 using local adsuser account. what is the range should i add in script file ? I'm using Ambari-2.6.1 version.

Re: Install HDP and HDF ambari clusters with non-root user

Super Mentor

@Raj K

Looks like you have removed the StackTrace from the originally asked query.

If for the UID issue which was asked originally to this threads is resolved, and your query/issue then please mark this HCC thread as answered by clicking on "Accept" link on the correct answer, That way it will help other HCC users to quickly find the answers.

.

For the later queries like below message please ignore them as AMBARI_INFRA does not come from HDP repo hence "stack-select" will not be able to determine it's package structure.

Skipping stack-select on AMBARI_INFRA because it does not exist in the stack-select package structure.<br>

.

About the Metrics Issue please open a separate HCC thread.

.