Problems with /usr/bin/hdp-select versions

[#1]

I consistently get this hdp-seletct error in Ambari whenever I try to deploy any service. Can anyone help?

[root@hwmn1 yum.repos.d]# cat /var/lib/ambari-agent/data/errors-593.txt Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py", line 37, in <module> AfterInstallHook().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 314, in execute method(env) File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py", line 31, in hook setup_stack_symlinks() File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py", line 52, in setup_stack_symlinks stack_select.select_all(version) File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/stack_select.py", line 135, in select_all Execute(command, only_if = only_if_command) File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'ambari-sudo.sh /usr/bin/hdp-select set all `ambari-python-wrap /usr/bin/hdp-select versions | grep ^2.6 | tail -1`' returned 1. ERROR: set command takes 2 parameters, instead of 1

usage: distro-select [-h] [<command>] [<package>] [<version>]

Set the selected version of HDP.

positional arguments: <command> One of set, status, versions, or packages <package> the package name to set <version> the HDP version to set

optional arguments: -h, --help show this help message and exit -r, --rpm-mode if true checks if there is symlink exists and creates the symlink if it doesn't

Commands: set : set the package to a specified version status : show the version of the package versions : show the currently installed versions packages : show the individual package names [root@hwmn1 yum.repos.d]# [root@hwmn1 yum.repos.d]# [root@hwmn1 yum.repos.d]# cat /var/lib/ambari-agent/data/errors-593.txt Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py", line 37, in <module> AfterInstallHook().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 314, in execute method(env) File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py", line 31, in hook setup_stack_symlinks() File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py", line 52, in setup_stack_symlinks stack_select.select_all(version) File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/stack_select.py", line 135, in select_all Execute(command, only_if = only_if_command) File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'ambari-sudo.sh /usr/bin/hdp-select set all `ambari-python-wrap /usr/bin/hdp-select versions | grep ^2.6 | tail -1`' returned 1. ERROR: set command takes 2 parameters, instead of 1

usage: distro-select [-h] [<command>] [<package>] [<version>]

Set the selected version of HDP.

positional arguments: <command> One of set, status, versions, or packages <package> the package name to set <version> the HDP version to set

optional arguments: -h, --help show this help message and exit -r, --rpm-mode if true checks if there is symlink exists and creates the symlink if it doesn't

Commands: set : set the package to a specified version status : show the version of the package versions : show the currently installed versions packages : show the individual package names [root@hwmn1 yum.repos.d]#

[#2]

I ran "/usr/bin/hdp-select versions" and it failed. I realized hadoop was not suppose to be in /usr/hdp directory. I deleted it and re-ran "/usr/bin/hdp-select versions" after removing the hadoop directory from /usr/hdp. I listed the directory again after the deploy failed again and noticed the hadoop directory had been recreated. Not sure why hadoop directory is beiing recreated in /usr/hdp during deploy.

Here is the sequence of commands.

[root@hwmn1 yum.repos.d]# /usr/bin/hdp-select versions

ERROR: Unexpected file/directory found in /usr/hdp: hadoop

[root@hwmn1 yum.repos.d]# cd /usr/hdp

[root@hwmn1 hdp]# ls -ltr

total 16 drwxr-xr-x 3 root root 4096 May 23 13:00 share

drwxr-xr-x 22 root root 4096 May 23 14:31 2.6.0.3-8

drwxr-xr-x 2 root root 4096 May 23 14:31 current

drwxr-xr-x 3 root root 4096 May 23 14:31 hadoop

[root@hwmn1 hdp]# rm -fR hadoop

[root@hwmn1 hdp]# ls -ltr

total 12 drwxr-xr-x 3 root root 4096 May 23 13:00 share

drwxr-xr-x 22 root root 4096 May 23 14:31 2.6.0.3-8

drwxr-xr-x 2 root root 4096 May 23 14:31 current

[root@hwmn1 hdp]# ls -ltr

total 16 drwxr-xr-x 3 root root 4096 May 23 13:00 share

drwxr-xr-x 22 root root 4096 May 23 14:31 2.6.0.3-8

drwxr-xr-x 2 root root 4096 May 23 14:35 current

drwxr-xr-x 3 root root 4096 May 23 14:35 hadoop

[root@hwmn1 hdp]# pwd

/usr/hdp

[root@hwmn1 hdp]# /usr/bin/hdp-select versions

ERROR: Unexpected file/directory found in /usr/hdp: hadoop

[root@hwmn1 hdp]#

[#3]

Yes, during deploy it will re-create following directory /usr/hdp/hadoop/.

from where you trying to deploy HDP in host machine...like thru ambari url or manual process ?

Have you setup correctly repo files ?

output :

yum repolist

[#4]

@Michael Allen

The error that you are getting indicates that you might be having some additional directories inside the "/usr/hdp" directory. Please remove any other unwanted directory from the mentioned directory.

[root@hwmn1 yum.repos.d]# /usr/bin/hdp-select versions 
ERROR: Unexpected file/directory found in /usr/hdp: hadoop

.

I suspect that you have "/usr/hdp/hadoop" directory added on your own which seems to be causing the issue here. Please try moving it to some other place and then try again.

.

Example: At my end i see the structure of the mentioned directory as following:

# cd /usr/hdp/
# ls -lart
total 24

drwxr-xr-x  1 root root 4096 Apr 19 19:11 share
drwxr-xr-x  1 root root 4096 Apr 19 18:26 ..
drwxr-xr-x 32 root root 4096 Apr 19 19:13 2.5.0.0-1245
drwxr-xr-x  1 root root 4096 Apr 19 19:13 2.6.0.3-8
drwxr-xr-x  1 root root 4096 May 13 09:40 current

.

Can you please share the "hdp-select" RPM version that you have on your host? You might be having an incorrect version of hdp-select i guess.

# rpm -qa | grep hdp-select
hdp-select-2.6.0.3-8.el6.noarch

.

Also the output of the repository.

# cat /etc/yum.repos.d/HDP.repo 

[#5 · Accepted solution]

@Michael Allen

Also initially we see that you are getting this error:

'ambari-sudo.sh /usr/bin/hdp-select set all `ambari-python-wrap /usr/bin/hdp-select versions | grep ^2.6 | tail -1`' returned 1. ERROR: set command takes 2 parameters, instead of 1

Above error can occur only when the second argument after the "all" is null.

For example: (If we do not pass the second argument then it will fail as following)

[root@sandbox hdp]# /usr/bin/hdp-select set all 
ERROR: set command takes 2 parameters, instead of 1

- So in your case i am suspecting that the second argument is becoming null (empty) In order to verify the same can you please check what is the output of the following command you are getting?

[root@sandbox bin]# echo `ambari-python-wrap /usr/bin/hdp-select versions | grep ^2.6 | tail -1`
2.6.0.3-8

- If you have HDP 2.6 repo properly configured and if your "/usr/hdp" directory has the correct contents including the "2.6.0.3-8" directory then it should return the result else the result might be empty. Example:

[root@sandbox bin]# echo `ambari-python-wrap /usr/bin/hdp-select versions | grep ^2.6 | tail -1`
2.6.0.3-8

[root@sandbox bin]# echo `ambari-python-wrap /usr/bin/hdp-select versions | grep ^2.7 | tail -1`

[root@sandbox bin]# ls -l /usr/hdp/
total 16
drwxr-xr-x 32 root root 4096 Apr 19 19:13 2.5.0.0-1245
drwxr-xr-x  1 root root 4096 Apr 19 19:13 2.6.0.3-8
drwxr-xr-x  1 root root 4096 May 24 04:09 current
drwxr-xr-x  1 root root 4096 Apr 19 19:11 share

.

So please check if your "/usr/hdp" directory is containing the "2.6.0.3-8" directory or not else the second argument of the hdp-select set all <null> command will show the same error.

.

[#6]

Thank you all for your suggestions. I did clear up any items you may have mentioned and still had the problem. I resolved this issue with hdp-select by re-provisioning all of the servers in the cluster and reinstalling Ambari. This particular item is closed.

[#7]

Make sure that you have installed this RPMS on all node.

libtirpc-devel-0.2.4-0.15.el7.x86_64

libtirpc-0.2.4-0.15.el7.x86_64