HDP Service installation fails - ValueError: invalid literal for int() with base 10

[#1]

SYMPTOM

While adding new service to the HDP cluster, it fails with below error:

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py", line 37, in <module>
    AfterInstallHook().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py", line 31, in hook
    setup_hdp_symlinks()
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py", line 49, in setup_hdp_symlinks
    hdp_select.select_all(version)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/hdp_select.py", line 122, in select_all
    Execute(command, only_if = only_if_command)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 238, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
    tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
    raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'ambari-sudo.sh /usr/bin/hdp-select set all `ambari-python-wrap /usr/bin/hdp-select versions | grep ^2.4.2.0-258 | tail -1`' returned 1. Traceback (most recent call last):
  File "/usr/bin/hdp-select", line 382, in <module>
    printVersions()
  File "/usr/bin/hdp-select", line 239, in printVersions
    result[tuple(map(int, versionRegex.split(f)))] = f
ValueError: invalid literal for int() with base 10: 'smartsense'
ERROR: set command takes 2 parameters, instead of 1
usage: hdp-select [-h] [<command>] [<package>] [<version>]
Set the selected version of HDP.
positional arguments:
  <command>   One of set, status, versions, or packages
  <package>   the package name to set
  <version>   the HDP version to set
optional arguments:
  -h, --help  show this help message and exit
  -r, --rpm-mode  if true checks if there is symlink exists and creates the symlink if it doesn't
Commands:
  set      : set the package to a specified version
  status   : show the version of the package
  versions : show the currently installed versions
  packages : show the individual package names

.

ROOT CAUSE

There is an extra unwanted directory under /usr/hdp on Ambari server

Please see below output:

[root@prodnode1 hdp]# ls -lrt /usr/hdp/
total 16
drwxr-xr-x. 19 root root 4096 Sep 26 08:31 2.4.2.0-258
drwxr-xr-x.  2 root root 4096 Nov 21 13:28 current
drwxr-xr-x.  2 root root 4096 Jan  2 15:37 smartsense
drwxr-xr-x.  3 root root 4096 Jan  2 15:39 share
[root@prodnode1 hdp]#

Note - It's not recommended to put any directory under /usr/hdp/ except 'share', 'current', 'versioned directory'

.

RESOLUTION

Move /usr/hdp/<unwanted-directory> to some other location than '/usr/hdp'

See below output:

#Before moving directory

[root@prodnode1 hdp]# hdp-select versions
Traceback (most recent call last):
  File "/usr/bin/hdp-select", line 382, in <module>
    printVersions()
  File "/usr/bin/hdp-select", line 239, in printVersions
    result[tuple(map(int, versionRegex.split(f)))] = f
ValueError: invalid literal for int() with base 10: 'smartsense'
[root@prodnode1 hdp]#

#Move directory outside

[root@prodnode1 hdp]# mv /usr/hdp/smartsense/ /root/
[root@prodnode1 hdp]#

#After moving directory

[root@prodnode1 hdp]# hdp-select versions
2.4.2.0-258

.

Please comment if you have any question. Happy Hadooping!! 🙂

[#2]

Do you know /usr/hdp/smartsense directory is created by the service or manually created?