Created 04-12-2017 03:30 PM
Hello Team,
We have development HDP 2.5.3 cluster consisting of 4 nodes and we are adding another node from ambari. After adding the node successfully in the cluster we are failing to install any services in that particular newly added node. Below are the logs from ambari during the failure while adding DataNode in the new node.
Looking for your guidance on how to fix this issue.
stderr: /var/lib/ambari-agent/data/errors-2053.txt
Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py", line 35, in <module> BeforeAnyHook().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute method(env) File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py", line 26, in hook import params File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py", line 191, in <module> hadoop_conf_dir = conf_select.get_hadoop_conf_dir(force_latest_on_upgrade=True) File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/conf_select.py", line 477, in get_hadoop_conf_dir select(stack_name, "hadoop", version) File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/conf_select.py", line 315, in select shell.checked_call(_get_cmd("set-conf-dir", package, version), logoutput=False, quiet=False, sudo=True) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 71, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 93, in checked_call tries=tries, try_sleep=try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 141, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 294, in _call raise Fail(err_msg) resource_management.core.exceptions.Fail: Execution of 'ambari-python-wrap /usr/bin/conf-select set-conf-dir --package hadoop --stack-version 2.5.3.0-37 --conf-version 0' returned 1. 2.5.3.0-37 Incorrect stack version Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-2053.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-2053.json', 'INFO', '/var/lib/ambari-agent/tmp'] stdout: /var/lib/ambari-agent/data/output-2053.txt 2017-04-12 19:51:03,007 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.5.3.0-37 2017-04-12 19:51:03,007 - Checking if need to create versioned conf dir /etc/hadoop/2.5.3.0-37/0 2017-04-12 19:51:03,007 - call[('ambari-python-wrap', '/usr/bin/conf-select', 'create-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.3.0-37', '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1} 2017-04-12 19:51:03,030 - call returned (1, '2.5.3.0-37 Incorrect stack version', '') 2017-04-12 19:51:03,031 - checked_call[('ambari-python-wrap', '/usr/bin/conf-select', 'set-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.3.0-37', '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False} Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-2053.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-2053.json', 'INFO', '/var/lib/ambari-agent/tmp'] Command failed after 1 tries
Created 04-13-2017 08:09 AM
Yes, we had 2.3.4 installed in this node. However we have taken the following steps and its working fine now.
1) Deleted 2.3.4 and current folders under /usr/hdp
2) Restarted the ambari-agent
3) Took care of the issues found in pre run checks (like deleting old rpms, folders and users, etc.)
4) Added the new host. However it required python-argparse rpm
5) Added Data Nodes, Node Managers and other services and they are working fine now.
Created 04-13-2017 08:09 AM
Yes, we had 2.3.4 installed in this node. However we have taken the following steps and its working fine now.
1) Deleted 2.3.4 and current folders under /usr/hdp
2) Restarted the ambari-agent
3) Took care of the issues found in pre run checks (like deleting old rpms, folders and users, etc.)
4) Added the new host. However it required python-argparse rpm
5) Added Data Nodes, Node Managers and other services and they are working fine now.