Ambari 2.1.2 : cannot start newly added datanode because /usr/hdp/current link is missing

[#1]

Hi,

I have an issue at installation time of datanode (Ambari 2.1.2, HDP 2.2.4) , it fails because at step "Datanode start" with the error:

""

  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 87, in action_create
    raise Fail("Applying %s failed, parent directory %s doesn't exist" % (self.resource, dirname))
resource_management.core.exceptions.Fail: Applying File['/usr/hdp/current/hadoop-client/conf/hadoop-policy.xml'] failed, parent directory /usr/hdp/current/hadoop-client/conf doesn't exist

""

Obviously the cause of this issue already happens at step "Datanode install", because the last log message is:

""

...

2016-01-12 19:44:17,233 - Skipping XmlConfig['core-site.xml'] due to only_if
2016-01-12 19:44:17,233 - Can only link configs for HDP-2.3 and higher.

""

Hmmm, strange, HDP2.2 should be supported by recent versions of Ambari...

Is this a known issue? any workarounds, hints what to do ?

Thanks, Gerd

[#2]

@Gerd Koenig

I would use compatiable ambari version with HDP and then upgrade ambari to save time.

[#3]

Hi @Neeraj Sabharwal,

thanks for getting back that quickly.

Referring to page: http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.2.1/bk_Installing_HDP_AMB/content/_determine_sta...

I am assuming that Ambari 2.1.2 is supporting HDP 2.2.4 ?!?!

Which version should I use instead ?

[#4]

@Gerd Koenig You are right on the compatibility. Is it new install? or cleaning up existing install then installing new version ?

[#5]

@Gerd Koenig

whats the output of following?

ls -l /usr/hdp

[#6]

@Neeraj Sabharwal

ok, tried doing it again after removing any hadoop_* packages on that node from a previous run, but again:

...

2016-01-12 20:29:26,813 - Skipping XmlConfig['core-site.xml'] due to only_if
2016-01-12 20:29:26,814 - Can only link configs for HDP-2.3 and higher.

and then:

$ ls -al /usr/hdp
total 12
drwxr-xr-x  3 root root 4096 Jan 12 20:28 .
drwxr-xr-x 16 root root 4096 Jan 12 20:28 ..
drwxr-xr-x 13 root root 4096 Jan 12 20:29 2.2.4.2-2

hence, the link 'current' is missing...thereby the datanode fails at starting up

[#7]

Maybe I will start completely from scratch with a fresh installation.

Which Ambari version would you recommend for setting up a fresh HDP 2.2.4 ? Yes, unfortunately I have to use that HDP version, an upgrade is planned for later this year 😉

[#8]

@Gerd Koenig Let's follow the correct approach to cleanup

https://community.hortonworks.com/content/idea/138/cleanup-scripts-to-remove-old-hdp-bits.html

[#9]

Hi Gerd, can you check the following,

rpm -qa | grep hadoop  # make sure it is installed and that /etc/hadoop/conf exists

ls -la /usr/hdp/current/hadoop-client  # should be a symlink to /etc/hadoop/conf, if not, create it

[#10]

Hi @Alejandro Fernandez,

thanks for your answer, but it is not exactly how it should look like 😉 (maybe you are referring to an older version, before adding the version number into the installation directory tree).

Currently, the linking should be:

/usr/hdp/current -> /usr/hdp/2.2.4.2-2

and

/usr/hdp/2.2.4.2-2/hadoop/conf -> /etc/hadoop/conf